Arithogeometric mean of infinity inconsistencies

[d-torrance]

We currently compute the agm of two numbers in three distinct ways:

• RR: using MPFR
• CC: using our own function in gmp.d (and I'm proposing we use FLINT in Organizing the e directory #4347)
• RRi/CCi: using FLINT

This results in inconsistent output when one of the arguments is infinite:

i1 : agmTests = f -> {
    agm(f numeric infinity, f 0),
    agm(f numeric infinity, f 2),
    agm(f 0, f numeric infinity),
    agm(f 2, f numeric infinity)}

                    
o1 = agmTests

o1 : FunctionClosure

i2 : agmTests toRR

o2 = {NotANumber, infinity, NotANumber, infinity}

o2 : List

i3 : agmTests toCC

o3 = {infinity, infinity, infinity, infinity}

o3 : List

i4 : agmTests toRRi

o4 = {[NotANumber,NotANumber] (an empty interval), [NotANumber,NotANumber] (an empty interval), [NotANumber,NotANumber] (an empty interval), [NotANumber,NotANumber] (an empty interval)}

o4 : List

i5 : agmTests toCCi

o5 = {[NotANumber,NotANumber]+[NotANumber,NotANumber]*ii (an empty interval), [NotANumber,NotANumber]+[NotANumber,NotANumber]*ii (an empty interval), [NotANumber,NotANumber]+[NotANumber,NotANumber]*ii (an empty interval),
     -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
     [NotANumber,NotANumber]+[NotANumber,NotANumber]*ii (an empty interval)}

o5 : List

Maybe it makes sense for different behavior between the real and complex cases, but certainly it seems like we'd want RR and RRi to agree and to CC and CCi to agree.

I asked Claude to classify what it thinks the output should be:

Case	agm(∞, 2)	agm(∞, 0)
Real	+∞	NaN
Complex	NaN	NaN