I'm still trying to understand why this code is necessary.

Is it only necessary when floats are double precision? Or also single precision?

Can you give an example of a number that needs this additional digit of precision?

The numbers that require this extra digit are not the the same for single-precision and double-precision (as the rounding effects differs), but a typical example for double-precision is 2.0 ** 100, which is the test case I have added to float_parse_doubleprec.py. Without the improved repr code, numbers which appear to be the same as per repr() would actually not match with regards to == operator:

>>> n = float('2.0')**100
>>> n2 = float(repr(n))
>>> n
1.267650600228229e+30
>>> n2
1.267650600228229e+30
>>> n2 == n
False

With the improved repr code, we get the expected behaviour:

>>> n = float('2.0')**100
>>> n2 = float(repr(n))
>>> n
1.2676506002282294e+30
>>> n2
1.2676506002282294e+30
>>> n2 == n
True

With float folding enabled, large expanded decimal numbers become more frequent in mpy files and this could quickly cause functional differences.

Without this improved repr code, the coverage tests fail in float_float2int_intbig due to the missing half-digit.

OK, thanks for the info. I now understand the problem. Let me restate it in two ways:

1. Printing a double with 16 digits of precision is not enough to represent every number. 17 digits is a little bit more than needed.
2. There exist multiple different numbers whose repr at 16 digits is the same. At 17 digits they all differ.

MicroPython at the moment uses 16 digits which is not enough to faithfully represent doubles.

So we do need to fix something here, but it's not clear what or how. My testing shows that in CPython doing '{:.17g}'.format(n) is enough digits to properly represent every number. Ie:

n == float('{:.17g}'.format(n))

The issue is that in MicroPython the above is not true. In fact there's no repr precision in MicroPython that allows you to satisfy the above for all n (eg even .20g doesn't work).

For some numbers, eg 39.0**100, the above is true in MicroPython for precision of 17 and 19 but false for 16, 18 and 20!

Numbers like 12.0**100 and 18.0**100 get worse going from 16 to 17 digits (in MicroPython).

(And others like 40.0**100 are accurate at 17, but at 16 are printed only with 15, and so the length increases by 2 digits to get full accuracy.)

The real issue here is that MicroPython's mp_format_float() code is inaccurate. Eg:

$ python -c "print(('{:.22g}'.format(2.0**100)))" 
1.267650600228229401497e+30
$ micropython -c "print(('{:.22g}'.format(2.0**100)))"
1.267650600228229428959e+30

That's a problem! It means we can't really use mp_format_float() to store floats in .mpy files, because it's inaccurate.

What to do?

There is #12008 which stores floats as binary in .mpy files. That's definitely one way to fix it, but I don't want to change the .mpy version number at this stage.

We could try to improve mp_format_float(), but I think that's very difficult, and probably needs a rewrite to make it more accurate.

Note that currently (without this PR) certain floats are already stored inaccurately in .mpy files, due to using 16 digit precision only. So does this PR make things worse? I think yes, because now more floats have the chance to be store and hence stored inaccurately. Eg those in tests/float/float2int_intbig.py.

That said, I don't think this PR (constant folding) will make things that much worse in .mpy files.

My suggestion to move forward is:

1. Switch to using 17 digits of precision for all repr printing (and I guess 7 or 8 for single precision). And don't worry about things like 1.2345 rendering as 1.23499999999. I think if you use repr(a_float) then you want accuracy. If you want a set number of digits, you'd be using a precision specifier like '{:.10g}'.format(a_float). (It's pretty expensive to call mp_format_float() so I'd rather not do it twice.)
2. Accept that floats in .mpy files are possibly inaccurate. Users can use array.array with binary representation, or escape the const folding with eg float("2.0") + 3.4 if they need 100% accuracy.
3. In the future, try to improve things.

If you don't like (1) because it makes 1.2345 print out as 1.234999999 at the REPL, then maybe we can adjust the code in py/persistentcode.c:save_obj to save floats explicitly with a higher precision (17 digits for double). (From my testing, always using 17 digits is slightly better than using 17 only if it grows 16 by one digit.)

(I didn't investigate single precision floats, but I assume all the above logic/reasoning applies there as well.)

Your understanding is correct, and indeed, single precision runs into the exact same kind of issues.

As you have guessed, I am not a fan of 1.234499999999, but if your testing shows that using 17 digits provides overall better results than conditionally shorting to 16, then this is the way to go.

I made a test:

import os, random, array, math
stats = [0, 0, 0, 0, 0]   
N = 10000000
for _ in range(N):
    #f = random.random()       
    f = array.array("d", os.urandom(8))[0]
    while f == math.isinf(f) or math.isnan(f):
        f = array.array("d", os.urandom(8))[0]
    str_f_repr = repr(f)           
    str_f_16g = "{:.16g}".format(f)                
    str_f_17g = "{:.17g}".format(f)
    str_f_18g = "{:.18g}".format(f)                                       
    str_f_19g = "{:.19g}".format(f)                                        
    f_repr = float(str_f_repr)                                      
    f_16g = float(str_f_16g)                                            
    f_17g = float(str_f_17g)
    f_18g = float(str_f_18g)
    f_19g = float(str_f_19g)
    stats[0] += f_repr == f                                           
    stats[1] += f_16g == f                                        
    stats[2] += f_17g == f                                              
    stats[3] += f_18g == f                                        
    stats[4] += f_19g == f             
print(list(100 * s / N for s in stats))

That generates 10 million random doubles and checks repr and formatting with 16 through 19 digits of precision, to see if the rendered number is equivalent to the original.

On Python it gives:

[100.0, 54.61923, 100.0, 100.0, 100.0]

So that means 100% of the numbers are accurate using repr and 17 or more digits of precision. Using 16 digits, only 54.6% represent correctly.

MicroPython unix port with this PR (ie repr using 16 or 17 digits) gives:

[52.24606, 37.99564, 53.94176, 54.37388, 54.12582]

That shows ... it's pretty terrible. But at least we see that using repr with this PR is much better than master which just uses 16 digits: 52.2% for repr vs 38% for always using 16 digits. But also, always using 17 digits gives a little more accuracy than repr, in this case 53.9% which is about 1.7% more than repr.

Note that MicroPython may also have inaccuracies converting str to float, so that may add to the errors here. But that's representative of .mpy loading because the loader uses the same float parsing code.

Also note that MicroPython was faster than CPython in the above test! That might be because MicroPython's float printing code is taking shortcuts. It also means that rendering the float twice in repr to try and extract the extra half digit is not taking up that much extra time.

Open questions:

• is it worth the extra 1% or so accuracy always showing 17 digits, or is it better to show a nicer number and doing the 16/17 trick?
• is it worth trying to improve mp_float_format() using a different algorithm, eg fully integer based?

I tried extending the 16/17 trick to the following:

--- a/py/objfloat.c
+++ b/py/objfloat.c
@@ -133,7 +133,7 @@ static void float_print(const mp_print_t *print, mp_obj_t o_in, mp_print_kind_t
         // digging too far (eg. 1.234499999999), so we keep the short form
         char ebuf[32];
         mp_format_float(o_val, ebuf, sizeof(ebuf), 'g', precision + 1, '\0');
-        if (strlen(ebuf) == strlen(buf) + 1) {
+        if (strlen(ebuf) - strlen(buf) <= 3) {
             memcpy(buf, ebuf, sizeof(buf));
         }
     }

That uses 17 digits if it extends up to 3 digits more. That gives better results in the above test:

[53.85271, 37.94894, 53.94477, 54.36183, 54.16032]

That means repr is almost as good as unconditionally doing 17 digits, 53.85% vs 53.94%. But note that these percentages change a bit across runs, due to the randomness.

I have been working on this today. It is now clear to me as well that the proper solution is indeed to fix mp_float_format(), rather than adding a quick fix in repr.

I have implemented today an alternate algorithm for mp_float_format(), based on the same idea as my previous enhancement to mp_parse_num_float: using an mp_float_uint_t to convert the mantissa, rather than working directly on floats. This brings the correct percentage up to 60%, which is a good start.

Once the mantissa is computed using an integer, it is possible fix it incrementally to ensure that the parse algorithm will get back to the original number. I have been working on that, and got up to 95% correct conversions using 18 digits. I still have a bit of work to fix corner cases (forced round-ups) before I can push that code.

Do you want me to send the new mp_float_format code in a separate PR ?

I have implemented today an alternate algorithm for mp_float_format(), based on the same idea as my previous enhancement to mp_parse_num_float: using an mp_float_uint_t to convert the mantissa, rather than working directly on floats.

Wow, very nice!

Do you want me to send the new mp_float_format code in a separate PR ?

Yes please.

This nlr_push is relatively expensive.

What are the cases in which it's needed? From what I can gather it's:

• divide by zero (both int and float)
• when complex numbers are produced without complex enabled (float only)
• negative shift (int only)
• negative power without float enabled (int only)
• unsupported operation (eg shifting a float)

The int cases are already handled explicitly. Maybe it's not much extra code to explicitly guard against the invalid float operations, so that this nlr_push protection is not needed?

It's a bit of a trade-off here, whether to use nlr_push and let mp_binary_op dictate what's valid and what isn't, or to make the code in the parser here know what's valid.

I thought about that initially, but I was afraid to miss a corner case which would then break compilation. For instance, on builds with limited integers (longlong), an exception might be raised if the result of an operation is beyond the supported longlong range. I was afraid that going into such detailled tests somehow breaks the modularity of MicroPython, hence my preference for catching the exception.

Are you mostly concerned by the impact on stack space, or more on compilation time ?