Skip to content

Commit fb265b8

Browse files
committed
py/formatfloat: Improve accuracy of float formatting code.
Following discussions in PR #16666, this commit updates the float formatting code to reduce the `repr` reversibility error, i.e. the percentage of valid floating point numbers that do not parse back to the same number when formatted by `repr`. The baseline before this commit is an error rate of 46%, when using double-precision floats. This new code initially brings the error down to 0.17%, by using an integer representation of the decimal mantissa rather than working on floats, followed by incremental improvements. The incremental improvement code makes the conversion slightly slower than CPython when tested on ports/unix, but not much. It can be disabled via a mpconfig setting, MICROPY_FLOAT_HIGH_QUALITY_REPR. The residual error rate appears to be due to the parse code itself, which is unable to produce some specific floating point values, regardless of the string provided as input. When the number obtained by parsing the `repr` string is not the same as the original one, the maximum relative error is < 2.7e-16, which is ~2**-52. On single-precision floats, the new code brings the error down to 0.29%, with a maximum relative error under 1e-7, which is ~2**-23. For bare metal machines with very limited ressources, disabling MICROPY_FLOAT_HIGH_QUALITY_REPR would bring the error rate to 5.11%, with a maximum relative error of 1e-6. Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>
1 parent 5f058e9 commit fb265b8

18 files changed

+594
-434
lines changed

ports/unix/coverage.c

Lines changed: 0 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -526,26 +526,6 @@ static mp_obj_t extra_coverage(void) {
526526
mp_emitter_warning(MP_PASS_CODE_SIZE, "test");
527527
}
528528

529-
// format float
530-
{
531-
mp_printf(&mp_plat_print, "# format float\n");
532-
533-
// format with inadequate buffer size
534-
char buf[5];
535-
mp_format_float(1, buf, sizeof(buf), 'g', 0, '+');
536-
mp_printf(&mp_plat_print, "%s\n", buf);
537-
538-
// format with just enough buffer so that precision must be
539-
// set from 0 to 1 twice
540-
char buf2[8];
541-
mp_format_float(1, buf2, sizeof(buf2), 'g', 0, '+');
542-
mp_printf(&mp_plat_print, "%s\n", buf2);
543-
544-
// format where precision is trimmed to avoid buffer overflow
545-
mp_format_float(1, buf2, sizeof(buf2), 'e', 0, '+');
546-
mp_printf(&mp_plat_print, "%s\n", buf2);
547-
}
548-
549529
// binary
550530
{
551531
mp_printf(&mp_plat_print, "# binary\n");

0 commit comments

Comments
 (0)
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy