-
-
Notifications
You must be signed in to change notification settings - Fork 8.3k
py: Fixes and test coverage for 64-bit big integer representations. #16953
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
py: Fixes and test coverage for 64-bit big integer representations. #16953
Conversation
ec883a8
to
c6f3856
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #16953 +/- ##
=======================================
Coverage 98.44% 98.44%
=======================================
Files 171 171
Lines 22208 22208
=======================================
Hits 21863 21863
Misses 345 345 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Code size report:
|
0dd8885
to
5d8e0c1
Compare
This comment was marked as outdated.
This comment was marked as outdated.
f8acf08
to
c862fb1
Compare
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
Ah of course, thanks! |
c862fb1
to
aa0917b
Compare
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
8555e92
to
3b737ad
Compare
fd40fb1
to
50ad726
Compare
50ad726
to
9ed365d
Compare
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
Thank you for asking. Feel free to do as you prefer. Let's say that if I don't see the new PR coming once this one gets integrated into the master, I will submit it :-) |
9ed365d
to
ba9e3fc
Compare
Drat, when I tested the code size improvement from unsigned |
The issue is because I've pushed a commit to try and improve the situation there, by adjusting which local has it's address taken. That improves things for most ports, but others (unix, esp8266) increase a little with my change. Still, I think that's a reasonable improvement. That change to |
995e3dd
to
ed0974a
Compare
Alright, a bit of finagling and we've got a very minor code size reduction on all the bare metal ports! 🎉 I am not sure which part is causing the minor increase in unix port sizes, but it's small enough to possibly be noise. I've rebased and simplified the commit history (removed the parsing fixes and improvements in |
ed0974a
to
aa1477c
Compare
@projectgus I'm happy to squash my optimisation commit down into your commit (the one just prior to mine). I don't need the attribution for that. |
aa1477c
to
c8f23ee
Compare
@dpgeorge OK, have squashed! (And rebased agian also.) |
These will run on all ports which support them, but importantly they'll also run on ports that don't support arbitrary precision but do support 64-bit long ints. Includes some test workarounds to account for things which will overflow once "long long" big integers overflow (added in follow-up commit): - uctypes_array_load_store test was failing already, now won't parse. - all the ffi_int tests contain 64-bit unsigned values, that won't parse as long long. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>
These tests cover the use of mp_obj_new_int_from_str_len when mp_parse_num_integer overflows the SMALLINT limit, and also the case where the value may not be null terminated. Placed in a separate test file so that extmod/json test doesn't rely on bigint support. Signed-off-by: Yoctopuce dev <dev@yoctopuce.com> Signed-off-by: Angus Gratton <angus@redyak.com.au>
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Relies on arbitrary precision math, so won't run on a port which has threads & limited bigint support. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>
The other performance tests run and pass with only 64-bit big integer support. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>
Long long big integer support now raises an exception on overflow rather than returning an undefined result. Also adds an error when shifting by a negative value. The new arithmetic checks are added in the misc.h header. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>
Makes it compatible with the __builtin_mul_overflow() syntax, used in follow-up commit. Includes optimisation in runtime.c to minimise the code size impact from additional param. Signed-off-by: Damien George <damien@micropython.org> Signed-off-by: Angus Gratton <angus@redyak.com.au>
If big integer support is 'long long' then mp_parse_num_integer() can parse to it directly instead of failing over from small int. This means strtoll() is no longer pulled in, and fixes some bugs parsing long long integers (i.e. can now parse negative values correctly, can now parse values which aren't NULL terminated). The (default) smallint parsing compiled code should stay the same here, macros and a typedef are used to abstract some parts of it out. When bigint is long long we parse to 'unsigned long long' first (to avoid the code size hit of pulling in signed 64-bit math routines) and the convert to signed at the end. One tricky case this routine correctly overflows on is int("9223372036854775808") which is one more than LLONG_MAX in decimal. No unit test case added for this as it's too hard to detect 64-bit long integer mode. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking good!
636b9bb
to
17fbc5a
Compare
Thanks @projectgus and @yoctopuce for your efforts here. Lots of small details have been addressed which is great! |
Thanks @projectgus, you deserve all the credit. This is a really nice improvement for bare metal ports with limited ressources. |
Summary
As pointed out by @yoctopuce in this discussion and #16932, there is poor test coverage for the optional 64-bit bigint representation and this representation has some bugs.
This PR aims to improve this:
int_64
equivalent.longlong
unix build variant that enables 64-bit long int mode. Mostly useful for CI testing.mp_parse_num()
to parse directly tolong long
in the 64-bit big integer configuration, saving code size.Thanks to @yoctopuce for suggesting and demonstrating a bunch of these ideas, and improvements on the original version of this PR.
Testing
--via-mpy
. All passing.Follow-up work (for new PRs)
longlong
config to also testMICROPY_OBJ_REPR_C
.Trade-offs and Alternatives