Skip to content

WIP py/objint.c: Add signed param to int.from_bytes(). #16798

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

WIP py/objint.c: Add signed param to int.from_bytes(). #16798

wants to merge 1 commit into from
+306 −39

Conversation

IhorNehrutsa
Copy link
Contributor

@IhorNehrutsa IhorNehrutsa commented Feb 22, 2025 •
edited
Loading

Summary

Support signed=True param:
result = int.from_bytes(bytearray(), byteorder='big'|'little', signed=False|True)

>>> int.from_bytes(b'\xff', byteorder='little', signed=False)
255
>>> int.from_bytes(b'\xff', byteorder='little', signed=True)
-1
>>>

Testing

test_int.py
Added:
micropython/tests/basics/int_from_bytes.py

Tested on ESP32, ESP32-S2 boards.

Trade-offs and Alternatives

The length of the byte array is limited to sizeof(mp_longint_impl_t) in this PR.
int.from_bytes(bytes(20)) will raise OverflowError: big-int overflow

@IhorNehrutsa
Copy link
Contributor Author 

# test_int.py


def test(x, order, signed):
    size = 4 if x < 2**30 and x >= -(2**30) else 8
    b = x.to_bytes(size, order, signed)
    y = int.from_bytes(b, order, signed)
    print(x, y, x == y, b)
    print()
    assert x == y


def do_test(n, order, signed):
    print("==========", "n:", n, "order:", order, "signed:", signed, "==========")
    test(0, order, signed)
    test(1, order, signed)
    test(2, order, signed)
    test(0xFF, order, signed)
    test(0xFFFF, order, signed)
    test(0xFFFFFF, order, signed)
    test(0x3FFFFFFF, order, signed)
    # test(0x7FFFffff, order, signed)
    # test(0xFFFFffff, order, signed)
    # test(0xffFFFFffff, order, signed)
    # test(0xffffFFFFffff, order, signed)
    # test(0xFFffffFFFFffff, order, signed)
    # test(0xFFFFffffFFFFffff, order, signed)

    if signed:
        print("----------", "n:", n, "order:", order, "signed:", signed, "----------")
        test(-0, order, signed)
        test(-1, order, signed)
        test(-2, order, signed)
        test(-0xFF - 1, order, signed)
        test(-0xFFFF - 1, order, signed)
        test(-0xFFFFFF - 1, order, signed)
        test(-0x3FFFFFFF - 1, order, signed)
        # test(-0x7FFFffff-1, order, signed)
        # test(-0xFFFFffff-1, order, signed)
        # test(-0xffFFFFffff-1, order, signed)
        # test(-0xffffFFFFffff-1, order, signed)
        # test(-0xFFffffFFFFffff-1, order, signed)
        # test(-0xFFFFffffFFFFffff-1, order, signed)


do_test(n=1, order="little", signed=False)
do_test(n=2, order="big", signed=False)
do_test(n=3, order="little", signed=True)
do_test(n=4, order="big", signed=True)
print("Ok.")

@IhorNehrutsa
Copy link
Contributor Author 

MicroPython v1.25.0-preview.308.g1f1bcb5ca.dirty on 2025-02-23; Generic ESP32S2 module with ESP32S2
>>> %Run -c $EDITOR_CONTENT
MPY: soft reboot
========== n: 1 order: little signed: False ==========
big_endian:0 signed:0 bufinfo.len:4 sizeof(result.uval):4 result.uval:0
0 0 True b'\x00\x00\x00\x00'

big_endian:0 signed:0 bufinfo.len:4 sizeof(result.uval):4 result.uval:1
1 1 True b'\x01\x00\x00\x00'

big_endian:0 signed:0 bufinfo.len:4 sizeof(result.uval):4 result.uval:2
2 2 True b'\x02\x00\x00\x00'

big_endian:0 signed:0 bufinfo.len:4 sizeof(result.uval):4 result.uval:255
255 255 True b'\xff\x00\x00\x00'

big_endian:0 signed:0 bufinfo.len:4 sizeof(result.uval):4 result.uval:65535
65535 65535 True b'\xff\xff\x00\x00'

big_endian:0 signed:0 bufinfo.len:4 sizeof(result.uval):4 result.uval:16777215
16777215 16777215 True b'\xff\xff\xff\x00'

big_endian:0 signed:0 bufinfo.len:4 sizeof(result.uval):4 result.uval:1073741823
1073741823 1073741823 True b'\xff\xff\xff?'

========== n: 2 order: big signed: False ==========
big_endian:1 signed:0 bufinfo.len:4 sizeof(result.uval):4 result.uval:0
0 0 True b'\x00\x00\x00\x00'

big_endian:1 signed:0 bufinfo.len:4 sizeof(result.uval):4 result.uval:1
1 1 True b'\x00\x00\x00\x01'

big_endian:1 signed:0 bufinfo.len:4 sizeof(result.uval):4 result.uval:2
2 2 True b'\x00\x00\x00\x02'

big_endian:1 signed:0 bufinfo.len:4 sizeof(result.uval):4 result.uval:255
255 255 True b'\x00\x00\x00\xff'

big_endian:1 signed:0 bufinfo.len:4 sizeof(result.uval):4 result.uval:65535
65535 65535 True b'\x00\x00\xff\xff'

big_endian:1 signed:0 bufinfo.len:4 sizeof(result.uval):4 result.uval:16777215
16777215 16777215 True b'\x00\xff\xff\xff'

big_endian:1 signed:0 bufinfo.len:4 sizeof(result.uval):4 result.uval:1073741823
1073741823 1073741823 True b'?\xff\xff\xff'

========== n: 3 order: little signed: True ==========
big_endian:0 signed:1 bufinfo.len:4 sizeof(result.ival):4 result.ival:0
0 0 True b'\x00\x00\x00\x00'

big_endian:0 signed:1 bufinfo.len:4 sizeof(result.ival):4 result.ival:1
1 1 True b'\x01\x00\x00\x00'

big_endian:0 signed:1 bufinfo.len:4 sizeof(result.ival):4 result.ival:2
2 2 True b'\x02\x00\x00\x00'

big_endian:0 signed:1 bufinfo.len:4 sizeof(result.ival):4 result.ival:255
255 255 True b'\xff\x00\x00\x00'

big_endian:0 signed:1 bufinfo.len:4 sizeof(result.ival):4 result.ival:65535
65535 65535 True b'\xff\xff\x00\x00'

big_endian:0 signed:1 bufinfo.len:4 sizeof(result.ival):4 result.ival:16777215
16777215 16777215 True b'\xff\xff\xff\x00'

big_endian:0 signed:1 bufinfo.len:4 sizeof(result.ival):4 result.ival:1073741823
1073741823 1073741823 True b'\xff\xff\xff?'

---------- n: 3 order: little signed: True ----------
big_endian:0 signed:1 bufinfo.len:4 sizeof(result.ival):4 result.ival:0
0 0 True b'\x00\x00\x00\x00'

big_endian:0 signed:1 bufinfo.len:4 sizeof(result.ival):4 result.ival:-1
-1 -1 True b'\xff\xff\xff\xff'

big_endian:0 signed:1 bufinfo.len:4 sizeof(result.ival):4 result.ival:-2
-2 -2 True b'\xfe\xff\xff\xff'

big_endian:0 signed:1 bufinfo.len:4 sizeof(result.ival):4 result.ival:-256
-256 -256 True b'\x00\xff\xff\xff'

big_endian:0 signed:1 bufinfo.len:4 sizeof(result.ival):4 result.ival:-65536
-65536 -65536 True b'\x00\x00\xff\xff'

big_endian:0 signed:1 bufinfo.len:4 sizeof(result.ival):4 result.ival:-16777216
-16777216 -16777216 True b'\x00\x00\x00\xff'

big_endian:0 signed:1 bufinfo.len:4 sizeof(result.ival):4 result.ival:-1073741824
-1073741824 -1073741824 True b'\x00\x00\x00\xc0'

========== n: 4 order: big signed: True ==========
big_endian:1 signed:1 bufinfo.len:4 sizeof(result.ival):4 result.ival:0
0 0 True b'\x00\x00\x00\x00'

big_endian:1 signed:1 bufinfo.len:4 sizeof(result.ival):4 result.ival:1
1 1 True b'\x00\x00\x00\x01'

big_endian:1 signed:1 bufinfo.len:4 sizeof(result.ival):4 result.ival:2
2 2 True b'\x00\x00\x00\x02'

big_endian:1 signed:1 bufinfo.len:4 sizeof(result.ival):4 result.ival:255
255 255 True b'\x00\x00\x00\xff'

big_endian:1 signed:1 bufinfo.len:4 sizeof(result.ival):4 result.ival:65535
65535 65535 True b'\x00\x00\xff\xff'

big_endian:1 signed:1 bufinfo.len:4 sizeof(result.ival):4 result.ival:16777215
16777215 16777215 True b'\x00\xff\xff\xff'

big_endian:1 signed:1 bufinfo.len:4 sizeof(result.ival):4 result.ival:1073741823
1073741823 1073741823 True b'?\xff\xff\xff'

---------- n: 4 order: big signed: True ----------
big_endian:1 signed:1 bufinfo.len:4 sizeof(result.ival):4 result.ival:0
0 0 True b'\x00\x00\x00\x00'

big_endian:1 signed:1 bufinfo.len:4 sizeof(result.ival):4 result.ival:-1
-1 -1 True b'\xff\xff\xff\xff'

big_endian:1 signed:1 bufinfo.len:4 sizeof(result.ival):4 result.ival:-2
-2 -2 True b'\xff\xff\xff\xfe'

big_endian:1 signed:1 bufinfo.len:4 sizeof(result.ival):4 result.ival:-256
-256 -256 True b'\xff\xff\xff\x00'

big_endian:1 signed:1 bufinfo.len:4 sizeof(result.ival):4 result.ival:-65536
-65536 -65536 True b'\xff\xff\x00\x00'

big_endian:1 signed:1 bufinfo.len:4 sizeof(result.ival):4 result.ival:-16777216
-16777216 -16777216 True b'\xff\x00\x00\x00'

big_endian:1 signed:1 bufinfo.len:4 sizeof(result.ival):4 result.ival:-1073741824
-1073741824 -1073741824 True b'\xc0\x00\x00\x00'

Ok.
>>>

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 22, 2025 •
edited
Loading

Code size report:

   bare-arm:  +200 +0.352% 
minimal x86:  +574 +0.306% [incl +32(data)]
   unix x64:  +624 +0.073% standard[incl +32(data)]
      stm32:  +272 +0.069% PYBV10
     mimxrt:  +264 +0.072% TEENSY40
        rp2:  +224 +0.024% RPI_PICO_W
       samd:  +264 +0.098% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:  +295 +0.065% VIRT_RV32

@IhorNehrutsa IhorNehrutsa changed the title py/objint.c: Code review of int_from_bytes(). WIP py/objint.c: Code review of int_from_bytes(). Feb 22, 2025
@IhorNehrutsa IhorNehrutsa force-pushed the int_from_bytes branch 12 times, most recently from 8f7925d to f3403bb Compare February 24, 2025 11:23
@IhorNehrutsa
Copy link
Contributor Author

@projectgus @dpgeorge @robert-hh

Need a help.
Is it possible to use MICROPY_LONGINT_IMPL_LONGLONG?

/home/runner/work/micropython/micropython/ports/esp32/build-ESP32_GENERIC/frozen_content.c:22:2: error: #error "incompatible MICROPY_LONGINT_IMPL"
   22 | #error "incompatible MICROPY_LONGINT_IMPL"
      |  ^~~~~

Thanks.

@IhorNehrutsa IhorNehrutsa changed the title WIP py/objint.c: Code review of int_from_bytes(). WIP py/objint.c: Add 'signed' param to int.from_bytes(). Feb 24, 2025
@IhorNehrutsa IhorNehrutsa changed the title WIP py/objint.c: Add 'signed' param to int.from_bytes(). WIP py/objint.c: Add signed param to int.from_bytes(). Feb 24, 2025
@projectgus
Copy link
Contributor

Support signed=True param: result = int.from_bytes(bytearray(), order='big'|'little', signed=False|True)

See also #16311

@projectgus
Copy link
Contributor

Is it possible to use MICROPY_LONGINT_IMPL_LONGLONG?

I think the error you have here may be caused by a partial build failure, you can try doing a clean build and see if it goes away. However I'm not certain every longint implementation works on every port.

@IhorNehrutsa
Copy link
Contributor Author

caused by a partial build failure, you can try doing a clean build

cd ~/micropython/ports/esp32
make clean

rm -R managed_components
rm -R build*

The same errors remain. :(

@projectgus
Copy link
Contributor

The same errors remain. :(

As well as changing the constant in mpconfigport.h, you'll need to change the build system so mpy-tool.py is run with the extra argument -mlongint_impl=longlong.

I think for esp32 port this will mean something like adding set(MICROPY_MPY_TOOL_FLAGS -mlongint_impl=longlong) in the CMake file. I haven't tested this, it's not really a supported configuration.

@dpgeorge dpgeorge added the py-core Relates to py/ directory in source label Mar 13, 2025
@IhorNehrutsa IhorNehrutsa force-pushed the int_from_bytes branch 2 times, most recently from 44e5e3c to 090cfca Compare March 14, 2025 13:48
@IhorNehrutsa
py/objint.c: Code review of int.from_bytes().
0489c31 
Support signed param:
result = int.from_bytes(bytearray(),
order='big'|'little', signed=False|True)

Add `length`, `byteorder`, `signed`
according to the micropython#16311.

Signed-off-by: Ihor Nehrutsa <Ihor.Nehrutsa@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
py-core Relates to py/ directory in source
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@IhorNehrutsa @projectgus @dpgeorge
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy