Skip to content

py/emitnative: Optimise native code unwinding jumps. #16608

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

agatti
Copy link
Contributor

@agatti agatti commented Jan 19, 2025

Summary

This PR introduces an optimisation for emitting unwinding jump blocks in native code. Currently the native emitter will emit a jump to the block's unwinding code even though that jump is located right before the unwinding sequence itself. So the unwind code sequence would look like this (as emitted by mpy-cross using "-march=debug"):

    ...
    jump(label_0)
label(label_0)
    ...
EXIT(0)

or:

    ...
    jump(label_0)
    dead_code ...
label(label_0)
    ...
EXIT(0)

With this change, the native emitter only emits jumps to the exit label (label_0 in the examples above) when unwinding if both the exception stack and the emitter stack are empty, as it means the jump is at the very end of the code block.

This fixes issue #16604.

Testing

The full test suite was executed with make BOARD=VIRT_RV32 RUN_TESTS_EXTRA='--via-mpy --emit native --mpy-cross-flags="-march=rv32imc"' test on the QEMU/VIRT_RV32 board, and all tests pass. The lack of extra jumps to the unwind label was checked with mpy-cross emitting code for the debug architecture.

Trade-offs and Alternatives

This change is dependent on the current native emitter behaviour w.r.t. where to place the unwinding code relative to the unwinding jump. If the assumption of adjacency for the unwinding jump and the unwinding code is invalidated due to other changes to the emitter then this PR will make it generate invalid code.

Ideally this should be turned into a patch that would optimise all jumps regardless their location in the code as long as the label is right before or right after the code sequence in question. However from what I can see it is way more complicated to achieve as the only place where this optimisation can be done is at the lower level of the emitter framework (arch-dependent) and only after the label offsets have been stabilised. Omitting instructions at that point would move label addresses around and thus invalidate computed jumps.

Copy link

github-actions bot commented Jan 19, 2025

Code size report:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:   +64 +0.008% standard
      stm32:   +24 +0.006% PYBV10
     mimxrt:   +24 +0.007% TEENSY40
        rp2:   +24 +0.003% RPI_PICO_W
       samd:   +24 +0.009% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:   +40 +0.009% VIRT_RV32

@agatti
Copy link
Contributor Author

agatti commented Jan 19, 2025

Hrm, the async tests pass for the Unix port on my Arch Linux machine do pass when running cd ../../tests && MICROPY_MICROPYTHON=../ports/unix/build-standard/micropython ./run-tests.py --emit native - not sure what's going on in the CI environment for that.

Edit:

Seems like modasyncio behaves differently when built for unix/coverage even when ports/unix/variants/coverage/mpconfigport.h has all the additional features disabled (so it should have the same feature set as unix/standard). Setting MICROPY_CONFIG_ROM_LEVEL_EVERYTHING contains something that makes this PR incompatible with modasyncio.

@agatti agatti force-pushed the optimise-native-unwinding branch 2 times, most recently from c6b1970 to 86bc821 Compare January 19, 2025 14:56
Copy link

codecov bot commented Jan 19, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.58%. Comparing base (6db2997) to head (86bc821).
Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #16608   +/-   ##
=======================================
  Coverage   98.58%   98.58%           
=======================================
  Files         167      167           
  Lines       21596    21596           
=======================================
  Hits        21291    21291           
  Misses        305      305           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@agatti agatti marked this pull request as draft January 19, 2025 22:02
This commit introduces an optimisation for emitting unwinding jump
blocks in native code.  Currently the native emitter will emit a jump to
the block's unwinding code even though that jump is located right before
the unwinding sequence itself.  So the unwind code sequence would look
like this (as emitted by mpy-cross using "-march=debug"):

    ...
    jump(label_0)
label(label_0)
    ...
EXIT(0)

or:

    ...
    jump(label_0)
    dead_code ...
label(label_0)
    ...
EXIT(0)

With this change, the native emitter only emits jumps to the exit label
(label_0 in the examples above) when unwinding if both the exception
stack and the emitter stack are empty, as it means the jump is at the
very end of the code block.

This fixes issue micropython#16604.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
@agatti agatti force-pushed the optimise-native-unwinding branch from 86bc821 to 3d3ba5e Compare January 19, 2025 22:10
@agatti
Copy link
Contributor Author

agatti commented Jan 19, 2025

So modasyncio fails tests due to DEBUG being set and some assertions triggering in the unix/coverage port, along with different behaviour in a couple of tests w.r.t. task execution order. Weird that unix/standard without assertions in release mode passes the very same tests though...

I've marked the PR as draft, as I need to investigate further on what's going on here.

@dpgeorge dpgeorge added the py-core Relates to py/ directory in source label Jan 23, 2025
@agatti
Copy link
Contributor Author

agatti commented Jul 20, 2025

Whilst the changes proposed here seem to work with production code, I couldn't find a way out to avoid the test failures when compiled with DEBUG=1.

Not sure if it's the tests that expect a specific execution order of the asyncio functions that, if perturbed, makes them fail. In any case, this can be sidestepped as the recompiler already takes care of this.

@dpgeorge Is this still worth looking at or can I just close it instead?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
py-core Relates to py/ directory in source
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy