Skip to content

Allow the GIL to be enabled on the unix port, and add a CI job to test this configuration #17668

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Jul 23, 2025
Merged

Allow the GIL to be enabled on the unix port, and add a CI job to test this configuration #17668

merged 4 commits into from
Jul 23, 2025

Conversation

dpgeorge
Copy link
Member

@dpgeorge dpgeorge commented Jul 13, 2025 •
edited
Loading

Summary

Up until now, thread tests have only been run (under CI at least) on a small number of build configurations. PR #17655 enables the thread tests on a lot more of the existing builds, and this PR adds a new job which builds and tests the unix port with the GIL enabled.

(This PR builds on #17655 and includes all the commits from that PR.)

Running tests with the GIL enabled will help to test the esp32 port's threading implementation, which also uses the GIL.

Edit: updated some thread tests to add time.sleep(0) to the busy-wait loop, so that they pass when used with the native emitter.

Testing

Will be tested by CI.

Edit: new CI job runs correctly, and takes 1 minute 55 seconds. It runs the thread tests in bytecode and native emitter mode.

Trade-offs and Alternatives

Adds a new CI job which adds to the CI time, but it should only be a couple of minutes, and run in parallel.

@dpgeorge dpgeorge added port-unix tests Relates to tests/ directory in source labels Jul 13, 2025
@github-actions GitHub Actions
Copy link

Code size report:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:    +0 +0.000% standard
      stm32:    +0 +0.000% PYBV10
     mimxrt:    +0 +0.000% TEENSY40
        rp2:    +0 +0.000% RPI_PICO_W
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:    +0 +0.000% VIRT_RV32

@codecov Codecov
Copy link

codecov bot commented Jul 13, 2025 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.37%. Comparing base (9219311) to head (677a0e0).
Report is 4 commits behind head on master.

Additional details and impacted files 
@@           Coverage Diff           @@
##           master   #17668   +/-   ##
=======================================
  Coverage   98.37%   98.37%           
=======================================
  Files         171      171           
  Lines       22210    22210           
=======================================
  Hits        21849    21849           
  Misses        361      361

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@dpgeorge dpgeorge added this to the release-1.26.0 milestone Jul 13, 2025
dlech
dlech reviewed Jul 13, 2025
View reviewed changes
@dpgeorge dpgeorge force-pushed the tests-add-unix-gil-build branch from 526493b to 62f720f Compare July 14, 2025 12:23
@dpgeorge dpgeorge requested a review from projectgus July 14, 2025 23:29
projectgus
projectgus approved these changes Jul 15, 2025
View reviewed changes
Copy link
Contributor

@projectgus projectgus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good to me. Suggest as part of this PR we also add some brief docs about the native emitter thread/scheduler starvation issue, and the sleep(0) workaround. I think the appropriate place is the dot point list in reference/speed_python.rst?

(While looking for that I also noticed that refernece/micropython.rst doesn't document the native annotation at all - it should probably have a short entry that links to the explanatory section in speed_python.rst? I can submit this in a follow-up, though.)

@hmaerki hmaerki mentioned this pull request Jul 15, 2025
Open
@dpgeorge dpgeorge force-pushed the tests-add-unix-gil-build branch from 62f720f to 13aae9e Compare July 17, 2025 05:05
@dpgeorge
Copy link
Member Author

Suggest as part of this PR we also add some brief docs about the native emitter thread/scheduler starvation issue, and the sleep(0) workaround.

I've now added this in the last commit.

While looking for that I also noticed that refernece/micropython.rst doesn't document the native annotation at all - it should probably have a short entry that links to the explanatory section in speed_python.rst?

Yes, would be good to document bytecode/native/viper decorators in docs/library/micropython.rst. Feel free to do that (but not urgent).

dpgeorge added 4 commits July 23, 2025 11:35
@dpgeorge
unix: Allow the GIL to be enabled.
b6460df 
The unix port can now be built with the GIL enabled, by passing
MICROPY_PY_THREAD_GIL=1 on the make command line.

Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
github/workflows: Add new CI job to test unix port with GIL enabled.
b15065b 
Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
tests/thread: Allow thread tests to pass with the native emitter.
b070765 
The native emitter will not release/bounce the GIL when running code, so
if it runs tight loops then no other threads get a chance to run (if the
GIL is enabled).  So for the thread tests, explicitly include a call to
`time.sleep(0)` (or equivalent) to bounce the GIL and give other threads a
chance to run.

For some tests (eg `thread_coop.py`) the whole point of the test is to test
that the GIL is correctly bounced.  So for those cases force the use of the
bytecode emitter for the busy functions.

Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
docs/reference/speed_python: Document schedule/GIL limitation of native.
677a0e0 
Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge dpgeorge force-pushed the tests-add-unix-gil-build branch from 13aae9e to 677a0e0 Compare July 23, 2025 01:37
@dpgeorge dpgeorge merged commit 677a0e0 into micropython:master Jul 23, 2025
130 of 134 checks passed
@dpgeorge dpgeorge deleted the tests-add-unix-gil-build branch July 23, 2025 02:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dlech dlech dlech left review comments

@projectgus projectgus projectgus approved these changes

Assignees
No one assigned
Labels
port-unix tests Relates to tests/ directory in source
Projects
None yet
Milestone
release-1.26.0
Development

Successfully merging this pull request may close these issues.
3 participants
@dpgeorge @projectgus @dlech
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy