tests/run-tests.py: Auto detect whether the target has threading and the GIL or not. #17655

dpgeorge · 2025-07-10T06:26:42Z

Summary

This PR makes the following changes to the test runner and CI:

detect if the target has the _thread module and, if so, add the threading tests to the set of tests to run
if the target has threading, detect if the target has the GIL enabled or not, and use that information to exclude thread mutating tests if the GIL is disabled
work around threading bug in tests/extmod/select_poll_eintr.py test
make tests/thread/stress_aes.py test do less work on stackless threading builds, so it finishes before the test timeout limit
increase test timeout for unix qemu test runs
skip some threading tests on macOS and on unix qemu

Currently the information whether a port has the GIL is hard-coded: unix/PC targets and the rp2 port are considered to be GIL-less, and all other targets have the GIL enabled (if they have threading enabled). With the change here, some code is run to detect the GIL. That uses the fact that native code won't bounce the GIL and can hog it, and times how long code runs.

Overall the changes here mean that the threading tests are run in many more configurations:

on the unix port with the native emitter
on the unix port with single precision float
on the unix port with stackless mode enabled
on the unix port using clang as a compiler
on unix qemu, the architectures MIPS, ARM and RISCV-64
on macOS

Testing

Tested locally on the unix port, esp32 and rp2. Let's also see how the CI works.

Trade-offs and Alternatives

This uses a tricky technique to determine GIL/non-GIL. An alternative would be to add some indication in a function whether the GIL is enabled, eg sys.implementation._thread_info. But that will increase code size.

codecov · 2025-07-10T06:30:50Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.41%. Comparing base (0b698b8) to head (9caf2ce).
Report is 2 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #17655      +/-   ##
==========================================
- Coverage   98.44%   98.41%   -0.04%     
==========================================
  Files         171      171              
  Lines       22208    22208              
==========================================
- Hits        21863    21855       -8     
- Misses        345      353       +8

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

github-actions · 2025-07-10T06:36:04Z

Code size report:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:   +88 +0.010% standard[incl +64(data)]
      stm32:    +0 +0.000% PYBV10
     mimxrt:    +0 +0.000% TEENSY40
        rp2:   +16 +0.002% RPI_PICO_W
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:    +0 +0.000% VIRT_RV32

dlech · 2025-07-10T15:15:05Z

tests/thread/stress_aes.py

+    try:
+
+        def stackless():
+            pass
+
+        micropython.heap_lock()
+        stackless()
+        micropython.heap_unlock()
+    except RuntimeError:
+        micropython.heap_unlock()
+        is_stackless = True


Suggested change

try:

def stackless():

pass

micropython.heap_lock()

stackless()

micropython.heap_unlock()

except RuntimeError:

micropython.heap_unlock()

is_stackless = True

def stackless():

pass

micropython.heap_lock()

try:

stackless()

except RuntimeError:

is_stackless = True

finally:

micropython.heap_unlock()

does it work to make the try/except a bit more focused on where the actual exception is expected?

You are right, it's probably better to be more focused like that. I will change it.

(I was just copying existing code for this from eg tests/micropython/heapalloc.py.)

dlech · 2025-07-10T15:55:32Z

ports/unix/mpthreadport.c

@@ -56,7 +56,7 @@
 #endif

 // This value seems to be about right for both 32-bit and 64-bit builds.
-#define THREAD_STACK_OVERFLOW_MARGIN (8192)
+#define THREAD_STACK_OVERFLOW_MARGIN (4 * 8192)


Suggested change

#define THREAD_STACK_OVERFLOW_MARGIN (4 * 8192)

#define THREAD_STACK_OVERFLOW_MARGIN (16 * 1024)

Also, the commit message could use an explanation of why this is changed.

I'm currently trying to work out if this change is even needed.

This change is not needed, it doesn't fix the issue I thought it would fix. So I took it out.

dlech · 2025-07-10T15:57:18Z

tests/feature_check/thread_gil.py

+    thread_done = True
+
+
+global thread_done


Do we actually need global statement at the top level?

You are right, now fixed.

dlech · 2025-07-10T16:11:19Z

tests/feature_check/thread_gil.py

+busy()
+while not thread_done:
+    time.sleep(0)
+dt = time.ticks_diff(time.ticks_ms(), t0)


Depending on time measurements seems like it could be flaky, especially on desktop systems. I'm not awake enough yet though to try to come up with an alternative that doesn't involve creating a deadlock.

Yes, it could be flaky. But from testing on bare-metal and PC, it looks reasonable. The timing is usually either very close to T or 2*T (depending if GIL is disable or enabled).

If it fails due to timing errors (running for too long), it will incorrectly think the GIL is enabled when it is not. Then it will run the mutate tests and they'll probably all fail. In such a case we can just rerun CI...

If it gets to be a real issue, could increase the timing here, eg double it (I just don't want the auto detection to take too long, because it's run at the start of each run of run-tests.py.)

I have some similar concerns, and generally prefer the approach of explicitly adding this information in sys.implementation or similar. Suggest we keep this approach now, but keep the explicit option in mind if it turns out to be flaky.

Having some information in sys.implementation could actually be useful for users, instead of them having to run a tricky detection test like this (if they ever need to know the threading implementation...).

And I guess if you have threading enabled you have some amount of flash to spare (although not necessarily on cc3200 which does enable threading...).

So maybe it's worth adding sys.implementation._thread after all?

I think it'd be useful to add. Especially if we don't add any _thread property on builds without threading support.

I've now changed this GIL detection to use sys.implementation._thread. It's a lot simpler.

And retested on PICO_W, ESP32_GENERIC and unix port. They detect the GIL correctly and the relevant tests run.

Much neater, I agree worth the extra size & complexity!

dpgeorge · 2025-07-10T23:08:17Z

Thanks @dlech for the review here. After I initially submitted this PR and all the CI failed on things I couldn't test locally, the scope of the PR increased quite a bit... so there was a lot of back and forth getting it to work.

dpgeorge · 2025-07-13T12:31:23Z

After a lot of back-and-forth with the CI, this PR is now in its final state. I updated the comment at the top with a better description of what this PR is aiming for.

projectgus · 2025-07-17T04:05:16Z

docs/library/sys.rst

+   The *_thread* entry was added in version 1.26.0 and if it exists then the
+   target has the ``_thread`` module.  The target enables the GIL (global
+   interpreter lock) if this attribute equals ``"GIL"``, otherwise the attribute
+   is ``"no-GIL"`` and the target does not enable the GIL.


Do you think MicroPython will ever have a variant that is non-GIL but thread safe (ala CPython --disable-gil?)

If yes then maybe we could make the attribute values GIL and unsafe (or similar), keeping the door open for a safe value in some future release.

(Would mean tests which test for no-GIL will need to test _thread != "GIL", I guess

Do you think MicroPython will ever have a variant that is non-GIL but thread safe

Yes, that's a possibility. But, I would suspect that we gradually evolve the current no-GIL mode to a more safer behaviour, rather than have separate options for GIL/unsafe-no-GIL/safe-no-GIL.

That said, I'm happy to change "no-GIL" to "unsafe". Although that could scare users, but OTOH it might push them to read the docs to understand what "unsafe" actually means.

Is there another word that could better describe the situation, eg "racey" or "raceful"?

I think it's OK if this scares users! It's quite difficult to write threaded Python correctly with a thread unsafe runtime.

But, I would suspect that we gradually evolve the current no-GIL mode to a more safer behaviour, rather than have separate options for GIL/unsafe-no-GIL/safe-no-GIL.

Agree this would be the best way to evolve it. From perspective of someone targeting MicroPython it may still be useful to programmatically tell if the current runtime is thread safe or not (i.e. even if no MicroPython release supports all three values, there might still be code that checks for all three.)

OK, I've now renamed "no-GIL" to "unsafe", and updated the docs.

Also tested again on esp32, rp2 and unix.

This is useful to distinguish between GIL and non-GIL builds. Signed-off-by: Damien George <damien@micropython.org>

This is a workaround for the case where threading is enabled without a GIL. In such a configuration, creating a new global variable is not atomic and threads have race conditions resizing/accessing the global dict. Signed-off-by: Damien George <damien@micropython.org>

Builds with stackless enabled are slower than non-stackless, and this test takes around 2 minutes on the unix port of MicroPython with the standard unix parameters. So detect stackless mode and reduce the time of the test. Signed-off-by: Damien George <damien@micropython.org>

When detecting the target platform, also check if it has threading and whether the GIL is enabled or not (using the new attribute `sys.implementation._thread`). If threading is available, add the thread tests to the set of tests to run (unless the set of tests is explicitly given). With this change, the unix port no longer needs to explicitly run the set of thread tests, so that line has been removed from the Makefile. This change will make sure thread tests are run with other testing combinations. In particular, thread tests are now run: - on the unix port with the native emitter - on macOS builds - on unix qemu, the architectures MIPS, ARM and RISCV-64 Signed-off-by: Damien George <damien@micropython.org>

The qemu emulation introduces enough overhead that the `tests/thread/stress_aes.py` test overruns the default timeout. So increase it to allow this test to pass. Signed-off-by: Damien George <damien@micropython.org>

This test passes sometimes and fails other times. Eventually that should be fixed, but for now just skip this test. Signed-off-by: Damien George <damien@micropython.org>

dpgeorge added the tests Relates to tests/ directory in source label Jul 10, 2025

dpgeorge force-pushed the tests-detect-thread-gil branch from d63cbda to b90d11e Compare July 10, 2025 15:40

dpgeorge changed the title ~~tests/run-tests.py: Auto detect whether the target has the GIL or not.~~ tests/run-tests.py: Auto detect whether the target has threading and the GIL or not. Jul 10, 2025

dlech reviewed Jul 10, 2025

View reviewed changes

dpgeorge force-pushed the tests-detect-thread-gil branch 3 times, most recently from bf0c6cb to 1266375 Compare July 11, 2025 14:22

dpgeorge mentioned this pull request Jul 11, 2025

unix/make: Drop i686-linux-gnu path. #17638

Merged

dpgeorge force-pushed the tests-detect-thread-gil branch 7 times, most recently from 05a9ceb to 28535b0 Compare July 13, 2025 12:21

dpgeorge mentioned this pull request Jul 13, 2025

Allow the GIL to be enabled on the unix port, and add a CI job to test this configuration #17668

Open

dpgeorge added this to the release-1.26.0 milestone Jul 13, 2025

dpgeorge requested a review from projectgus July 14, 2025 23:29

projectgus approved these changes Jul 15, 2025

View reviewed changes

dpgeorge force-pushed the tests-detect-thread-gil branch from 28535b0 to 3c1c9ab Compare July 15, 2025 03:38

projectgus reviewed Jul 17, 2025

View reviewed changes

dpgeorge added 5 commits July 17, 2025 15:14

py/modsys: Add sys.implementation._thread attribute.

341bc9c

This is useful to distinguish between GIL and non-GIL builds. Signed-off-by: Damien George <damien@micropython.org>

tools/ci.sh: Increase timeout for unix qemu test runs.

822676a

The qemu emulation introduces enough overhead that the `tests/thread/stress_aes.py` test overruns the default timeout. So increase it to allow this test to pass. Signed-off-by: Damien George <damien@micropython.org>

dpgeorge added 2 commits July 17, 2025 15:16

tools/ci.sh: Skip thread/stress_heap.py test on macOS test run.

3939791

This test passes sometimes and fails other times. Eventually that should be fixed, but for now just skip this test. Signed-off-by: Damien George <damien@micropython.org>

tools/ci.sh: Skip thread/stress_recurse.py on unix qemu test runs.

9caf2ce

This test passes sometimes and fails other times. Eventually that should be fixed, but for now just skip this test. Signed-off-by: Damien George <damien@micropython.org>

dpgeorge force-pushed the tests-detect-thread-gil branch from 3c1c9ab to 9caf2ce Compare July 17, 2025 05:19

dpgeorge mentioned this pull request Jul 18, 2025

Test REPR_B & fix test failures #17688

Open

	#define THREAD_STACK_OVERFLOW_MARGIN (4 * 8192)
	#define THREAD_STACK_OVERFLOW_MARGIN (16 * 1024)

Uh oh!

tests/run-tests.py: Auto detect whether the target has threading and the GIL or not. #17655

Are you sure you want to change the base?

tests/run-tests.py: Auto detect whether the target has threading and the GIL or not. #17655

Conversation

dpgeorge commented Jul 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Testing

Trade-offs and Alternatives

Uh oh!

codecov bot commented Jul 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions bot commented Jul 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

projectgus Jul 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dpgeorge commented Jul 10, 2025

Uh oh!

dpgeorge commented Jul 13, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

projectgus Jul 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

dpgeorge commented Jul 10, 2025 •

edited

Loading

codecov bot commented Jul 10, 2025 •

edited

Loading

github-actions bot commented Jul 10, 2025 •

edited

Loading

projectgus Jul 15, 2025 •

edited

Loading

projectgus Jul 17, 2025 •

edited

Loading