Skip to content

tests/run-tests.py: Auto detect whether the target has threading and the GIL or not. #17655

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jul 23, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions docs/library/sys.rst
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,8 @@ Constants
* *_mpy* - supported mpy file-format version (optional attribute)
* *_build* - string that can help identify the configuration that
MicroPython was built with
* *_thread* - optional string attribute, exists if the target has threading
and is either "GIL" or "unsafe"

This object is the recommended way to distinguish MicroPython from other
Python implementations (note that it still may not exist in the very
Expand All @@ -95,6 +97,14 @@ Constants
* On microcontroller targets, the first element is the board name and the second
element (if present) is the board variant, for example ``'RPI_PICO2-RISCV'``

The *_thread* entry was added in version 1.26.0 and if it exists then the
target has the ``_thread`` module. If the target enables the GIL (global
interpreter lock) then this attribute is ``"GIL"``. Otherwise the attribute
is ``"unsafe"`` and the target has threading but does not enable the GIL,
and mutable Python objects (such as `bytearray`, `list` and `dict`) that are
shared amongst threads must be protected explicitly by locks such as
``_thread.allocate_lock``.

.. admonition:: Difference to CPython
:class: attention

Expand Down
1 change: 0 additions & 1 deletion ports/unix/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -259,7 +259,6 @@ test: $(BUILD)/$(PROG) $(TOP)/tests/run-tests.py
test_full_no_native: $(BUILD)/$(PROG) $(TOP)/tests/run-tests.py
$(eval DIRNAME=ports/$(notdir $(CURDIR)))
cd $(TOP)/tests && MICROPY_MICROPYTHON=../$(DIRNAME)/$(BUILD)/$(PROG) ./run-tests.py
cd $(TOP)/tests && MICROPY_MICROPYTHON=../$(DIRNAME)/$(BUILD)/$(PROG) ./run-tests.py -d thread
cd $(TOP)/tests && MICROPY_MICROPYTHON=../$(DIRNAME)/$(BUILD)/$(PROG) ./run-tests.py --via-mpy $(RUN_TESTS_MPY_CROSS_FLAGS) -d basics float micropython
cat $(TOP)/tests/basics/0prelim.py | ./$(BUILD)/$(PROG) | grep -q 'abc'

Expand Down
24 changes: 20 additions & 4 deletions py/modsys.c
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,18 @@ static const MP_DEFINE_STR_OBJ(mp_sys_implementation__build_obj, MICROPY_BOARD_B
#define SYS_IMPLEMENTATION_ELEMS__BUILD
#endif

#if MICROPY_PY_THREAD
#if MICROPY_PY_THREAD_GIL
#define SYS_IMPLEMENTATION_ELEMS__THREAD \
, MP_ROM_QSTR(MP_QSTR_GIL)
#else
#define SYS_IMPLEMENTATION_ELEMS__THREAD \
, MP_ROM_QSTR(MP_QSTR_unsafe)
#endif
#else
#define SYS_IMPLEMENTATION_ELEMS__THREAD
#endif

#if MICROPY_PREVIEW_VERSION_2
#define SYS_IMPLEMENTATION_ELEMS__V2 \
, MP_ROM_TRUE
Expand All @@ -120,27 +132,31 @@ static const qstr impl_fields[] = {
#if defined(MICROPY_BOARD_BUILD_NAME)
MP_QSTR__build,
#endif
#if MICROPY_PY_THREAD
MP_QSTR__thread,
#endif
#if MICROPY_PREVIEW_VERSION_2
MP_QSTR__v2,
#endif
};
static MP_DEFINE_ATTRTUPLE(
mp_sys_implementation_obj,
impl_fields,
3 + MICROPY_PERSISTENT_CODE_LOAD + MICROPY_BOARD_BUILD + MICROPY_PREVIEW_VERSION_2,
3 + MICROPY_PERSISTENT_CODE_LOAD + MICROPY_BOARD_BUILD + MICROPY_PY_THREAD + MICROPY_PREVIEW_VERSION_2,
SYS_IMPLEMENTATION_ELEMS_BASE
SYS_IMPLEMENTATION_ELEMS__MPY
SYS_IMPLEMENTATION_ELEMS__BUILD
SYS_IMPLEMENTATION_ELEMS__THREAD
SYS_IMPLEMENTATION_ELEMS__V2
);
#else
static const mp_rom_obj_tuple_t mp_sys_implementation_obj = {
{&mp_type_tuple},
3 + MICROPY_PERSISTENT_CODE_LOAD,
// Do not include SYS_IMPLEMENTATION_ELEMS__BUILD or SYS_IMPLEMENTATION_ELEMS__V2
// because SYS_IMPLEMENTATION_ELEMS__MPY may be empty if
// Do not include SYS_IMPLEMENTATION_ELEMS__BUILD, SYS_IMPLEMENTATION_ELEMS__THREAD
// or SYS_IMPLEMENTATION_ELEMS__V2 because SYS_IMPLEMENTATION_ELEMS__MPY may be empty if
// MICROPY_PERSISTENT_CODE_LOAD is disabled, which means they'll share
// the same index. Cannot query _build or _v2 if MICROPY_PY_ATTRTUPLE is
// the same index. Cannot query _build, _thread or _v2 if MICROPY_PY_ATTRTUPLE is
// disabled.
{
SYS_IMPLEMENTATION_ELEMS_BASE
Expand Down
6 changes: 6 additions & 0 deletions tests/basics/sys1.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,12 @@
# Effectively skip subtests
print(str)

if hasattr(sys.implementation, '_thread'):
print(sys.implementation._thread in ("GIL", "unsafe"))
else:
# Effectively skip subtests
print(True)

try:
print(sys.intern('micropython') == 'micropython')
has_intern = True
Expand Down
8 changes: 8 additions & 0 deletions tests/extmod/select_poll_eintr.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,14 @@ def thread_main():
print("thread gc end")


# Pre-allocate global variables here so the global dict is not resized by the main
# thread while the secondary thread runs. This is a workaround for the bug described
# in https://github.com/micropython/micropython/pull/11604
poller = None
t0 = None
result = None
dt_ms = None

# Start a thread to interrupt the main thread during its call to poll.
lock = _thread.allocate_lock()
lock.acquire()
Expand Down
4 changes: 3 additions & 1 deletion tests/feature_check/target_info.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,6 @@
"xtensawin",
"rv32imc",
][sys_mpy >> 10]
print(platform, arch)
thread = getattr(sys.implementation, "_thread", None)

print(platform, arch, thread)
17 changes: 12 additions & 5 deletions tests/run-tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -251,22 +251,27 @@ def detect_test_platform(pyb, args):
output = run_feature_check(pyb, args, "target_info.py")
if output.endswith(b"CRASH"):
raise ValueError("cannot detect platform: {}".format(output))
platform, arch = str(output, "ascii").strip().split()
platform, arch, thread = str(output, "ascii").strip().split()
if arch == "None":
arch = None
inlineasm_arch = detect_inline_asm_arch(pyb, args)
if thread == "None":
thread = None

args.platform = platform
args.arch = arch
if arch and not args.mpy_cross_flags:
args.mpy_cross_flags = "-march=" + arch
args.inlineasm_arch = inlineasm_arch
args.thread = thread

print("platform={}".format(platform), end="")
if arch:
print(" arch={}".format(arch), end="")
if inlineasm_arch:
print(" inlineasm={}".format(inlineasm_arch), end="")
if thread:
print(" thread={}".format(thread), end="")
print()


Expand Down Expand Up @@ -810,8 +815,8 @@ def run_tests(pyb, tests, args, result_dir, num_threads=1):
skip_tests.add("cmdline/repl_sys_ps1_ps2.py")
skip_tests.add("extmod/ssl_poll.py")

# Skip thread mutation tests on targets that don't have the GIL.
if args.platform in PC_PLATFORMS + ("rp2",):
# Skip thread mutation tests on targets that have unsafe threading behaviour.
if args.thread == "unsafe":
for t in tests:
if t.startswith("thread/mutate_"):
skip_tests.add(t)
Expand Down Expand Up @@ -1329,6 +1334,8 @@ def main():
)
if args.inlineasm_arch is not None:
test_dirs += ("inlineasm/{}".format(args.inlineasm_arch),)
if args.thread is not None:
test_dirs += ("thread",)
if args.platform == "pyboard":
# run pyboard tests
test_dirs += ("float", "stress", "ports/stm32")
Expand All @@ -1337,9 +1344,9 @@ def main():
elif args.platform == "renesas-ra":
test_dirs += ("float", "ports/renesas-ra")
elif args.platform == "rp2":
test_dirs += ("float", "stress", "thread", "ports/rp2")
test_dirs += ("float", "stress", "ports/rp2")
elif args.platform == "esp32":
test_dirs += ("float", "stress", "thread")
test_dirs += ("float", "stress")
elif args.platform in ("esp8266", "minimal", "samd", "nrf"):
test_dirs += ("float",)
elif args.platform == "WiPy":
Expand Down
2 changes: 1 addition & 1 deletion tests/thread/stress_aes.py
Original file line number Diff line number Diff line change
Expand Up @@ -277,7 +277,7 @@ def thread_entry(n_loop):
n_thread = 2
n_loop = 2
else:
n_thread = 20
n_thread = 10
n_loop = 5
for i in range(n_thread):
_thread.start_new_thread(thread_entry, (n_loop,))
Expand Down
20 changes: 15 additions & 5 deletions tools/ci.sh
Original file line number Diff line number Diff line change
Expand Up @@ -720,7 +720,8 @@ function ci_unix_stackless_clang_build {
}

function ci_unix_stackless_clang_run_tests {
ci_unix_run_tests_helper CC=clang
# Timeout needs to be increased for thread/stress_aes.py test.
MICROPY_TEST_TIMEOUT=90 ci_unix_run_tests_helper CC=clang
}

function ci_unix_float_clang_build {
Expand Down Expand Up @@ -779,7 +780,8 @@ function ci_unix_macos_run_tests {
# Issues with macOS tests:
# - float_parse and float_parse_doubleprec parse/print floats out by a few mantissa bits
# - ffi_callback crashes for an unknown reason
(cd tests && MICROPY_MICROPYTHON=../ports/unix/build-standard/micropython ./run-tests.py --exclude '(float_parse|float_parse_doubleprec|ffi_callback).py')
# - thread/stress_heap.py is flaky
(cd tests && MICROPY_MICROPYTHON=../ports/unix/build-standard/micropython ./run-tests.py --exclude '(float_parse|float_parse_doubleprec|ffi_callback|thread/stress_heap).py')
}

function ci_unix_qemu_mips_setup {
Expand All @@ -797,8 +799,11 @@ function ci_unix_qemu_mips_build {
}

function ci_unix_qemu_mips_run_tests {
# Issues with MIPS tests:
# - thread/stress_aes.py takes around 50 seconds
# - thread/stress_recurse.py is flaky
file ./ports/unix/build-coverage/micropython
(cd tests && MICROPY_MICROPYTHON=../ports/unix/build-coverage/micropython ./run-tests.py)
(cd tests && MICROPY_MICROPYTHON=../ports/unix/build-coverage/micropython MICROPY_TEST_TIMEOUT=90 ./run-tests.py --exclude 'thread/stress_recurse.py')
}

function ci_unix_qemu_arm_setup {
Expand All @@ -818,8 +823,10 @@ function ci_unix_qemu_arm_build {
function ci_unix_qemu_arm_run_tests {
# Issues with ARM tests:
# - (i)listdir does not work, it always returns the empty list (it's an issue with the underlying C call)
# - thread/stress_aes.py takes around 70 seconds
# - thread/stress_recurse.py is flaky
file ./ports/unix/build-coverage/micropython
(cd tests && MICROPY_MICROPYTHON=../ports/unix/build-coverage/micropython ./run-tests.py --exclude 'vfs_posix.*\.py')
(cd tests && MICROPY_MICROPYTHON=../ports/unix/build-coverage/micropython MICROPY_TEST_TIMEOUT=90 ./run-tests.py --exclude 'vfs_posix.*\.py|thread/stress_recurse.py')
}

function ci_unix_qemu_riscv64_setup {
Expand All @@ -837,8 +844,11 @@ function ci_unix_qemu_riscv64_build {
}

function ci_unix_qemu_riscv64_run_tests {
# Issues with RISCV-64 tests:
# - thread/stress_aes.py takes around 140 seconds
# - thread/stress_recurse.py is flaky
file ./ports/unix/build-coverage/micropython
(cd tests && MICROPY_MICROPYTHON=../ports/unix/build-coverage/micropython ./run-tests.py)
(cd tests && MICROPY_MICROPYTHON=../ports/unix/build-coverage/micropython MICROPY_TEST_TIMEOUT=180 ./run-tests.py --exclude 'thread/stress_recurse.py')
}

########################################################################################
Expand Down
Loading
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy