natmod: Allow linking static libraries. #16863
Conversation
|
For the ESP32 target, successful build run logs can be found here https://github.com/agatti/micropython/actions/runs/13666850878/job/38209736600 and here https://github.com/agatti/micropython/actions/runs/13666850878/job/38209737465 The commit checker fails to see the sign-off line, but I guess fixing that is its own PR. |
|
Code size report: |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #16863 +/- ##
=======================================
Coverage 98.54% 98.54%
=======================================
Files 169 169
Lines 21877 21877
=======================================
Hits 21558 21558
Misses 319 319 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
73fdab4 to
ee0c65f
Compare
|
I have cleared the ESP-IDF cache in GitHub Actions, and now the esp32 tests pass on this PR. |
| if not is_ld_script(): | ||
| return [CachedArFile(fn)] | ||
| else: | ||
| return [CachedArFile(item) for item in expand_ld_script(fn)] |
There was a problem hiding this comment.
I see you inverted the logic here by swapping the if/else. But I think @projectgus had a different idea in mind in his comment #15838 (comment)
I think what was intended was:
ar_header = "!<arch>\012"
with open(fn, "rb") as f:
is_ar_file = f.read(len(ar_header)) == ar_header
if is_ar_file:
return [CachedArFile(fn)]
else:
return [CachedArFile(item) for item in expand_ld_script(fn)]There was a problem hiding this comment.
I see that now - I'm on it. I was in a bit of a hurry back then to get the RV32 stuff working (I assumed it worked out of the box like when I tried it last time).
024bf70 to
06ead3e
Compare
|
This should be it - since last time you've looked at the code I've added the proper logic reversal for the magic bytes check, the platform name comparisons in the mpy build CI step now use explicit strings (as in quoted platform names), and I've added a message to the original PR's commit explaining the picolibc workaround and providing a bit more information about the change itself. |
This commit introduces an additional symbol resolution mechanism to the natmod linking process. This allows the build scripts to look for required symbols into selected libraries that are provided by the compiler installation (libgcc and libm at the moment). For example, using soft-float code in natmods, whilst technically possible, was not an easy process and required some additional work to pull it off. With this addition all the manual (and error-prone) operations have been automated and folded into `tools/mpy_ld.py`. Both newlib and picolibc toolchains are supported, albeit the latter may require a bit of extra configuration depending on the environment the build process runs on. Picolibc's soft-float functions aren't in libm - in fact the shipped libm is nothing but a stub - but they are inside libc. This is usually not a problem as these changes cater for that configuration quirk, but on certain compilers the include paths used to find libraries in may not be updated to take Picolibc's library directory into account. The bare metal RISC-V compiler shipped with the CI OS image (GCC 10.2.0 on Ubuntu 22.04LTS) happens to exhibit this very problem. To work around that for CI builds, the Picolibc libraries' path is hardcoded in the Makefile directives used by the linker, but this can be changed by setting the PICOLIBC_ROOT environment library when building natmods. Signed-off-by: Volodymyr Shymanskyy <vshymanskyi@gmail.com> Co-authored-by: Alessandro Gatti <a.gatti@frob.it>
This commit expands the CI tests by checking whether the example native modules are able to be built for the Xtensa architecture. This was made possible by the changes to mpy_ld that allow symbol resolution across standard compiler-provided libraries. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This commit removes the assumption made by the CI scripts that the system-provided python executable is simply named "python". The scripts will now look for a binary called "python3" first, and then fall back to "python" if that is not found. Whilst this is currently the case for the CI environment, there are no guarantees for this going forward. For example minimal CI environments set up by some developers, using the same base OS, have their python executable called "python3". Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
06ead3e to
31a008c
Compare
Summary
This is an update to #15838 - as requested in #15838 (comment).
All the good stuff is from @vshymanskyy, the not-so-good stuff is mine:
make cleanCACHEDIR.TAGfile, so most archival/backup programs will skip that directory (if rust can do it why can't we?)pythonexecutable in the CI environment (minor regression from 8b1ed44), blocking manual testing on my provisioned build VMs :)Testing
All natmod-related CI jobs ran without issues on my own repo's branch.