Skip to content

esp32: update IDF to 5.4.2 #17724

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 23, 2025
Merged

Conversation

dpgeorge
Copy link
Member

Summary

This PR updates the esp32 port to use IDF 5.4.2 (instead of 5.4.1).

The CI will now use IDF 5.4.2, and the README recommends this as the officially supported version. Downloads will also use the new version.

Thanks to @projectgus, here is a summary of the change in firmware size and RAM usage moving from 5.4.1 to 5.4.2:

BOARD BOARD_VARIANT IDF Version Binary Size Static IRAM Size Static DRAM Size
ESP32_GENERIC v5.4.1 1656557 117286 56136
v5.4.2 1658709 116894 53948
ESP32_GENERIC D2WD v5.4.1 1371197 111174 55840
v5.4.2 1372697 110714 53616
ESP32_GENERIC SPIRAM v5.4.1 1481400 116430 56264
v5.4.2 1482900 115998 54068
ESP32_GENERIC_S3 v5.4.1 1635902 16383 143295
v5.4.2 1645006 16383 142059
ESP32_GENERIC_S3 SPIRAM_OCT v5.4.1 1639314 16383 146727
v5.4.2 1648446 16383 145519

Comparing with 5.4.1, firmware size is up by about 1.5k on ESP32 and 9k on ESP32-S3. But IRAM usage (of the IDF) is down by about 500 byte on ESP32 and DRAM usage is down by about 20k on ESP32 and 10k on ESP32-S3.

Testing

I ran the full test suite (Python, .mpy, native, hardware, BLE, WiFi) on ESP32, ESP32-S2, ESP32-S3 and ESP32-C3. I did not see any regressions.

However, I did see a change in BLE event behaviour which makes tests/multi_bluetooth/ble_mtu.py and tests/multi_bluetooth/ble_mtu_peripheral.py now fail on ESP32 with IDF 5.4.2. The change in behaviour is that MTU_EXCHANGE events can now occur before CENTRAL_CONNECT/PERIPHERAL_CONNECT events. That seems a bit strange, because the MTU exchange occurs after the connection. And looking at the timing of the events there is exactly 100ms between them, ie MTU_EXCHANGE fires and then exactly 100ms later CENTRAL_CONNECT/PERIPHERAL_CONNECT fires.

I don't know if this is a bug in (Espressif's) NimBLE, a subtle change in scheduling with still valid behaviour, an intended change, a change allowed under the BLE spec, or something else. But I doubt we can fix that (easily) on our side and so in order to move forward with updating to IDF 5.4.2 I have adjusted the relevant tests so they can pass (basically, the test just needs to wait a bit between doing the connect and doing the MTU exchange, so the other side sees the original/correct ordering of events).

I have tested the modified tests on PYBD_SF6, RPI_PICO_W, ESP32 (IDF 5.4.1 and IDF 5.4.2) and ESP32-S3 (IDF 5.4.1 and IDF 5.4.2). They pass.

Trade-offs and Alternatives

  • We need to keep up with IDF releases, so there's not much alternative. And this is only a patch release update, so really shouldn't have many semantic changes.
  • Instead of trying to hunt down why the BLE event ordering has changed, I just opted to tweak the test.

Copy link

codecov bot commented Jul 20, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.44%. Comparing base (17fbc5a) to head (d7ba279).

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #17724   +/-   ##
=======================================
  Coverage   98.44%   98.44%           
=======================================
  Files         171      171           
  Lines       22208    22208           
=======================================
  Hits        21863    21863           
  Misses        345      345           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link

Code size report:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:    +0 +0.000% standard
      stm32:    +0 +0.000% PYBV10
     mimxrt:    +0 +0.000% TEENSY40
        rp2:    +0 +0.000% RPI_PICO_W
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:    +0 +0.000% VIRT_RV32

Copy link
Contributor

@projectgus projectgus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

dpgeorge added 2 commits July 23, 2025 12:52
This is a patch release of the IDF.  Comparing with 5.4.1, firmware size is
up by about 1.5k on ESP32 and 9k on ESP32-S3.  But IRAM usage (of the IDF)
is down by about 500 byte on ESP32 and DRAM usage is down by about 20k on
ESP32 and 10k on ESP32-S3.

Testing on ESP32, ESP32-S2, ESP32-S3 and ESP32-C3 shows no regressions,
except in BLE MTU ordering (the MTU exchange event occuring before the
connect event).

Signed-off-by: Damien George <damien@micropython.org>
With the recent update to ESP-IDF 5.4.2, there is a change in BLE event
behaviour which makes `tests/multi_bluetooth/ble_mtu.py` and
`tests/multi_bluetooth/ble_mtu_peripheral.py` now fail on ESP32 with IDF
5.4.2.

The change in behaviour is that MTU_EXCHANGE events can now occur before
CENTRAL_CONNECT/PERIPHERAL_CONNECT events.  That seems a bit strange,
because the MTU exchange occurs after the connection.  And looking at the
timing of the events there is exactly 100ms between them, ie MTU_EXCHANGE
fires and then exactly 100ms later CENTRAL_CONNECT/PERIPHERAL_CONNECT
fires.

It's unknown if this is a bug in (Espressif's) NimBLE, a subtle change in
scheduling with still valid behaviour, an intended change, a change allowed
under the BLE spec, or something else.

But in order to move forward with updating to IDF 5.4.2, the relevant tests
have been adjusted so they can pass.  The test just needs to wait a bit
between doing the connect and doing the MTU exchange, so the other side
sees the original/correct ordering of events.  This wait is done using the
multitest synchronisation primitives (broadcast and wait).

Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge dpgeorge force-pushed the esp32-update-idf-542 branch from d7ba279 to c6423d5 Compare July 23, 2025 02:56
@dpgeorge dpgeorge merged commit c6423d5 into micropython:master Jul 23, 2025
68 of 69 checks passed
@dpgeorge dpgeorge deleted the esp32-update-idf-542 branch July 23, 2025 02:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy