Define `vfs.rom_ioctl()`, add romfs commands to mpremote, and implement ROM partition support on stm32, rp2, esp32, esp8266 #16857

dpgeorge · 2025-03-04T02:31:03Z

Summary

This is a continuation of #8381 to add general ROMFS support to the ports.

So far, we have the vfs.VfsRom filesystem format defined and driver implemented. That was merged via #16446. That allows creating and mounting a ROMFS filesystem if you have the ROMFS image in memory somewhere.

But there's not yet any way to use the ROMFS on the ports, because they don't have a way to get the ROMFS image on them, or access it.

This PR adds support for ROMFS on stm32, rp2, esp32 and esp8266.

I wanted to keep this PR relatively self-contained and minimal (although it's still rather large), so it's easier to review. Then further work can be done later on, eg to add support to more ports. PR #8381 does contain implementations of the remaining ports.

This PR has the following:

Defines a new function vfs.rom_ioctl() which is used as a general interface to access the read-only memory of a device. It can query, erase and write this memory.
A general helper function to mount a ROMFS at /rom in the filesystem.
When MICROPY_VFS_ROM is enabled, there is automatic mounting of the first ROM partition (if found using vfs.rom_ioctl) at the end of the call to mp_init(), so that all ports behave the same way. This also makes it much easier to enable the ROMFS feature, without adding anything to _boot.py (which is hard to do conditionally).
New mpremote commands which use vfs.rom_ioctl() to query, build and deploy ROMFS images.
Documentation for the new mpremote commands.
Implementations of vfs.rom_ioctl() in the following ports: stm32, rp2, esp32, esp8266.
PYBD_SF2/3/6 boards have a ROMFS partition defined, using previously inaccessible QSPI flash.
A new esp8266 board variant is defined for 2M+ flash parts that adds a 320k ROMFS partition.

Note that no other boards have a ROMFS partition enabled by default, although it's possible to enable it yourself by modifying the board definition (or creating your own board definition with it enabled). The reason for not enabling it by default on more boards is because it's a breaking change for the user, since it allocates a lot of flash to the ROMFS and takes it away from the firmware and possible frozen code. In the future we can work out how to migrate all boards to have a ROMFS, but for now that's left as an unsolved problem.

Testing

Tested on PYBD_SF2, PYBD_SF6, esp8266 using the new variant.

Also tested on rp2 and esp32 boards by modifying the board definition as described in the commit messages for those ports.

Trade-offs and Alternatives

This is a big new feature! It's hard to get big things right first go so that's why I wanted to take smaller steps to add independent parts at a time. Also thanks to @projectgus for advice in this regard.

For now ROMFS is an advanced feature and most users would need to modify board definitions, or create custom ones, to use it. Alternatives would be to just enable it on all boards and provide some docs for users freezing a lot of code to migrate to the new system. Or, postpone this feature even longer until we can work out how to better configure the ROMFS partitions. But that will delay things even more and this feature has been in the works for years now.

codecov · 2025-03-04T02:35:09Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.54%. Comparing base (6be7570) to head (be0fce9).
Report is 12 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master   #16857   +/-   ##
=======================================
  Coverage   98.54%   98.54%           
=======================================
  Files         169      169           
  Lines       21864    21877   +13     
=======================================
+ Hits        21545    21558   +13     
  Misses        319      319

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

github-actions · 2025-03-04T02:46:03Z

Code size report:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:   +16 +0.002% standard
      stm32:    +0 +0.000% PYBV10
     mimxrt:    +0 +0.000% TEENSY40
        rp2:   +32 +0.003% RPI_PICO_W
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:    +0 +0.000% VIRT_RV32

projectgus

This is neat! I was able to play around with it on esp8266 quite easily, and the deploy cycle looks like it should be pretty fast unless you have a ton of .mpy files!

I left some comments but they're all relatively minor things, I think could address in a follow-up if you want to merge this sooner.

projectgus · 2025-03-04T03:52:08Z

docs/reference/mpremote.rst

@@ -347,6 +348,29 @@ The full list of supported commands are:
  This happens automatically when ``mpremote`` terminates, but it can be used
  in a sequence to unmount an earlier mount before subsequent command are run.

+.. _mpremote_command_romfs:
+
+- **romfs** -- manage ROM partitions on the device:


Suggested change

- **romfs** -- manage ROM partitions on the device:

- **romfs** -- manage ROMFS partitions on the device:

.. note:: This feature is currently experimental and only enabled on some ports and boards, via the `MICROPY_VFS_ROM` build flag.

Related, I noticed in some of the mpremote error messages that the terms "ROM partition" and "ROMFS partition" are used interchangeably. Is there a meaningful distinction for the user? As "ROM" is already a very overloaded term then it might be clearer to only talk about "ROMFS" and "ROMFS partition",

Good catch, thanks.

I forgot to mention this in the PR description. I did want to have some distinction between ROM being the place in memory and ROMFS being the thing that goes in that place.

Because maybe in the future the ROM (partitions) can be used for something else, not just putting a ROMFS in them. Well even now that's possible, the vfs.rom_ioctl() call is essentially giving the user a place to store things, that's non-volatile and can be mapped to memory. (It's kind of adding a cross-port way of accessing internal flash, like pyb.Flash and rp2.Flash etc already do.) And that's why I called it rom_ioctl() and not romfs_ioctl(). Because this function call has nothing to do with ROMFS (it's at a layer lower).

But as you say, ROM is quite overloaded, so maybe it's worth calling everything ROMFS even if it's not used for a VfsRom filesystem?

Maybe then the function should indeed be called vfs.romfs_ioctl()?

I think rom_ioctl() is OK, my comment was more around standardising the docs and help strings so there's no confusion about whether "ROM partition" and "ROMFS partition" are different types of things.

I've now changed all "ROM" to "ROMFS". There's now a distinction between "ROMFS partition" and "ROMFS image" and it's used consistently in the docs and mpremote.

projectgus · 2025-03-04T03:54:27Z

ports/qemu/main.c

@@ -77,3 +77,7 @@ void nlr_jump_fail(void *val) {
    mp_printf(&mp_plat_print, "uncaught NLR\n");
    exit(1);
 }
+
+mp_obj_t mp_vfs_rom_ioctl(size_t n_args, const mp_obj_t *args) {
+    return MP_OBJ_NEW_SMALL_INT(-MP_EINVAL);


Suggested change

return MP_OBJ_NEW_SMALL_INT(-MP_EINVAL);

return MP_OBJ_NEW_SMALL_INT(-MP_ENODEV);

... maybe nitpicky, I don't know if this is worth distinguishing? That's the closest OS error code I can see for NotImplementedError.

I was thinking that EINVAL means invalid ioctl.

But probably it should at least implement MP_VFS_ROM_IOCTL_GET_NUMBER_OF_SEGMENTS and return 0 for that.

Then the rest of the ioctls are invalid and return -MP_EINVAL.

?

Edit: changed MP_ENODEV to MP_EINVAL in the last sentence above, that's what I intended to write.

I guess my thinking was more:

This isn't implemented at all - MP_ENODEV

This is implemented but the ioctl you asked for is invalid here - MP_EINVAL

But I think it's fuzzy enough to use MP_EINVAL everywhere, too.

This has been resolved by adding a MICROPY_VFS_ROM_IOCTL config option and disabling that on qemu and unix, so this function is not even needed anymore.

But then I did add it on unix for the coverage build, in a separate commit here.

projectgus · 2025-03-04T03:56:37Z

ports/unix/main.c

@@ -803,3 +804,7 @@ void nlr_jump_fail(void *val) {
    fprintf(stderr, "FATAL: uncaught NLR %p\n", val);
    exit(1);
 }
+
+mp_obj_t mp_vfs_rom_ioctl(size_t n_args, const mp_obj_t *args) {
+    return MP_OBJ_NEW_SMALL_INT(-MP_EINVAL);


Suggested change

return MP_OBJ_NEW_SMALL_INT(-MP_EINVAL);

return MP_OBJ_NEW_SMALL_INT(-MP_ENODEV);

(Same caveat as above.)

Resolved as above.

projectgus · 2025-03-04T03:58:38Z

tools/mpremote/mpremote/commands.py

+    state.transport.exec("import vfs")
+    num_rom_partitions = state.transport.eval("hasattr(vfs, 'rom_ioctl') and vfs.rom_ioctl(1)")
+    if num_rom_partitions is False:
+        print("No ROM partitions available")


Is it possible to distinguish between "No ROM partitions available" and "ROMFS support is not enabled in this MicroPython build"? IIUC on some ports there is is a meaningful difference, which effects what a user would do if they want to use romfs.

Also, failure here and elsewhere in this function should probably result in a non-zero process exit code for scripting use.

Is it possible to distinguish between "No ROM partitions available" and "ROMFS support is not enabled in this MicroPython build"?

Yes, will do that. ROMFS is supported if vfs.rom_ioctl() exists. Then can use rom_ioctl(1) to get the number of partitions.

Also, failure here and elsewhere in this function should probably result in a non-zero process exit code for scripting use

This particular one is the result of romfs query, which should just print out info, not have an error.

mpromet now distinguishes between rom_ioctl existing or not, and a ROMFS partition being there or not.

projectgus · 2025-03-04T04:21:14Z

tools/mpremote/mpremote/commands.py

+    elif args.command[0] == "deploy":
+        _do_romfs_deploy(state, args)
+    else:
+        raise CommandError(f"romfs: '{args.command[0]}' is not a command")


Suggested change

raise CommandError(f"romfs: '{args.command[0]}' is not a command")

raise CommandError(f"romfs: '{args.command[0]}' is not a command. Pass romfs --help for a list.")

projectgus · 2025-03-04T04:22:40Z

tools/mpremote/mpremote/main.py

+        "-o",
+        help="output file",
+    )
+    cmd_parser.add_argument("command", nargs=1, help="romfs command (e.g. query, build, deploy)")


Suggested change

cmd_parser.add_argument("command", nargs=1, help="romfs command (e.g. query, build, deploy)")

cmd_parser.add_argument("command", nargs=1, help="romfs command. Supported commands: query, build, deploy.")

(As this is the only place the tool actually prints the list of accepted commands, I think.)

The other way to improve this would be to add another argparse sub-command parser so we get generated list of commands and their associated options, but that seems like something that can be done later when there are more sub-commands.

I struggled a bit with this sub-command parsing. Was trying to copy how fs <subcommand> works.

Help message adjusted.

projectgus · 2025-03-04T04:28:59Z

tools/mpremote/mpremote/main.py

@@ -228,6 +229,26 @@ def argparse_mip():
    return cmd_parser


+def argparse_romfs():
+    cmd_parser = argparse.ArgumentParser(description="manage ROM partitions")
+    _bool_flag(cmd_parser, "mpy", "m", True, "download as compiled .mpy files (default)")


Suggested change

_bool_flag(cmd_parser, "mpy", "m", True, "download as compiled .mpy files (default)")

_bool_flag(cmd_parser, "mpy", "m", True, "Automatically compile .py files to .mpy when buliding the ROMFS (default)")

(Suggested rewording as the verb "download" is unclear in this context.)

Reworded as suggested.

projectgus · 2025-03-04T04:42:37Z

extmod/vfs.c

+        mp_obj_t romfs = mp_call_function_1(MP_OBJ_FROM_PTR(&mp_type_vfs_rom), rom);
+        mp_obj_t mount_point = MP_OBJ_NEW_QSTR(MP_QSTR__slash_rom);
+        mp_call_function_2(MP_OBJ_FROM_PTR(&mp_vfs_mount_obj), romfs, mount_point);
+        mp_obj_list_append(mp_sys_path, MP_OBJ_NEW_QSTR(MP_QSTR__slash_rom_slash_lib));


I got caught out that I couldn't import a Python module from the top-level of the rootfs without doing sys.path.append('/rom'). How about adding both /rom and /rom/lib to sys.path by default?

(I'm thinking along the lines that the top-level is the application code either as .py files or package directories, and /rom/lib is managed by some future variant of 'mip install'.)

I don't really see a downside - I guess someone might decide to put a lot of non-Python assets in the top level and slow down import a little, but that seems like a much less common pattern than putting a Python file there.

Adding to the search path, ie sys.path, really does slow down imports. Because you need to stat .py and .mpy for each import (assuming the first fails), and then for packages you also need to stat __init__.py and __init__.mpy. This is something we need to optimise in general.

So, I really wanted to keep sys.path down to a minimum. I did start out adding just /rom to the path, but after discussion with you and Jim, changed that to /rom/lib.

The user can easily change the path themselves.

Note that neither boot.py nor main.py will execute from the ROMFS, because those boot up files are searched in the current directory.

I'd really like to improve all this in MicroPython 2.0. But for now... maybe just keep it as /rom/lib and document it? Or change it to '/rom`? Or just omit altogether and make the user add it as they need?

Side note: mip install won't work with /rom/lib in the path before /flash/lib, because it tries to install into the ROMFS...

I guess a key question is what behaviour we want for (eventually) installing MIP packages into the ROMFS. I'd figured that it's desirable to keep them separated somewhere, so it's easy for the developer to see the difference between "my application" and "libraries I've installed from elsewhere". So my thinking was that /rom/lib would be the natural place to put the externa llibraries, as it mirrors the way it works now.

I don't think there's any way to have this distinction and only add one entry to the path, though...

Or just omit altogether and make the user add it as they need?

On balance I think it's preferable to have helpful/usable defaults. Then the developer who wants to optimise their package import speed can be the one who tweaks sys.path to remove entries they don't need.

(But if we're going with one default entry only, probably /rom is less surprising for people building ROMFS from a directory and then using it on their board.)

On balance I think it's preferable to have helpful/usable defaults. Then the developer who wants to optimise their package import speed can be the one who tweaks sys.path to remove entries they don't need

Yes, that's a good point. So maybe then the defaults can be /rom and /rom/lib, which is how we recommend to layout a project. Then users can tweak as needed.

I'll do a few benchmarks to see if adding a second path makes much of a difference.

Both /rom and /rom/lib are now added automatically to sys.path.

dpgeorge · 2025-03-04T12:28:09Z

I added some coverage testing to get CI to pass.

robert-hh · 2025-03-04T13:18:25Z

@dpgeorge Is this PR an addition to #8381 or a replacement. It looks more like the latter.

dpgeorge · 2025-03-04T13:49:29Z

Is this PR an addition to #8381 or a replacement.

I took #8381 and pulled out some things, then cleaned things up, to get this PR.

After this is merged I will rebase/rework #8381, which will add all the remaining functionality (but probably not until after 1.25.0 release).

So this PR is not really in addition to or a replacement. It's more of a precursor.

iabdalkader · 2025-03-04T15:05:55Z

I'm also a bit confused by this PR. Will ROMFS work if only this PR gets merged in 1.25.0? If so, what will be missing? Also, in #8381 I sent commits to enable ROMFS for Arduino boards, is it possible to cherry-pick them for inclusion in 1.25.0? They're pretty much the same as PYBD_SFx boards.

robert-hh · 2025-03-04T15:13:33Z

I've built an RP2 firmware with this PR and VFSROM was available. So it's in a working state. I have to check whether the drivers for SAMD, MIMXRT and NRF work. They may need some tuning.

robert-hh · 2025-03-04T16:46:57Z

@iabdalkader As a test I added a subset of the driver changes for SAMD51 to this PR just for one SAMD51 board, and VFSROM is available. That makes me confident that adapting these is straightforward. Thanks, Damien.

iabdalkader · 2025-03-04T16:51:13Z

@robert-hh Thanks for testing! I was hoping mimxrt would have romfs in 1.25.0 but that's okay. I hope that my Arduino boards commits make it through.

robert-hh · 2025-03-04T19:16:30Z

In addition I just cherry-picked the three commits from PR #8381. That works. Only the path setting done in _boot.py seems to get lost. It seems that these lines are not needed any more.
Edit: Just picking all SAMD related commits works as well.

dpgeorge · 2025-03-05T00:50:54Z

I ran some tests to time how long import takes when /rom is added to sys.path (in addition to /rom/lib).

Below are all the stat'ing that py/builtinimport.c does during an import of a single module. The second column is the time in microseconds taken for the stat. The third column is the path that is stat'd.

The test does import time, which must search the filesystem before falling back to the built-in time module.

PYBD_SF2

sys.path is:

['', '.frozen', '/rom', '/rom/lib', '/flash', '/flash/lib']

With an empty ROMFS (exists but has no files in it).

stat   664 time
stat   681 time.py
stat   682 time.mpy
stat    28 .frozen/time
stat    14 .frozen/time.py
stat    27 .frozen/time.mpy
stat     4 /rom/time
stat     4 /rom/time.py
stat     3 /rom/time.mpy
stat     3 /rom/lib/time
stat     4 /rom/lib/time.py
stat     4 /rom/lib/time.mpy
stat   681 /flash/time
stat   669 /flash/time.py
stat   669 /flash/time.mpy
stat   551 /flash/lib/time
stat   549 /flash/lib/time.py
stat   551 /flash/lib/time.mpy

With a ROMFS with 25 files in /rom and 10 files in /rom/lib:

stat   662 time
stat   684 time.py
stat   683 time.mpy
stat    28 .frozen/time
stat    20 .frozen/time.py
stat    14 .frozen/time.mpy
stat    64 /rom/time
stat    42 /rom/time.py
stat    40 /rom/time.mpy
stat    65 /rom/lib/time
stat    56 /rom/lib/time.py
stat    55 /rom/lib/time.mpy
stat   684 /flash/time
stat   683 /flash/time.py
stat   684 /flash/time.mpy
stat   548 /flash/lib/time
stat   535 /flash/lib/time.py
stat   550 /flash/lib/time.mpy

ESP8266

sys.path is:

['', '.frozen', '/rom', '/rom/lib', '/lib', '/']

With an empty ROMFS:

stat  1125 time
stat  1102 time.py
stat  1036 time.mpy
stat   148 .frozen/time
stat   137 .frozen/time.py
stat   138 .frozen/time.mpy
stat    17 /rom/time
stat     9 /rom/time.py
stat    10 /rom/time.mpy
stat    10 /rom/lib/time
stat     9 /rom/lib/time.py
stat    10 /rom/lib/time.mpy
stat  1934 /lib/time
stat  1831 /lib/time.py
stat  1832 /lib/time.mpy
stat  1100 //time
stat  1023 //time.py
stat  1068 //time.mpy

With a ROMFS with 25 files in /rom and 10 files in /rom/lib:

stat  1146 time
stat  1105 time.py
stat  1032 time.mpy
stat   152 .frozen/time
stat   138 .frozen/time.py
stat   138 .frozen/time.mpy
stat   124 /rom/time
stat    68 /rom/time.py
stat    70 /rom/time.mpy
stat   109 /rom/lib/time
stat    92 /rom/lib/time.py
stat    94 /rom/lib/time.mpy
stat  1897 /lib/time
stat  1833 /lib/time.py
stat  1833 /lib/time.mpy
stat  1100 //time
stat  1023 //time.py
stat  1068 //time.mpy

The tests above show that it's significantly faster to stat from ROMFS (which is good!) than from a FAT filesystem (and littlefs is about the same speed as FAT). Even searching frozen code can be slower than ROMFS (esp8266 had about 25 frozen modules, and it does a linear search through that list).

So I think we can safely add both /rom and /rom/lib to sys.path. The benefits to the user (things just work) outweigh the minor efficiency cost. And it's very easy for the user to customise their sys.path to optimise things, in particular it's best to remove the empty string entry from sys.path, at the very least.

dpgeorge · 2025-03-05T02:50:32Z

Will ROMFS work if only this PR gets merged in 1.25.0? If so, what will be missing?

ROMFS already can work because vfs.VfsRom is in the code base. Eg the tests for VfsRom work under CI (just need to enable MICROPY_VFS_ROM).

What this PR adds is (as mentioned in the top message) vfs.rom_ioctl() to erase/write the ROMFS partitions on a given board, and support in mpremote to create and deploy ROMFS images.

What's missing in this PR:

vfs.rom_ioctl support in nrf, samd, mimxrt and renesas-ra ports
enabling of ROMFS by default on all of the boards (it's only enabled on PYBD_SFx and a new esp8266 board variant)

So, that work is left for #8381.

I was hoping mimxrt would have romfs in 1.25.0 but that's okay.

We can add support for mimxrt in a follow up PR.

I hope that my Arduino boards commits make it through.

We can also enable it on those boards.

Only the path setting done in _boot.py seems to get lost. It seems that these lines are not needed any more.

This ROMFS mounting logic is now handled automatically (in C) when you call mp_init().

dpgeorge · 2025-03-05T03:01:16Z

I've updated based on all the feedback.

@iabdalkader I'll do a follow-up PR to enable ROMFS on the Arduino boards.

iabdalkader · 2025-03-05T07:27:34Z

I'll do a follow-up PR to enable ROMFS on the Arduino boards.

@dpgeorge Thanks! I checked and they're exactly the same as PYBD_SFx support, so that much hasn't changed.

This allows defining a `memoryview` instance, either statically or on the C stack. Signed-off-by: Damien George <damien@micropython.org>

This is a generic interface to allow querying and modifying the read-only memory area of a device, if it has such an area. Signed-off-by: Damien George <damien@micropython.org>

This function will attempt to create a `VfsRom` instance and mount it at location "/rom" in the filesystem. Signed-off-by: Damien George <damien@micropython.org>

This is put in `mp_init()` to make it consistent across all ports. Signed-off-by: Damien George <damien@micropython.org>

These commands use the `vfs.rom_ioctl()` function to manage the ROM partitions on a device, and create and deploy ROMFS images. Signed-off-by: Damien George <damien@micropython.org>

This commit implements `vfs.rom_ioctl()` to query, erase and write both internal and external flash, depending on how the board configures its flash memory. A board can configure ROM as follows. To use internal flash memory: #define MICROPY_HW_ROMFS_ENABLE_INTERNAL_FLASH (1) To use external flash memory (QSPI memory mapped): #define MICROPY_HW_ROMFS_ENABLE_EXTERNAL_QSPI (1) #define MICROPY_HW_ROMFS_QSPI_SPIFLASH_OBJ (&spi_obj) Then the partition must be defined as symbols in the linker script: _micropy_hw_romfs_part1_start _micropy_hw_romfs_part1_size And finally the partition needs to be enabled: #define MICROPY_HW_ROMFS_ENABLE_PART1 (1) There's support for a second, optional partition via: _micropy_hw_romfs_part2_start _micropy_hw_romfs_part2_size #define MICROPY_HW_ROMFS_ENABLE_PART1 (1) Signed-off-by: Damien George <damien@micropython.org>

Using unused and previously inaccessible external QSPI flash. Signed-off-by: Damien George <damien@micropython.org>

Not enabled by default on any board. A board can enable a ROMFS partition by defining `MICROPY_HW_ROMFS_BYTES` in its `mpconfigboard.h` file. For example: #define MICROPY_HW_ROMFS_BYTES (128 * 1024) The ROMFS partition is placed at the end of the flash allocated for the firmware, giving less space for the firmware. It then lives between the firmware and the read/write filesystem. Signed-off-by: Damien George <damien@micropython.org>

Not enabled by default on any board. For a board to enable ROMFS it must: - Add `#define MICROPY_VFS_ROM (1)` to its `mpconfigboard.h` file. - Use `partitions-4MiB-romfs.csv` as its partitions file (or a similar partitions definition that has an entry labelled "romfs"). Signed-off-by: Damien George <damien@micropython.org>

Not enabled by default on any board. For a board to enable ROMFS it must: - Add `#define MICROPY_VFS_ROM (1)` to its `mpconfigboard.h` file. - Add a FLASH_ROMFS partition to the linker script and expose the partition with: _micropy_hw_romfs_start = ORIGIN(FLASH_ROMFS); _micropy_hw_romfs_size = LENGTH(FLASH_ROMFS); Signed-off-by: Damien George <damien@micropython.org>

The same as the 2M flash variant but with a 320KiB ROM partition. Signed-off-by: Damien George <damien@micropython.org>

Signed-off-by: Damien George <damien@micropython.org>

robert-hh · 2025-03-06T08:18:58Z

Thank you for providing this feature. Should I make a PR for the MIMXRT, SAMD, NRF and RENESAS port support it, or will you collect these files from PR #8381?

dpgeorge · 2025-03-06T12:04:28Z

Should I make a PR for the MIMXRT, SAMD, NRF and RENESAS port support it, or will you collect these files from PR #8381?

I'll rebase #8381 on master and it'll support all those ports there, using the work you already did.

robert-hh · 2025-03-06T13:01:04Z

Wouldn't there be many merge conflicts?
Note: MIMXRT, SAMD and NRF mount /rom in _boot.py in #8381. That is not needed any more. Besides that, the ports build and work.

robert-hh · 2025-05-27T07:41:57Z

I'll rebase #8381 on master and it'll support all those ports there, using the work you already did.

What is the state of adding VFSROM to the MIMXRT, SAMD, NRF and RENESAS port?

dpgeorge requested a review from projectgus March 4, 2025 02:31

dpgeorge added port-esp8266 port-stm32 port-esp32 extmod Relates to extmod/ directory in source port-rp2 labels Mar 4, 2025

dpgeorge added this to the release-1.25.0 milestone Mar 4, 2025

projectgus approved these changes Mar 4, 2025

View reviewed changes

dpgeorge force-pushed the extmod-vfs-rom-ioctl branch from e6fa0fb to ffcc0f6 Compare March 4, 2025 12:12

dpgeorge force-pushed the extmod-vfs-rom-ioctl branch from ffcc0f6 to 7364a2d Compare March 5, 2025 02:25

dpgeorge added 6 commits March 6, 2025 12:52

py/objarray: Add MP_DEFINE_MEMORYVIEW_OBJ convenience macro.

9dd4cef

This allows defining a `memoryview` instance, either statically or on the C stack. Signed-off-by: Damien George <damien@micropython.org>

extmod/modvfs: Add vfs.rom_ioctl function and its ioctl constants.

89e6c58

This is a generic interface to allow querying and modifying the read-only memory area of a device, if it has such an area. Signed-off-by: Damien George <damien@micropython.org>

extmod/vfs: Add mp_vfs_mount_romfs_protected() helper.

d4b8ca2

This function will attempt to create a `VfsRom` instance and mount it at location "/rom" in the filesystem. Signed-off-by: Damien George <damien@micropython.org>

py/runtime: Automatically mount ROMFS as part of mp_init.

840b641

This is put in `mp_init()` to make it consistent across all ports. Signed-off-by: Damien George <damien@micropython.org>

tools/mpremote: Add romfs query, build and deploy commands.

0c98c60

These commands use the `vfs.rom_ioctl()` function to manage the ROM partitions on a device, and create and deploy ROMFS images. Signed-off-by: Damien George <damien@micropython.org>

dpgeorge added 6 commits March 6, 2025 12:52

stm32/boards: Enable ROMFS partitions on PYBD_SFx boards.

45c36f8

Using unused and previously inaccessible external QSPI flash. Signed-off-by: Damien George <damien@micropython.org>

esp8266/boards: Add FLASH_2M_ROMFS variant with 320k ROM partition.

6bec36a

The same as the 2M flash variant but with a 320KiB ROM partition. Signed-off-by: Damien George <damien@micropython.org>

unix/main: Add coverage test for mounting ROMFS filesystem at startup.

be0fce9

Signed-off-by: Damien George <damien@micropython.org>

dpgeorge force-pushed the extmod-vfs-rom-ioctl branch from 7364a2d to be0fce9 Compare March 6, 2025 01:55

dpgeorge merged commit be0fce9 into micropython:master Mar 6, 2025
67 checks passed

dpgeorge deleted the extmod-vfs-rom-ioctl branch March 6, 2025 02:46

dpgeorge mentioned this pull request Mar 6, 2025

stm32: On Arduino boards, enable 4MiB ROMFS partition in external flash #16869

Merged

exception13 mentioned this pull request May 6, 2025

ports/esp32: Added ESP32 ROMFS variant. #17256

Open

-- **romfs** -- manage ROM partitions on the device:
+- **romfs** -- manage ROMFS partitions on the device:
+.. note:: This feature is currently experimental and only enabled on some ports and boards, via the `MICROPY_VFS_ROM` build flag.

	return MP_OBJ_NEW_SMALL_INT(-MP_EINVAL);
	return MP_OBJ_NEW_SMALL_INT(-MP_ENODEV);

	raise CommandError(f"romfs: '{args.command[0]}' is not a command")
	raise CommandError(f"romfs: '{args.command[0]}' is not a command. Pass romfs --help for a list.")

	cmd_parser.add_argument("command", nargs=1, help="romfs command (e.g. query, build, deploy)")
	cmd_parser.add_argument("command", nargs=1, help="romfs command. Supported commands: query, build, deploy.")

	_bool_flag(cmd_parser, "mpy", "m", True, "download as compiled .mpy files (default)")
	_bool_flag(cmd_parser, "mpy", "m", True, "Automatically compile .py files to .mpy when buliding the ROMFS (default)")

Uh oh!

Define vfs.rom_ioctl(), add romfs commands to mpremote, and implement ROM partition support on stm32, rp2, esp32, esp8266 #16857

Define vfs.rom_ioctl(), add romfs commands to mpremote, and implement ROM partition support on stm32, rp2, esp32, esp8266 #16857

Uh oh!

Conversation

dpgeorge commented Mar 4, 2025

Summary

Testing

Trade-offs and Alternatives

Uh oh!

codecov bot commented Mar 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions bot commented Mar 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

projectgus left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dpgeorge Mar 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

projectgus Mar 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Define `vfs.rom_ioctl()`, add romfs commands to mpremote, and implement ROM partition support on stm32, rp2, esp32, esp8266 #16857

Define `vfs.rom_ioctl()`, add romfs commands to mpremote, and implement ROM partition support on stm32, rp2, esp32, esp8266 #16857

codecov bot commented Mar 4, 2025 •

edited

Loading

github-actions bot commented Mar 4, 2025 •

edited

Loading

dpgeorge Mar 4, 2025 •

edited

Loading

projectgus Mar 4, 2025 •

edited

Loading

iabdalkader commented Mar 4, 2025 •

edited

Loading

robert-hh commented Mar 4, 2025 •

edited

Loading