Saving RAM is a constant theme for people using micropython.

A super-simple action a user can take to save RAM is to use frozen bytecode but that's only possible if the .py is available to be compiled in with the firmware.

If that's not an option, the user can still save the .py to flash to at least not occupy RAM and simply pass the pointer-to-flash to mp_lexer_t *mp_lexer_new_from_str_len()

But this still leaves a huge source of (unnecessary) RAM consumption during the compilation process and the eventually output.

SUGGESTION: the memory used during the compilation process that is O(1), with respect to the size of the input, should remain in RAM but the O(N) stuff should be optionally placed in flash/nonvolatile memory.