py/{compile,runtime}: Fix *args after kwarg. #11441
Conversation
|
Code size report: |
|
Technically, this is a breaking change for the MPY ABI but since it is a bug, I'm not sure what to suggest about bumping the MPY version for this or not. The current compiler (released in v1.19 and v1.20) generates invalid bytecode which will will trigger undefined behavior in the interpreter both before and after the fix for the interpreter is applied. Correct bytecode generated after this fix is applied will trigger undefined behavior if run in in the old broken interpreter versions. Since the old broken interpreter has undefined behavior in any case, it makes this seem not strictly a breaking change. |
This fixes a compiler/runtime bug where *args after a kwarg was not handled correctly. Prior to this change, if `*args` was encountered in a function call after a keyword argument, the compiler would push a single object and increment the positional arg count. However, two objects for the keyword argument key and value had already been pushed. This caused inconsistencies that the runtime could not resolve since it expects all of the positional args first followed by key/value pairs for the keyword args. To fix it, we need to conditionally change what happens when `*args` is encountered depending on if it is before or after the first keyword argument. If it is before, everything is handled as before. If after, instead of pushing a single object and incrementing the positional arg count, we push two objects and increment the keyword arg count. This makes it possible for the runtime to handle it with minimal changes. In the runtime, we have to add some extra checks to handle the new case of the possibility that one of the `n_kw` args is a `*arg`. We already have a case where `**arg` is handled as a keyword argument where the key is `MP_OBJ_NULL`. We now do the same for `*arg` as well. The existing `star_args` flags is used to determine if the value corresponding to a key of `MP_OBJ_NULL` is `*arg` or `**arg`. A couple of test that failed before this fix are added. Fixes: micropython#11439 Signed-off-by: David Lechner <david@pybricks.com>
1faf1cc to
49cec95
Compare
Codecov Report
📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more @@ Coverage Diff @@
## master #11441 +/- ##
=======================================
Coverage 98.39% 98.39%
=======================================
Files 156 156
Lines 20603 20619 +16
=======================================
+ Hits 20272 20288 +16
Misses 331 331
|
|
Not sure why the macos test failed. It doesn't seem like it could possibly be affected by these changes and I can't reproduce locally on my Mac. |
|
I re-ran the macos CI (are you able to do that yourself, as the author of this PR?) and it now passes. |
No, that option is not available for me. |
|
In addition to Developer Certificate of Origin (Signed-off-by:) are we now also requiring commits to be cryptographically signed? |
That was not the intention. |
|
IMO the complexity added here to the runtime is too much. And honestly I think this syntax is bad and should not be used, one should always specify positional arguments before keyword arguments. @jimmo pointed out the following inconsistency with this syntax (running on CPython): >>> def f(a, b): pass
...
>>> # the following is an error
>>> f(b=1, 2)
File "<stdin>", line 1
f(b=1, 2)
^
SyntaxError: positional argument follows keyword argument
>>>
>>> # but the following is allowed
>>> f(b=1, *(2,))
>>> My suggestion would be that we just disallow this syntax, disallow Users can still write CPython compatible code: just put |
+1 I doubt there's a lot of code out there which uses this syntax |
This fixes a compiler/runtime bug where *args after a kwarg was not handled correctly.
Prior to this change, if
*argswas encountered in a function call after a keyword argument, the compiler would push a single object and increment the positional arg count. However, two objects for the keyword argument key and value had already been pushed. This caused inconsistencies that the runtime could not resolve since it expects all of the positional args first followed by key/value pairs for the keyword args.To fix it, we need to conditionally change what happens when
*argsis encountered depending on if it is before or after the first keyword argument. If it is before, everything is handled as before. If after, instead of pushing a single object and incrementing the positional arg count, we push two objects and increment the keyword arg count. This makes it possible for the runtime to handle it with minimal changes.In the runtime, we have to add some extra checks to handle the new case of the possibility that one of the
n_kwargs is a*arg. We already have a case where**argis handled as a keyword argument where the key isMP_OBJ_NULL. We now do the same for*argas well. The existingstar_argsflags is used to determine if the value corresponding to a key ofMP_OBJ_NULLis*argor**arg.A couple of test that failed before this fix are added.
Fixes: #11439