Sigh. It's not my commit message that's failing the test!

Wow, thanks for this! It's very nice to see that it can be implemented in pure Python. I don't know anything about TaskGroup so will need to learn how it works.

It's not my commit message that's failing the test!

You're right. That CI check will need fixing so it allows "Revert" commit messages.

It's not my commit message that's failing the test!

You're right. That CI check will need fixing so it allows "Revert" commit messages.

Actually the CI check is working OK, it's only supposed to check the commit messages that are unique to the PR.

If you rebase (and force push) on latest master it will hopefully get the CI green.

Pushed. Writing tests, found a significant bug (related to cancellation of course). Investigating.

Huh. I have no idea where those test failures are coming from.

The CI logs say error is:

+micropython-coverage: ../../py/emitnative.c:2806: emit_native_raise_varargs: Assertion `n_args == 1' failed.

It looks like the bytecode emitter allows 2 args but the native emitter only allows 1.

It looks like the bytecode emitter allows 2 args but the native emitter only allows 1.

Well, yeah, I saw that line, but what has that got to do with implementing __hash__ and/or how do I fix that? I didn't touch no emitters … so either I'm doing something really dumb here, or I found a bug which I'm not qualified to fix, esp. since the code works on my system(s).

Assuming you are using the unix port, to reproduce locally, try make -C ports/unix test_full or run micropython with -X emit=native.

There are a number of tests that are disabled when emit=native because of raise_varargs already, basics/try_reraise.py, basics/try_reraise2 and basics/try_finally_return2.py to name a few (search for raise_varargs in tests/run-tests.py).

I suspect you have written some Python code similar to one of these that causes the same error.

I'm running this on a Pyboard 1.1 - TaskGroups are a very nice feature.

Something Google seems loath to divulge is how to access return values from the members of a group (assuming they terminate normally).

@peterhinch Glad that my code is of use. I'll try to fix up the CI errors shortly; battling with Covid aftereffects, so if somebody else wants to fix up this PR, feel free.

You need to return any values explicitly, e.g. by setting an object attribute to it or queuing the value or whatever.

Sorry you're unwell - I hope you feel better soon.

Writing tests, found a significant bug (related to cancellation of course). Investigating.

Is this still outstanding or is the code ready for review?

I very much hope this is implemented: in many applications it's a big improvement over gather.

Is this still outstanding or is the code ready for review?

No, that's done. The workaround I implemented admittedly isn't particularly clean, but it gets the job done and doesn't alter non-taskgroup code (as that might introduce bugs or slowdowns).

@dpgeorge @jimmo This is to advocate for TaskGroup with a real world example where they fit the bill perfectly.

Consider a communication link between two peers using an unreliable medium such as WiFi. This may be near the limit of range and the link can fail in a variety of ways. The simplest way to recover is the "belt and braces" approach: if a peer encounters an unrecoverable error, it takes the link down for a period long enough for the other peer also to suffer an unrecoverable error. That way both peers start from a "power up" state.

A TaskGroup might then comprise:

• Task trying to keep the WiFi available through brief outages.
• Task waiting on received messages.
• Task pinging the other peer and checking a response.
• Intermittently a task is added to the group. It sends a message, awaits an ack, and terminates.

If any of these experience an error that cannot be handled locally, the exception is passed up to be handled by the task that created the group. When this occurs, every task in the group terminates in an orderly way, running cleanup code in finally clauses or via context managers.

The TaskGroup is ideal for this - and in my testing works fine :) I rest my case...

@peterhinch Exactly. Unstructured tasks (like asyncio's baseline tasks) mean that you have to keep track of what's still running and what might have to be cancelled and/or restarted on your own. Code doing that tends to contain a heap of bugs you can't test for, and basically doesn't scale.

In contrast, when a taskgroup's async context manager has ended you know that there's no dangling tasks, unprocessed finally clauses, unreported exceptions, or similar nonsense around; you can proceed with confidence. When (not if) the framework ensures that whole classes of errors can't happen, you no longer have to test for them, or even think about them.

The latter point is much more powerful than you'd assume at first glance.

Yes I know this is one of >300 open PRs … nevertheless, I'd appreciate feedback WRT this patch's mergeability.

Once again I try and use this, convinced that it was merged already, only to crash and burn 🙈

Having task groups in CPython may have ruined me…

Yeah. Sigh. I'll do a re-merge shortly.

Ugh. Don't ask me what this CI error means …

Updated to current master.

The CPython discussion on what to do about the Taskgroup.cancel method has stalled for now (and mainly centered on the name of the feature, as opposed to how it's supposed to work). That doesn't surprise me because the semantics of cancellation are somewhat murky in asyncio, but as cancelling all tasks in a taskgroup is something reasonable to do …

+mpy-cross: ../py/emitnative.c:3010: emit_native_raise_varargs: Assertion n_args == 1' failed.`

Umm, that used not to happen AFAIR?

It should be that the scheduler always iterates the singleton immediately

Well, it appears not to. In any case, by its nature async code is async: nothing prevents me from calling a bare asyncio.sleep and then doing something else before using the result in an await. Thus we shouldn't depend on that whether or not it's supposed to apply in the standard case.

    async def not_too_fast(delay, proc):
        dly = asyncio.sleep(delay)
        res = await proc()
        await dly
        return res

In MicroPython this is a nicely concise way of saying "give me the result of proc in exactly two seconds, except when proc takes longer than that`. (This doesn't work the same way on CPython, but not by design AFAICT.)