Skip to content

gh-137026: Add an explainer guide for asyncio #137215

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 116 commits into
base: main
Choose a base branch
from

Conversation

anordin95
Copy link

@anordin95 anordin95 commented Jul 29, 2025

Explainer guide for asyncio

gh-137026: HOWTO article for asyncio, and a reference to it from the main page of the asyncio docs.


Hi!

I've used Python's asyncio a couple times now, but never really felt confident in my mental model of how it fundamentally works and therefore how I can best leverage it. The official docs provide good documentation for each specific function in the package, but, in my opinion, are missing a cohesive overview of the systems design and architecture. Something that could help the user understand the why and how behind the recommended patterns. And a way to help the user make informed decisions about which tool in the asyncio toolkit they ought to grab, or to recognize when asyncio is the entirely wrong toolkit. I thought I'd take a stab at filling that gap and contributing back to a community that's given so much!


📚 Documentation preview 📚: https://cpython-previews--137215.org.readthedocs.build/en/137215/howto/a-conceptual-overview-of-asyncio.html

@python-cla-bot

This comment was marked as resolved.

@bedevere-app bedevere-app bot added docs Documentation in the Doc dir skip news labels Jul 29, 2025
@github-project-automation github-project-automation bot moved this to Todo in Docs PRs Jul 29, 2025
@anordin95 anordin95 changed the title - Add an explainer guide for asyncio. Add an explainer guide for asyncio [gh-137026](https://github.com/python/cpython/issues/137026) Jul 29, 2025
@anordin95 anordin95 changed the title Add an explainer guide for asyncio [gh-137026](https://github.com/python/cpython/issues/137026) Add an explainer guide for asyncio [gh-137026] Jul 29, 2025
@AA-Turner AA-Turner changed the title Add an explainer guide for asyncio [gh-137026] gh-137026: Add an explainer guide for asyncio Jul 29, 2025
Copy link
Member

@StanFromIreland StanFromIreland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will review further once the lines are wrapped, as now it will invalidate suggestions.

Copy link
Member

@ZeroIntensity ZeroIntensity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to commit to reviewing this if nobody else has the time. I think is generally an improvement to the documentation, but definitely needs some work still. Thanks for working on this!

Copy link
Member

@StanFromIreland StanFromIreland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are still many minor infringements of the Style guide (e.g. simple English) which could be addressed in one pass.

You can also use Sphinx cross references when discussing specific keywords, functions etc.

@anordin95
Copy link
Author

anordin95 commented Aug 8, 2025

Can we continue throughout the doc to deemphasize queue?

Yep! Was just taking a little break. All references to queue besides the one in that first analogy are now gone.

@anordin95
Copy link
Author

anordin95 commented Aug 8, 2025

@ZeroIntensity, @willingc: It should be noted, I just added a blurb about Task garbage collection at the end of the "tasks" section. I was initially on the fence but I think it's important, useful context to include.

It's important to be aware that the task itself is not added to the event loop,
only a callback to the task is.
This matters if the task object you created is garbage collected before it's
called by the event loop.
For example, consider this program::
async def hello():
print("hello!")
async def main():
hello_task = asyncio.create_task(hello())
return
asyncio.run(main())
Because the coroutine ``main()`` exits before awaiting the task and no other
references to the task are made, the task object ``hello_task`` *might* be
garbage collected before the event loop invokes it.
That example still actually ends up running ``hello_task``, because
``asyncio`` and Python's garbage collector work pretty hard to ensure this
sort of thing doesn't happen.
But that's no reason to be reckless!

That example highlights how using only ``await coroutine`` could
unintentionally hog control from other tasks and effectively stall the event
loop.
:func:`asyncio.run` can help you detect such occurences with the ``debug=True``
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To clarify, the way I'd expect most developers to use this is not by modifying the code, but by running python in dev mode locally (-X dev or PYTHONDEVMODE=1). A link should take users to: https://docs.python.org/3/library/asyncio-dev.html#asyncio-debug-mode

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the link 👍

Hm. I agree but that's effectively a shortcut. Pedagogically, I think it's clearer to mention the debug=True flag.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm. I agree but that's effectively a shortcut. Pedagogically, I think it's clearer to mention the debug=True flag.

If you say so. I didn't even know that parameter existed and can't imagine ever using it. All I need to know as a user is that this feature exists and is enabled as part of dev mode..

Once it pauses or completes, it returns control to the event loop.
The event loop will then select another job from its pool and invoke it.
You can *roughly* think of the collection of jobs as a queue: jobs are added and
then processed one at a time, generally (but not always) in order.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

generally (but not always) in order.

I still find this bit awkward as it's again suggesting that order of execution is unpredictable. Can we say that the jobs are processed in the order that they were scheduled or something?

Copy link
Author

@anordin95 anordin95 Aug 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm. If there's phrasing you have in mind, that the folks who are wary of the queue analogy would also be on board with, I'm open to it!

anordin95 and others added 2 commits August 8, 2025 10:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Todo
Development

Successfully merging this pull request may close these issues.

9 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy