Skip to content

asyncio: fix import #7267

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

asyncio: fix import #7267

wants to merge 1 commit into from

Conversation

DrTom
Copy link

@DrTom DrTom commented May 15, 2021

fixes #7266

@dlech
Copy link
Contributor

dlech commented May 15, 2021

FYI, the reasoning behind the current implementation was discussed at #5332 (review)

@DrTom
Copy link
Author

DrTom commented May 16, 2021

@dlech: as far as I understand the discussion is around problems for IDE tooling etc. But what I ran into is a real bug. As far as I understand the lazy import was predominantly done to save memory. But I have not seen any evidence/numbers how much good that does in reality. In my personal experience I have not seen problems from overhead of imports in frozen code. There are other "things" which are much more likely to result in memory problems.

So where does that leave us with this bug? I can see two options: apply the fix or write up some documentation how to work around the issues.

@kevinkk525
Copy link
Contributor

Yes the main goal was memory saving (afaik). There is no need for absolute numbers in my opinion but you can easily check. It saves some bytes for every module you don't use. On most platforms this might be irrelevant and even on an esp8266 I would be able to live with 500Bytes less but why waste RAM?

I can see that the lazy import doesn't work in your use-case but in my opinion that's an acceptable problem because your use-case isn't a recommended way of programming either:

See pep8:

Wildcard imports (from <module> import *) should be avoided, as they make it unclear which names are present in the namespace, confusing both readers and many automated tools. There is one defensible use case for a wildcard import, which is to republish an internal interface as part of a public API (for example, overwriting a pure Python implementation of an interface with the definitions from an optional accelerator module and exactly which definitions will be overwritten isn't known in advance).

@DrTom
Copy link
Author

DrTom commented May 16, 2021

@kevinkk525 There is a good reason why I need the * import in my case.

When it comes to premature optimization I am all for numbers to do informed decisions. Here are some for the ESP32 Wroom:

>>> import uasyncio
>>> gc.collect(); gc.mem_free()
107776
>>> import uasyncio.stream
>>> gc.collect(); gc.mem_free()
106848
>>> import uasyncio.funcs
>>> gc.collect(); gc.mem_free()
106672
>>> import uasyncio.event
>>> gc.collect(); gc.mem_free()
106080
>>> import uasyncio.lock
>>> gc.collect(); gc.mem_free()
105744

A typical use case is to pull in stream and funcs which leaves us at a saving of less than 1k or about 0.8%. So we have to weight this against unnecessary incompatibilities vs CPython.

@kevinkk525
Copy link
Contributor

kevinkk525 commented May 16, 2021

Well if you have a good reason (please tell us), then you could simply do the exports differently because you'll likely be the only one to use a wildcard import on uasyncio.
So you could just as well do something like

from uasyncio import *
from uasyncio.stream import *
from uasyncio.funcs import *

Or is there a reason not to do that?

I see a savings of almost 2k if no stream,funcs,event or lock is being used. That is a significant amount. Maybe not on your esp32 wroom and in your project but overall, 2k is quite something.

Of course I'm just another developer. Personally I just don't see a need here to fix a problem that maybe 0.8% of all users might encounter and can easily work around, while giving everyone else a disadvantage of up to 2k more RAM usage.

@DrTom
Copy link
Author

DrTom commented May 17, 2021

@kevinkk525

while giving everyone else

False! Correct: those importing asyncio but not the asyncio.stream.

@DrTom
Copy link
Author

DrTom commented May 17, 2021

@kevinkk525 I believe discussions about workarounds should be made in the corresponding issue. But it is anyways trivial and really no need for that.

Costly it is for others to run into a bug and find the reason. This PR is about that. No more no less. The disadvantages in numbers are listed above. The maintainers should be able to decide whether to include the PR or reject it.

@peterhinch
Copy link
Contributor

Bear in mind that uasyncio is used on ESP8266 where every byte of RAM counts.

@dpgeorge dpgeorge added the extmod Relates to extmod/ directory in source label May 18, 2021
@dpgeorge
Copy link
Member

I think the right way to solve this is to fix from uasyncio import * so that it actually works. I think this can be done by implementing __all__ for modules.

Thanks for the PR and discussion!

@dpgeorge dpgeorge closed this Jun 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
extmod Relates to extmod/ directory in source
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Import / AttributeError error in new asyncio
5 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy