-
-
Notifications
You must be signed in to change notification settings - Fork 158
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement lazy loading of pygame submodules (surfarray
, sndarray
)
#3232
base: main
Are you sure you want to change the base?
Conversation
# lastly, the "optional" pygame modules | ||
|
||
_MissingModule = MissingModule |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it would make sense to store this as a local variable on __getattr__
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The MissingModule
class is deleted at the end of the module, and __getattr__
still needs it, so it must be stored somewhere (not as local variable).
A few options:
- Private alias of
MissingModule
- Move
MissingModule
to another file - Bind
MissingModule
as a default keyword argument in__getattr__
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here's an idea:
>>> a = 36
>>>
>>> def func():
... print(func.b)
...
>>>
>>> func.b = a
>>> del a
>>> func()
36
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well I suggested that before but ultimately opted for a private alias because function attributes are kind of weird and dynamic. From my reasoning, it would've look strange and I would rather avoid these rare python features.
If it is necessary to avoid exposing _MissingModule
in the pygame namespace, then sure, but I think the current approach is okay.
There's a block in here called Also I think this could be worth a small versionchanged or note in the docs. |
Also, for reviewer reference, the command to test this is |
@damusss uncovered a problem |
Motivation:
Previously, I discovered via profiling that
numpy
was being imported or loaded in my pygame program, when there were no mandatory runtime dependencies on it, though I had it installed.The pygame submodules
surfarray
andsndarray
both attempt to import numpy if possible, and pygame always tries to load these submodules. (If numpy is not available, they are replaced byMissingModule
.)When running
import pygame
,numpy
(if available) loading takes up at least 40% of the time, around 100 milliseconds depending on the hardware. However, thesurfarray
andsndarray
modules are not used in many pygame programs, and some users installnumpy
globally, or, like me, have dev dependencies that use numpy.Implementation:
To fix this,
numpy
shouldn't be loaded unless it is actually needed by the user. This requires lazy loading/importing. Obviously, the implementation should be backwards-compatible.Considered lazy import implementations
Five ways to implement lazy loading that I know of:
ModuleType
subclass, as in https://pypi.org/project/lazy-imports/__getattr__
feature (python 3.7+), similar to as in https://snarky.ca/lazy-importing-in-python-3-7/Only the fifth is suitable because of backwards compatibility. It also happens to be one of the simplest.
Problems with 1 - 4:
MissingModule
behavior (whennumpy
is unavailable)MissingModule
behaviorMissingModule
upon import failureMissingModule
behaviorThe pygame submodules
surfarray
andsndarray
will be lazily imported, deferring execution/loading of them to when the user themselves access it (if they do), e.g.pygame.surfarray.array2d
orimport pygame.sndarray
.If they do use it,
numpy
will usually not be loaded at an inconvenient time (to cause framedrops) unless the first use (including imports) is of the formpygame.surfarray.___
inside a function (easy user fix).Backwards compatibility / Testing:
All combinations of these factors must be compatible
sndarray
,surfarray
is or is not accessed by userimport pygame.surfarray
from pygame import surfarray
pygame.surfarray.array2d
from pygame.surfarray import array2d
In addition:
pygame
module (same error message)I did not include test cases in this PR, though I think the implementation is probably compatible.
Notes:
Since
MissingModule
class is deleted at the end of__init__.py
and unavailable to__getattr__
, there is a new private alias_MissingModule
in the pygame namespace.I don't like the
MissingModule
behavior that much because half of the access forms bypass it.If this PR is merged, then more optional pygame submodules may be lazily imported in the future.