Look into improving Arcade's startup time

[einarf]

Arcade can be pretty slow at startup. Especially the first time you launch it. The OS will do some caching causing future startups faster. Investigate what is taking up all this time. We know font loading is one of the reasons.

[pushfoo]

Do we need to preload these default fonts? The examples seem to be the main place they're used.

arcade/arcade/__init__.py

Lines 571 to 583 in c5ac350

	if not getattr(sys, 'is_pyglet_doc_run', False):
	# Auto load fonts
	load_font(":resources:fonts/ttf/Kenney Blocks.ttf")
	load_font(":resources:fonts/ttf/Kenney Future.ttf")
	load_font(":resources:fonts/ttf/Kenney Future Narrow.ttf")
	load_font(":resources:fonts/ttf/Kenney High.ttf")
	load_font(":resources:fonts/ttf/Kenney High Square.ttf")
	load_font(":resources:fonts/ttf/Kenney Mini.ttf")
	load_font(":resources:fonts/ttf/Kenney Mini Square.ttf")
	load_font(":resources:fonts/ttf/Kenney Pixel.ttf")
	load_font(":resources:fonts/ttf/Kenney Pixel Square.ttf")
	load_font(":resources:fonts/ttf/Kenney Rocket.ttf")
	load_font(":resources:fonts/ttf/Kenney Rocket Square.ttf")

Defaults in the API seem to use Arial or calibri:

arcade/arcade/gui/widgets/text.py

Line 64 in bf28ded

font_name=("Arial",),

arcade/arcade/text.py

Line 176 in 2a7c1f1

font_name: FontNameOrNames = ("calibri", "arial"),

arcade/arcade/text.py

Line 638 in 2a7c1f1

font_name: FontNameOrNames = ("calibri", "arial"),

arcade/arcade/text.py

Line 553 in 2a7c1f1

font_name: FontNameOrNames = ("calibri", "arial"),

Once I'm done with some current projects, I can look at replacing this preload list with a libre font that's metrically compatible with Arial as part of #1135.

[ClintLiddick]

I would love a patch to remove this default font loading. It is both slow, surprising (I'm not using any text in my application, but pay the font loading cost), and incidentally causes issues with Bazel build integration because Bazel does not support any files with whitespace (so it ignore them from the wheel): bazelbuild/rules_python#617

[einarf]

So that means there are two issues:

• Resource files containing spaces
• Avoid triggering pyglet's system font loader

[einarf]

@ClintLiddick Quick fix for whitespace in files names now in development at least : f012e7b

We can probably find some way to disable the font loading. The issue is not the font loading itself. That's pretty fast. The real issue is that pyglet will scan all system fonts when you access the pyglet.text module proxy. This is then cached in the OS so it will be more or less instant next time your run your game (within the cache time)

Still. It's definitely annoying when you don't use text.

[pushfoo]

Recap of what I was told in the pyglet Discord today:

• Win32 GDI (Win 7-ish and fallback?) font loading currently scans the system font directory
• I suggested a pyglet config flag
• Ben suggested deferring font loading by default to allow people who ship their own fonts to use them

That's an upstream change as I understand it, but one that seems like it should be simple if I can figure out where the font loading actually gets invoked.

[einarf]

We did some testing the other day. The font scanning is gone on windows since we no longer use GDI.

• Media/sound initialization is the most time consuming
• The input/jotstick module can add another 0.2 - 0.4s for directinput

Startup times will rely on OS. I haven't tested for linux and mac.

This is the worst case situation for me after profiling over a few days

    355/1    0.002    0.000    4.193    4.193 <frozen importlib._bootstrap>:1349(_find_and_load)
      258    3.987    0.015    4.000    0.016 {built-in method time.sleep}
    355/1    0.001    0.000    3.883    3.883 <frozen importlib._bootstrap>:1304(_find_and_load_unlocked)
    346/1    0.002    0.000    3.883    3.883 <frozen importlib._bootstrap>:911(_load_unlocked)
    314/1    0.001    0.000    3.883    3.883 <frozen importlib._bootstrap_external>:988(exec_module)
    786/2    0.001    0.000    3.883    1.942 <frozen importlib._bootstrap>:480(_call_with_frames_removed)
    314/1    0.002    0.000    3.883    3.883 {built-in method builtins.exec}
      2/1    0.000    0.000    3.883    3.883 arcade\__init__.py:1(<module>)
        1    0.000    0.000    3.823    3.823 arcade\sound.py:1(<module>)
        1    0.000    0.000    3.822    3.822 pyglet\media\__init__.py:1(<module>)
    84/42    0.000    0.000    3.236    0.077 {built-in method builtins.__import__}
3958/3714    0.002    0.000    3.185    0.001 <frozen importlib._bootstrap>:1390(_handle_fromlist)
       35    0.002    0.000    3.049    0.087 ctypes\__init__.py:343(__init__)
        1    0.000    0.000    3.048    3.048 pyglet\media\codecs\__init__.py:40(add_default_codecs)
        1    0.000    0.000    3.039    3.039 pyglet\media\codecs\__init__.py:88(have_ffmpeg)
        1    0.000    0.000    3.039    3.039 pyglet\media\codecs\ffmpeg_lib\__init__.py:1(<module>)
        7    0.000    0.000    3.032    0.433 pyglet\lib.py:81(load_library)
       21    0.000    0.000    3.027    0.144 ctypes\__init__.py:459(LoadLibrary)
        1    0.000    0.000    1.964    1.964 pyglet\media\codecs\ffmpeg_lib\libavcodec.py:1(<module>)
        1    0.000    0.000    0.652    0.652 pyglet\media\codecs\ffmpeg_lib\libavformat.py:1(<module>)
        1    0.000    0.000    0.531    0.531 pyglet\media\codecs\ffmpeg_lib\libavutil.py:1(<module>)
        1    0.000    0.000    0.447    0.447 arcade\application.py:1(<module>)
        1    0.000    0.000    0.418    0.418 pyglet\media\codecs\ffmpeg_lib\libswscale.py:1(<module>

[caffeinepills]

Recap of what I was told in the pyglet Discord today:

• Win32 GDI (Win 7-ish and fallback?) font loading currently scans the system font directory
• I suggested a pyglet config flag
• Ben suggested deferring font loading by default to allow people who ship their own fonts to use them

That's an upstream change as I understand it, but one that seems like it should be simple if I can figure out where the font loading actually gets invoked.

Clarification, Windows 7 does use DirectWrite as well in Pyglet, it just doesn't have all the features that 8.1+ does (colored font characters/emojis for example).

Right now when you use a tuple of font names, a call of have_font is called for each font name on creation. This triggers pyglet to check the loaded custom fonts if a font exists, if it doesn't, then it will enumerate all system fonts to check for the name. While the initial call can take a while, a result against the database is cached (changed since arcade 2.6 was released) at least. On my system it took 0.00449824333190918 for 1031 entries, which isn't too bad in my opinion. Even if I doubled my fonts, it would only be 0.008 seconds.

That being said I did investigate a bit, and there is a way to check if a font exists for GDI+ without enumerating all of the fonts. With that fix in, it took 0.00024770002346485853 to check if a font exists. I can push this out to pyglet so it can be in the next version. As long as the latest pyglet is still compatible with Arcade 2.6, or whoever is using GDI+ for legacy/performance reasons, it should be significantly faster. I would be curious to see the load time difference between the old way and new way.

[einarf]

@caffeinepills We don't really worry about 2.6 at this points, but I'm sure that's a great addition for other pyglet users if people are using GID. I'm not sure how widespread that is.

Looking into media and input initialization will probably benefit more people? I don't know if there's any improvements that can be done here.

In arcade we should start by not importing sound/media/input on arcade import by default.

[benjamin-kirkbride]

We did some testing the other day. The font scanning is gone on windows since we no longer use GDI.

• Media/sound initialization is the most time consuming
• The input/jotstick module can add another 0.2 - 0.4s for directinput

Startup times will rely on OS. I haven't tested for linux and mac.

This is the worst case situation for me after profiling over a few days

    355/1    0.002    0.000    4.193    4.193 <frozen importlib._bootstrap>:1349(_find_and_load)
      258    3.987    0.015    4.000    0.016 {built-in method time.sleep}
    355/1    0.001    0.000    3.883    3.883 <frozen importlib._bootstrap>:1304(_find_and_load_unlocked)
    346/1    0.002    0.000    3.883    3.883 <frozen importlib._bootstrap>:911(_load_unlocked)
    314/1    0.001    0.000    3.883    3.883 <frozen importlib._bootstrap_external>:988(exec_module)
    786/2    0.001    0.000    3.883    1.942 <frozen importlib._bootstrap>:480(_call_with_frames_removed)
    314/1    0.002    0.000    3.883    3.883 {built-in method builtins.exec}
      2/1    0.000    0.000    3.883    3.883 arcade\__init__.py:1(<module>)
        1    0.000    0.000    3.823    3.823 arcade\sound.py:1(<module>)
        1    0.000    0.000    3.822    3.822 pyglet\media\__init__.py:1(<module>)
    84/42    0.000    0.000    3.236    0.077 {built-in method builtins.__import__}
3958/3714    0.002    0.000    3.185    0.001 <frozen importlib._bootstrap>:1390(_handle_fromlist)
       35    0.002    0.000    3.049    0.087 ctypes\__init__.py:343(__init__)
        1    0.000    0.000    3.048    3.048 pyglet\media\codecs\__init__.py:40(add_default_codecs)
        1    0.000    0.000    3.039    3.039 pyglet\media\codecs\__init__.py:88(have_ffmpeg)
        1    0.000    0.000    3.039    3.039 pyglet\media\codecs\ffmpeg_lib\__init__.py:1(<module>)
        7    0.000    0.000    3.032    0.433 pyglet\lib.py:81(load_library)
       21    0.000    0.000    3.027    0.144 ctypes\__init__.py:459(LoadLibrary)
        1    0.000    0.000    1.964    1.964 pyglet\media\codecs\ffmpeg_lib\libavcodec.py:1(<module>)
        1    0.000    0.000    0.652    0.652 pyglet\media\codecs\ffmpeg_lib\libavformat.py:1(<module>)
        1    0.000    0.000    0.531    0.531 pyglet\media\codecs\ffmpeg_lib\libavutil.py:1(<module>)
        1    0.000    0.000    0.447    0.447 arcade\application.py:1(<module>)
        1    0.000    0.000    0.418    0.418 pyglet\media\codecs\ffmpeg_lib\libswscale.py:1(<module>

Can say how you did the profiling? I can test on Linux

[einarf]

Can say how you did the profiling? I can test on Linux
@benjamin-kirkbride

You might want different sorting: https://docs.python.org/3/library/profile.html#instant-user-s-manual

python -m cProfile -s tottime -m arcade > stats.txt

Startup time will vary due os caching and devices going to sleep.

[einarf]

Font loading in __init__ removed in 3.0 #2236