AttributeError: 'NoneType' object has no attribute 'startswith'

[suchmememanyskill]

Hi all,

I have gotten issue reports in a program that i help maintain called Decky that is a plugin injector and manager for Steam, specifically meant for the Steam Deck. The architecture of this application is pretty weird, so it's possible the following issue is caused by us. Note: this seems to be a windows-specific issue, and i cannot reproduce it locally. It may be related to a recent windows update. I have no idea where to start to figure out this issue.

Traceback (most recent call last):
 File "decky_loader\plugin\sandboxed_plugin.py", line 88, in initialize
 File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
 File "<frozen importlib._bootstrap>", line 1138, in _find_and_load_unlocked
 File "<frozen importlib._bootstrap>", line 1078, in _find_spec
 File "PyInstaller\loader\pyimod02_importers.py", line 274, in find_spec
 File "<frozen importlib._bootstrap_external>", line 1312, in __iter__
 File "<frozen importlib._bootstrap_external>", line 1299, in _recalculate
 File "<frozen importlib._bootstrap_external>", line 1312, in __iter__
 File "<frozen importlib._bootstrap_external>", line 1301, in _recalculate
 File "PyInstaller\loader\pyimod02_importers.py", line 317, in <lambda>
 File "PyInstaller\loader\pyimod02_importers.py", line 275, in find_spec
AttributeError: 'NoneType' object has no attribute 'startswith'

Link to source file of the stack trace that's in our codebase

# Line that produces the above stack trace
from .imports import decky

• Pyinstaller version: 6.8.0, 6.9.0
• Version of Python: 3.11.7
• CI Build: https://github.com/SteamDeckHomebrew/decky-loader/actions/runs/9847726917/job/27188284525
• Platform: Windows
• Did you also try this on another platform? Does it work there? No, but we have recieved no such reports on Steam Deck.

[rokm]

Everyday we learn something new; unfortunately around here, that something usually translates into something horrible...

As the traceback suggests, the problem is that the path list passed to PyiFrozenImport.find_spec contains a None, so this part raises an error:

pyinstaller/PyInstaller/loader/pyimod02_importers.py

Lines 274 to 276 in 09ae973

	for p in path:
	if not p.startswith(SYS_PREFIX):
	continue

We should probably add a guard against that, similarly to how built-in finders also guard against non-string entries (e.g., here).

---

But how did we end up with None in that list in the first place?

It turns out that the problem is indeed a self-inflicted one, and boils down to this part of your entry-point script:

    # Append the system and user python paths
    sys.path.extend(get_system_pythonpaths())

This ends up appending empty string ("") to sys.path in your main process (well, or it doesn't - I'll rant about this later on).

Your plugins' code is executed in multiprocessing sub-processes, so sys.path gets passed from the main process to the worker process. When gathering values to pass to the worker process, the multiprocessing code (for the spawn start method, at least), explicitly looks for empty-string entry to replace it with multiprocessing.process.ORIG_PATH (here). And in the frozen application, multiprocessing.process.ORIG_PATH is set to None (see here). Which explains how the worker process ends up with sys.path that has contains None.

---

A slightly-related intermission: binding sys.path to a variable can be dangerous and misleading, because sys.path can be rebound (by any other module) to a different list instance between the moment where you bound it to your variable and when you make use of your variable.

To illustrate: you bind sys.path to syspath in line 6 of sandboxed_plugin.py:

from sys import exit, path as syspath, modules as sysmodules

And later on, in line 81 of sandboxed_plugin.py (within the SandboxedPlugin.initialize method) you add an entry to the list:

            syspath.append(path.join(environ["DECKY_PLUGIN_DIR"], "py_modules"))

And yet, when you do the import in line 88 that triggers the error, the sys.path that PyiFrozenImporter receives from importlib does not contain that entry, and it contains a None that syspath does not contain.

Why?

Because the binding in line 6 happens when decky_loader.plugin.sandboxed_plugin is imported, which for the purposes of this example happens in line 30 of decky_loader.main, which imports decky_loader.loader, whose line 18 imports decky_loader.plugin.plugin, whose line 7 in turn imports decky_loader.plugin.sandboxed_plugin).

The worker subprocess codepath then diverges in line 228, where multiprocessing.freeze_support is called. This in turn enters our run-time hook, which ends up calling multiprocessing.spawn.spawn_main:

pyinstaller/PyInstaller/hooks/rthooks/pyi_rth_multiprocessing.py

Line 48 in 09ae973

multiprocessing.spawn.spawn_main(**kwds)

This ends up reading the values that are passed to the worker process from the parent, which replaces sys.path with new list instance received from the parent process (this happens in prepare, called from _main, which is called from spawn_main). After this setup, the target function is executed, i.e., SandboxedPlugin.initialize is called. So when the line 80 is reached, the list bound to sys.path is different from the one that is bound to syspath. Which basically makes your modifications of syspath a no-op.

---

Now, let us return to that get_system_pythonpaths() that returned empty string (here):

def get_system_pythonpaths() -> list[str]:
    try:
        # run as normal normal user if on linux to also include user python paths
        proc = subprocess.run(["python3" if localplatform.ON_LINUX else "python", "-c", "import sys; print('\\n'.join(x for x in sys.path if x))"],
        # TODO make this less insane
                              capture_output=True, user=localplatform.localplatform._get_user_id() if localplatform.ON_LINUX else None, env={} if localplatform.ON_LINUX else None) # pyright: ignore [reportPrivateUsage]
        return [x.strip() for x in proc.stdout.decode().strip().split("\n")]
    except Exception as e:
        logger.warn(f"Failed to execute get_system_pythonpaths(): {str(e)}")
        return []

This fails to check for return code of the process it attempts to spawn, so if pyhon (or python3 on linux) is not in PATH, it ends up returning a list that contains an empty string (because proc.stdout itself is an empty (byte-)string).

If python interpreter does happen to be in PATH, it will return all its sys.path entries.

Which is obviously a bad idea for the frozen application, because that python environment might be using completely different python version than the one that application was built with.

---

And lastly, this:

def main():
    setproctitle(f"Decky Loader {get_loader_version()} ({getproctitle()})")
    setthreadtitle("Decky Loader")
    if ON_WINDOWS:
        # Fix windows/flask not recognising that .js means 'application/javascript'
        import mimetypes
        mimetypes.add_type('application/javascript', '.js')

        # Required for multiprocessing support in frozen files
        multiprocessing.freeze_support()
    else:
        if get_effective_user_id() != 0:
            logger.warning(f"decky is running as an unprivileged user, this is not officially supported and may cause issues")

You should probably (re)read https://pyinstaller.org/en/stable/common-issues-and-pitfalls.html#multi-processing, otherwise you might be in for a rude surprise if you ever try to build for macOS, or if you ever end up building with python 3.14 for linux.

Also, if it works for you, you might want consider to moving that freeze_support call to the very top of your entry-point script, to avoid unnecessary imports and setup in worker processes.

[suchmememanyskill]

Thank you for your extremely detailed response! I'll see if i can get to implementing it tonight.

Issue seems relatively straightforward to fix from our side, can't believe i missed it. I'll also other suggestions/concerns, looks like mistakes we never ran up against.

Thanks again!

[suchmememanyskill]

@rokm One additional note i'd like to make; your reasoning for why get_system_pythonpaths() doesn't work as intended isn't actually correct, it seems. subprocess.run throws an exception on missing exec file.

The reasoning is slightly more dubious. In recent-ish versions of windows, microsoft (unhelpfully) includes a dummy python executable that opens the microsoft store, and if any arguments are provided, will pass them to an installed intepreter. If no intepreters are installed, it will send a message to stderr and exit with error code 9009.

[rokm]

Good point - thanks for the correction!