Disable user site packages (~/.local) for specific in-container Python components

[achimnol]

When users install custom Python packages into the user-site directory (~/.local) which is provided as an auto-mounted docker volume in containers, sometimes they break the dependency of Jupyter making it to crash when starting the app.

We could minimize the impact of dependency breakage due to user-installed packages as follows:

However, it is not possible to 100% prevent such problems because we cannot disable user-site directories for specific packages for a single Python process running both ipykernel and user programs.

Note

Some futuristic idea: We could make ipykernel to use subinterpreters in Python 3.12+ to isolate the package import namespaces (with different user-site settings) within a single process.

Still, I think it would be better to explicitly disable user-sites for the designated components to minimize the problem's surface area.

[achimnol]

Some investigation results:

• ipykernel still depends on too significant portion of the Jupyter package ecosystem, including juypter_client and jupyter_core, which are the frequent breakers.
• Python's import order prefers venv packages first, and then read user-site packages, and then falls back to the system-site packages. Therefore, if we apply virtualenvs in the image-provided Python setups, we can no longer provide the .local-based runtime customization of container environments.

So, I'm closing this issue but keep it as a historical record for now.
We need a proper isolation mechanism implemented in the ipython-side to separate the package spaces.

[achimnol]

Reopening this issue with a shrinked scope: let's disable the user-site directory only for the krunner itself, not the user-executed service apps which use the container's own python runtime.