[FR]: Remove need for runfiles directory to be writable

[jsharpe]

What is the current behavior?

The venv is recreated whenever the binary is executed. This prevents packaging up the binary and installing it to a system outside of bazel as an application deployment that can be run by other users from a common directory e.g. an /apps directory, possibly served read-only over nfs or similar.

Describe the feature

I would like either for the venv creation to fallback to a user writable location e.g. XDG_CACHE or to remove the need to recreate the venv links so it can be done once by an admin user and then subsequent users can just execute from the precreated venv.

[jsharpe]

Another place where this could be used is in an immutable container - the container is built and then at execution time the container image is made immutable and so writing to the runfiles directory would potentially not be possible in this case too.

[jsharpe]

Also the current behaviour would prevent multiple copies of a binary running at the same time since it would delete files being used by the first process on the execution of the second process.

[voxeljorge]

We've run into this issue with when running py_binary targets from this ruleset in an immutable environment. One example is when running code in a os-only aws lambda runtime, the lambda is unpacked in an immutable env and the py_binary fails to run.

[voxeljorge]

@alexeagle this issue is stopping us from migrating a couple of our lambdas to rules_py, and it seems somewhat straightforward to resolve. The runner could accept an env var as proposed above. Either XDG_CACHE or something specifically named after rules_py seems reasonable, any specific preference?

[thesayyn]

Little bit of context and progress here;

We have recently started using uv for venv generation which supports relative paths astral-sh/uv#3717 it's still experimental but it's almost there.

But we are still blocked on python/cpython#83650 which still forces us to use an absolute path pyenv.cfg file, bummer.

However, i have been hitting my head against the wall trying to solve this and we have solved this with @voxeljorge. It's approximately #427 but needs a static symlink from /tmp/venv back to runfiles tree to allow venv being created in /tmp/venv but available in the runfiles tree.