Poetry passes Python version to virtualenv incorrectly.

[chrisgavin]

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate. (I couldn't find one; sorry if there is and I've missed it.)

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option). (N/A)

• OS version and name: Ubuntu 18.04

• Poetry version: 1.1.4

• Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/chrisgavin/3002117cc3262a46cf85b545d30aac90

Issue

Apologies if this is not an issue with Poetry but instead an issue with VirtualEnv. The problem seems to stem from them both having slightly different expectations so it's unclear which tool the issue lies in.

For a bit of context, I'm using PyEnv. My system Python is Python 3.6 and my project Python is Python 3.8(.6) as specified in .python-version. Poetry is installed with PipX and hence itself runs in a VirtualEnv created with the system Python.

When I attempt to run Python using Poetry in my project folder this is the output I get:

$ poetry run python3 --version
The currently activated Python version 3.6.9 is not supported by the project (>=3.8).
Trying to find and use a compatible version. 
Using python3 (3.8.6)
Python 3.6.9

As you can see Poetry believes the version of Python used is correct, but it is in fact not.

Poetry recognizes that the version of Python it is running with is not suitable for use with the project, which is correct. It then attempts to find a version of Python on the path which is suitable and finds the PyEnv shim for python3 which is backed by Python 3.8.6 due to the .python-version file. Again this is correct. What seems to go wrong is that when creating the VirtualEnv Poetry passes the argument --python python3. According to the VirtualEnv documentation, the argument for --python is interpreted as "either absolute path or identifier string". I'm not exactly how an "identifier string" is resolved to a Python executable in this context (it seems to be somewhat described here), but it seems to interpret it as being the system Python at /usr/bin/python3. This means that the wrong version of Python is used.

I think the correct thing to do is to look up the absolute path to the python3 binary that Poetry tested the version of and pass that to instead of "python3" which is interpreted as an identifier string. I've done a hacky implementation to verify that this does fix the problem (though I wouldn't use this direct implementation as I'm not sure it works on Windows).

After applying this patch:

diff --git a/poetry/utils/env.py b/poetry/utils/env.py
index 22db246..21af561 100644
--- a/poetry/utils/env.py
+++ b/poetry/utils/env.py
@@ -726,6 +726,8 @@ class EnvManager(object):
                     continue
 
                 if supported_python.allows(Version.parse(python_patch)):
+                    import distutils.spawn
+                    python = distutils.spawn.find_executable(python)
                     io.write_line("Using <c1>{}</c1> ({})".format(python, python_patch))
                     executable = python
                     python_minor = ".".join(python_patch.split(".")[:2])

The output then seems to be much more what I'd expect:

$ poetry run python3 --version          
The currently activated Python version 3.6.9 is not supported by the project (>=3.8).
Trying to find and use a compatible version. 
Using /home/chris/.local/opt/pyenv/shims/python3 (3.8.6)
Python 3.8.6

Thank you for making such an awesome tool! Please let me know if there's any more information I can provide, if I should take a crack at a real implementation for the path resolving logic, or if this is not actually a bug and instead just down to my local Python setup being borked.

[ghost]

Would love this to be merged/looked at.

[finswimmer]

This should be solved now by setting virtualenvs.prefer-active-python to true.

Feel free to leave a comment if this doesn't work for your.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.