Poetry using the wrong Python version (not related to pyenv)

[zmitchell]

• I have searched the issues of this repo and believe that this is not a duplicate.
• I have searched the documentation and believe that my question is not covered.

Question

I'm having an issue adding a package to a project because poetry is picking up on the wrong Python version. I'm pretty confident that something has gone awry in my system, but I'm doing my best to salvage the situation and not have to go through the whole process of installing Raspbian from scratch on this Raspberry Pi. I installed poetry using the recommended method (curl ...). Can you point me in the right direction? As mentioned in the title, I'm not using pyenv. All of the relevant issues I found were related to pyenv, so I'm not really sure how to get poetry to use the correct Python.

Here are some terminal outputs to help you diagnose the issue:

pi@widget:temp_logger$ python --version
Python 3.5.3
pi@widget:temp_logger!$ sudo python --version
Python 2.7.13
pi@widget:temp_logger$ poetry --version
Poetry 0.12.9
pi@widget:temp_logger$ poetry debug:info

Poetry
======

 * Version: 0.12.9
 * Python:  2.7.13


Virtualenv
==========

 * Python:         2.7.13
 * Implementation: CPython
 * Path:           NA


System
======

 * Platform: linux2
 * OS:       posix
 * Python:   /usr


pi@widget:temp_logger$ poetry add adafruit-blinka

[RuntimeError]
The current Python version (2.7.13) is not supported by the project (^3.5)
Please activate a compatible Python version.

add [-D|--dev] [--git GIT] [--path PATH] [-E|--extras EXTRAS] [--optional] [--python PYTHON] [--platform PLATFORM] [--allow-prereleases] [--dry-run] [--] <name> (<name>)...

[sdispater]

As you can see in debug:info, Poetry picks up Python 2.7.13 and not Python 3.5.3. What is the output of which python? Does it point to your 2.7.13 Python or 3.5.3?

[zmitchell]

Ah, here we go. It turns out python was aliased to python3, and doing which python uncovered that. Now python --version returns 2.7.13, and which python returns /usr/bin/python. So how do I get poetry to use Python 3 instead?

[buriy]

@sdispater the issue is even worse.

Poetry starting with 0.12.5 stopped working with the python version from the virtualenv, only checks system version of Python.
Version 0.12.4 is the last version that worked propertly.

After this command $ sudo pip3 install poetry it shows:

$ poetry update
[RuntimeError]                                                
The current Python version (3.6.7) is not supported by the project (~2.7)  
Please activate a compatible Python version.                               

$ poetry debug:info

Poetry
======

 * Version: 0.12.9
 * Python:  3.6.7


Virtualenv
==========

 * Python:         2.7.15
 * Implementation: CPython
 * Path:           /Projects/myproject/.venv
 * Valid:          True


System
======

 * Platform: linux
 * OS:       posix
 * Python:   /usr

If you would install it with sudo pip2 install poetry, it will work with python 2.7 projects, but won't work with python 3.6 projects.
I don't use "source .venv/bin/activate" and I shouldn't be forced to.

[sdispater]

@buriy Commit 94304e9 should fix your issue.

It will land in the next bugfix release.

[buriy]

@sdispater Thanks, then these are different issues with the one from topic starter.
Sorry for hijacking thread.

[buriy]

@zmitchell Then, If I got it right, try reinstalling poetry. If installed with python2, it's bound to using python2 unless you use pyenv or virtualenv.

[zmitchell]

I reinstalled with Python 3 using curl ... | python3 and it still only picks up 2.7.

[jtbeach]

@zmitchell In 0.12.x, poetry is not bound to the interpreter version you curl the installer script through during installation -- so it doesn't matter which one you install it with with.

If python points to python2 and your project needs python3, you have 2 choices:

1. (preferable) Use pyenv and run pyenv local 3.7.0 or whatever your project wants in the root folder of your project. If you have your pyenv shims setup correctly in your shell startup files, python will now execute Python 3.7.0 whenever python runs from that folder or its subfolders, regardless of what python points to globally.

2. Manually create a virtualenv with Python 3 and activate it prior to running poetry commands. You can use direnv or similar to make that a little more automated, but I still prefer using pyenv -- once you get used to it, it's great

[buriy]

@jtbeach

2. Manually create a virtualenv with Python 3 and activate it prior to running poetry commands. You can use direnv or similar to make that a little more automated, but I still prefer using pyenv -- once you get used to it, it's great

Plese note that this option for the local .venv folder is broken in version 0.12.5 and will be fixed in version 0.12.10, which is not yet published (I rechecked it's not fixed in 0.12.9).

[sdispater]

@zmitchell For now, your best bet is to use pyenv (which is the easiest way to manager different Python versions if you are not on Windows) until #621 is implemented (work in progress here https://github.com/sdispater/poetry/tree/env-command)

[sdispater]

@buriy Release 0.12.10 is out! 

[buriy]

@sdispater thanks a lot!
Option 2 mentioned by @jtbeach works again.

[asodeur]

Not sure if this is the original issue or a new Windows related issue (seeing this on 0.12.10 and 1.0.0a1).

SystemEnv.run("python", ...) will always use the primary system interpreter on Windows even if poetry is running on another one not installed in a virtual env. This is because SystemEnv._bin("python") is looking for python.exe in SystemEnv._bin_dir which on Windows is <Python root>/Scripts, however the interpreter is there for virtual envs only. for standard Python installations the interpreter is <Python root>/python.exe.

Running poetry from inside a virtual env, ie option 2 above works fine.

[airstandley]

I've also been running into issues trying to get poetry installing to the correct version of python when not using a virtual environment (poetry config settings.virtualenvs.create false).

After doing some digging it looks like SystemEnv is still determining the binary to call by appending sys.base_prefix/bin to the executable name given. This approach will not work on linux systems that have versions of both python2 and python3 installed.
It seems that a system python environment should be using the system python and simply calling the binary without a specific path. At the moment there seems to be no way to get poetry to install to a linux system python3 installation without changing the /usr/bin/python symlink prior to running poetry install. This is far from ideal. At least if just python was used a shim could be inserted on the path which would allow poetry to install to the correct python version without risking issues with system libraries that call /usr/bin/python.

Ideally it would be nice to just use the version of python poetry was executed with when installing to the system environment.

[Peque]

Using Poetry 1.0 (prerelease) seems to fix this issue:

poetry self:update --preview
poetry env use python3.7

See #731 for more information.

[geryogam]

@Peque Well the new poetry env command provides an explicit way to fix the issue (which is great). But the implicit way is still broken. Poetry should implicitly create a built-in virtual environment with the Python version specified by the tool.poetry.dependencies.python property of the pyproject.toml file, not the Python version resolved by the #!/usr/bin/env python shebang line of the ~/.poetry/bin/poetry script.

[sdispater]

This has been improved in the latest beta release (see #1477 for more information).

[tatianafrank]

The only thing that works for me (until i can use the new env feature) is running pyenv shell <version>. Unfortunately pyenv local and pyenv global dont work and my virtual environment is not getting picked up otherwise. Obviously this is not a good solution but a workaround for now..

[AlJohri]

Upgrading to the latest beta (1.0.0b8) fixed the issue for me:

The currently activated Python version 3.7.5 is not supported by the project (^3.8).
Trying to find and use a compatible version.
Using python3 (3.8.0)
Creating virtualenv al-research-project in /Users/aljohri/Development/al-research-project/.venv

p.s. hi @tatianafrank!

EDIT: Actually, no it didn't fix it. Despite the message that says it will use 3.8.0, the virtualenv uses poetry's python version:

$ /Users/aljohri/Development/al-research-project/.venv/bin/python --version
3.7.5

Here's some debug information:

As you can see, I'm using asdf (which is basically equivalent to pyenv but allows for multiple languages).

Additionally, I have poetry installed via pipx (a common way to install python CLI tools in their own isolated virtualenvs):

$ pipx install poetry --pip-args=--pre
  installed package poetry 1.0.0b8, Python 3.7.5
  These apps are now globally available
    - poetry
done! ✨ 🌟 ✨
$ which poetry
/Users/aljohri/.local/bin/poetry

[ekhaydarov]

None of the above works if youre trying to run poetry from crontab

[perklet]

Using curl -sSL ... to install is still broken on macOS catalina with homebrewed python3. However, using pip3 install poetry works for me.

[JulianFerry]

I think it would be nice to be able to specify poetry config python-version 3.7 so that poetry generally defaults to that version of python.

For example poetry new should default to using that version when creating a new project. Currently I have to remember to update the python version from 2.7 to 3.7. In the cases where I forget that leads to some confusion when I call poetry add but can't access that package in my python interpreter.

Currently when I call poetry new, if I then call poetry env use python3.7 I get:

[NoCompatiblePythonVersionFound]
The specified Python version (3.7.7) is not supported by the project (^2.7).
Please choose a compatible version or loosen the python constraint specified in the pyproject.toml file.`

If I edit the pyproject.toml file to python = "^3.7" instead of 2.7, when I call poetry add I see the following warning message:

The currently activated Python version 2.7.16 is not supported by the project (^3.7).
Trying to find and use a compatible version.
Using python3 (3.7.7)

Which is slightly confusing since no version of python has been "activated" at this point, and I have aliased python to python3, so it is only the poetry env use configuration which is incorrect.

[martynasoxylabs]

I solve this problem on my Mac by replacing following line

#!/usr/bin/env python

to

#!/usr/bin/env python3.8

in /home/<name>/.poetry/bin/poetry file. Then it picks python3.8 automatically and there are no annoying warnings anymore.

[mafeko]

Just another copy&paste takeaway for the working solution on unix-systems:

sed -i 's/#\!\/usr\/bin\/env python/#\!\/usr\/bin\/env python3/g' ~/.poetry/bin/poetry

[iuriguilherme]

I have poetry version 1.1.4 installed via pyenv exec pip --user --upgrade poetry at ~/.local/bin/poetry and this is the first line of that file:

#!/home/user/.pyenv/versions/3.9.2/bin/python3.9

When I attempt a poetry install in a project folder, it uses the system python (which is 3.5 and therefore useless).

Tried the pyenv exec poetry to the same result. Even ~/.local/bin/poetry will still yeld the following error:

SolverProblemError
The current project's Python requirement (3.5.3) is not compatible with some of the required packages Python requirement:

[chamini2]

I still experience this, is there a new issue to subscribe to for the fix?

[dreamflasher]

Same issue here, is there any way to directly specify the python bin path for poetry?

[edgarrmondragon]

@dreamflasher poetry env use already takes an executable, so this might work for you:

poetry env use $(pyenv which python3.10)

[dreamflasher]

Thank you, but unfortunately this didn't work. I needed to add (sudo update-alternatives) a symlink to python3.9 as well as modify the poetry bin.

[V-iktor]

Still an issue.

python --version
Python 3.9.16

(saleor-py3.9) x@x-MacBook-Pro-2 saleor % poetry install

Current Python version (3.10.9) is not allowed by the project (~3.9).

[MichaelVoelkel]

Yeah exactly

[iuriguilherme]

poetry and pipenv select the python you specify on pyproject.toml or Pipfile, as long as they're findable in the PATH or similar. Those tools are not affected by pyenv shims.

[MichaelVoelkel]

I don't even have pyenv in path, it will give "not found". My issue was that I had Python3.9 installed and Python3.10 also. However, python3 symlink would point to 3.10 and python2 to 3.9. Poetry for some reason insisted, although I stated 3.10, on using python3.9, no clue why. It was certainly in the path BUT behind the paths in the front that pointed to 3.10.
Anyways, I removed Py3.9 for good, so that solves it

[bghorvath]

Still an issue for me too, fresh pyenv and poetry install on WSL2, installed both using the official script.
Pyenv is activated, which python, python --version points to the my correct pyenv install (3.11.2). I'm trying to spawn a poetry shell (after poetry env use 3.11), but it says my activated python is 3.10 (my system python)

[trompx]

I use poetry 1.5.0 inside a docker container with asdf to handle python versions. Poetry was installed with python 3.8.10 (installed with asdf and set as global version).
I have POETRY_VIRTUALENVS_PREFER_ACTIVE_PYTHON=true and POETRY_VIRTUALENVS_IN_PROJECT=true.

In .tool-versions I have python 3.8.10 and I get the following error:

The current project's Python requirement (>=2.7,<2.8 || >=3.4) is not compatible with some of the required packages Python requirement:
  - pandas requires Python >=3.7.1, so it will not be satisfied for Python >=2.7,<2.8 || >=3.4,<3.7.1
Because XXX depends on pandas (1.3.5) which requires Python >=3.7.1, version solving failed.

Which I don't get as I am using python 3.8.10.
Also, output of poetry debug info:

Poetry
Version: 1.5.0
Python:  3.8.10

Virtualenv
Python:         3.8.10
Implementation: CPython
Path:           /home/www-data/dev/proja/.venv
Executable:     /home/www-data/dev/proja/.venv/bin/python
Valid:          True

System
Platform:   linux
OS:         posix
Python:     3.8.10
Path:       /home/www-data/.asdf/installs/python/3.8.10
Executable: /home/www-data/.asdf/installs/python/3.8.10/bin/python3.8

Is there any specific volume or env var I need to pass in the docker container while running the poetry command?

[finswimmer]

@trompx This is described in the FAQ: https://python-poetry.org/docs/faq/#why-is-poetry-telling-me-that-the-current-projects-python-requirement-is-not-compatible-with-one-or-more-packages-python-requirements

[sumeeterito]

also still an issue for me. I am on poetry 1.5.1 and when i run

• poetry env use 3.7.16
• poetry add scikit-learn

I get

Error: 'numpy' must be installed before running the build.

  at ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry/installation/chef.py

you can see that the poetry environment is referencing a python 3.8.

even poetry run python --version yields 3.7.16; but still it references python 3.8

[finswimmer]

@sumeeterito This issue is closed for a long time and yours seems unrelated to me. Please open a new ticket.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.