pip install pip reports wrong version was installed

[cjerdonek]

• Pip version: 9.0.1
• Python version: 3.6.1
• Operating system: Mac OS X and Ubuntu 14.04

Description:

When installing a different version of pip, pip reports that a different version was installed.

To reproduce:

$ python -m venv temp-venv
$ temp-venv/bin/pip --version
pip 9.0.1 from /.../temp-venv/lib/python3.6/site-packages (python 3.6)
$ temp-venv/bin/pip install --ignore-installed pip==9.0.0
Collecting pip==9.0.0
  Downloading pip-9.0.0-py2.py3-none-any.whl (1.3MB)
    100% |████████████████████████████████| 1.3MB 1.0MB/s 
Installing collected packages: pip
Successfully installed pip-9.0.1

The line above should say that pip-9.0.0 was installed.

$ temp-venv/bin/pip --version
pip 9.0.0 from /.../temp-venv/lib/python3.6/site-packages (python 3.6)

This was previously reported for pip 8.1.2 / Python 2.7.12+ in issue #4250.

[xavfernandez]

Thanks a lot for your reproducible steps, it works (or in this case, fails) perfectly.

The issue seems to be with the --ignore-installed option.
After the previous steps, you get two pip-*.dist-info:

$ ls temp-venv/lib/python3.6/site-packages/
easy_install.py  pip  pip-9.0.0.dist-info  pip-9.0.1.dist-info  pkg_resources  __pycache__  setuptools  setuptools-28.8.0.dist-info

The issue does not occur without the --ignore-installed option. Without it, pip correctly reports: Successfully installed pip-9.0.0

[pfmoore]

I'd expect two dist-info files - you said to ignore the installed version and install another version, so after the install, both would be there. Of course, having 2 versions of pip installed in the same location at the same time is unsupported, so this is essentially a broken install (created at the deliberate request of the user).

I believe that the report of what is installed uses pkg_resources to query the installed version, rather than remembering what it was asked to install and just reporting that. When faced with a system with two dist-info files, I can imagine pkg_resources getting confused. I don't know if this would count as a bug, though, given that the system is in an invalid state.

You could argue that pip should refuse the request to --ignore-installed in this case, but I'm not sure how easy it would be to detect this. After all if the venv used system-site-packages, and the copy of pip already installed was in the system location, not the venv, that would be valid, and (AFAIK) would work correctly. So I'd be OK with pip checking for an invalid request like this, but I'd be equally fine with simply saying "you shouldn't do that" and leaving it at that - I view --ignore-installed as an "I know what I'm doing" type of flag, so expecting the user to understand the consequences of his request is OK with me.

[xavfernandez]

Mmmh I'm in fact wondering if it really is a bug.
--ignore-installed is an escape hatch to basically tell pip, "don't bother uninstalling what's there" which explains the remaining .dist-info directory (and yes, using it might break your system).
If what you need is "reinstall the package no matter what", you have the --upgrade --force-reinstall option.

[cjerdonek]

Is there a preferred way to "downgrade" pip then? The pip install --ignore-installed at the top seems to work (aside from the message).

[dstufft]

You should be able to just do pip install pip==9.0.0.

[cjerdonek]

There seems to be at least one use-case where the --ignore-installed option was working better than the bare pip install for installing a specific version.

If installing Python 3.6.1 from source, then pip 9.0.1 is available as pip3.6. Running pip3.6 install --ignore-installed pip==9.0.1 makes it so the user can also invoke pip using the bare pip, but pip3.6 install pip==9.0.1 does not.

What would be the way to re-install pip 9.0.1 so that it can be accessed using the bare pip (if pip 9.0.1 is already installed)?

That's how I wound up with --ignore-installed. It seemed to work for both downgrading and reinstalling the same version.

[dstufft]

pip install --upgrade --force-reinstall pip==9.0.1

[cjerdonek]

Okay, thanks a lot, @dstufft. I was hoping a single command would work for both. But adding an if-statement to check if the version matches isn't so bad.

[dstufft]

You should be able to do pip install --upgrade --force-reinstall pip==9.0.2 as well FWIW.

Going to close this since it seems we've gotten this solved differently :)

[dstufft]

Er, that's pip install --upgrade --force-reinstall pip==9.0.0.

[cjerdonek]

You're right. Great! Thanks again.

[piotr-dobrogost]

rant on
I'm curious what's so great about having to specify --upgrade to make a downgrade? Don't get me wrong but sometimes I feel like ecosystem around pip is some kind of isolated world where things normally regarded as wrong/bad/inappropriate/insane here are great solutions. I feel like this deserves issue on its own. Should I create one or is it as designed and I should not bother?
rant off
I'm also curious why paradoxically --force-reinstall is not enough by itself?

[pfmoore]

There's lots of history if you want to read #3786 and #3792. Neither of those specifically deal with why --upgrade is the flag to do a downgrade, but that's likely more to do with the expected use case (which is to install a newer version) than anything else. Also, it's not immediately obvious when looking at a completely general requirement whether it would result in an upgrade or a downgrade.

So, my assumption is:

1. Backward compatibility
2. No-one can think of a better name

And as usual, (3) we're a group of volunteers with limited time and this question is way down the priority list :-)

I doubt an issue for this is worth it. But if you want to raise a PR, that would be OK I guess (but make sure you handle the backward compatibility questions, and corner cases like pip install --upgrade --force-reinstall pip!=9.0.1 (upgrade or downgrade?)!)

[piotr-dobrogost]

Thanks for links to issues. I'm following pip for a few years now so it's not like I'm not aware of its complicated history. That suggesting to pass --upgrade to perform downgrade still makes me uncomfortable in spite of me following pip for all this time I take as a sign I'm still sane. In cases like this I can't stop wondering how bizarre such suggestions must appear to users not following pip...

I doubt an issue for this is worth it

I suspected this would be the answer. As sad as it is I take it as a firm sign of where this project is in terms of user experience. Are you aware of what signal does it send to users? Imagine telling to the new user something like: Yes, passing --upgrade to perform downgrade does not make sense. No, do not create issue for this, really; we do not need this issue. And in case your friend using pip is also surprised, please tell him he should not file one as well. Really. Just keep telling people it's what it is and try to explain them the project does not need issue for this. Really.
As to we're a group of volunteers; it's well understood but it should never be an argument for not having github's active issues for well, project's active issues.

pip install --upgrade --force-reinstall pip!=9.0.1

--upgrade here should not be needed. This should be just pip install pip!=9.0.1 if you want to have pip optionally installed or pip install --force-reinstall pip!=9.0.1 if you want to reinstall pip even if the version selected to install is the version already installed.

I'm still curious why paradoxically --force-reinstall is not enough by itself?

[xavfernandez]

FWIW I was also suprised by the --force-reinstall which basically do nothing except in --upgrade mode.
Maybe the use of --force-reinstall should automatically imply the --upgrade option.

[pfmoore]

I agree - it seems like a good idea for --force-reinstall to imply --upgrade.

[piotr-dobrogost]

Btw, I find the name of this option mouthful. I think either --force or --reinstall would be enough.

[cjerdonek]

Thanks all for the discussion and for raising the suggestion, @piotr-dobrogost. I came back here to ask / suggest the same thing -- about whether --force-reinstall can imply --upgrade. I'm happy people seem to agree.

A couple points to add: With this change, the naming issue of --upgrade being used to downgrade won't be as bad because --upgrade won't be needed (at least in this case). Also, the current behavior partly explains this issue and why I went down the road of trying options like --ignore-installed in the first place.

By the way, @piotr-dobrogost, just to clarify: when I said "great" I was thanking Donald for his replies and not suggesting that the "UI" is great.

I will open an issue for this unless someone beats me to it.

[cjerdonek]

I will open an issue for this unless someone beats me to it.

Oh, this has already been filed as issue #1139.

[cjerdonek]

FYI, I added a PR with tests for the --force-reinstall / --upgrade issue raised here. I added it on the original issue (issue #1139).