implement INSTALLER from PEP376

[qwcode]

when pip is installing dist-info (only for wheels currently), we should write the 'INSTALLER' file, so pip can know later if it installed it.

http://legacy.python.org/dev/peps/pep-0376/#installer

for consistency, we could also start writing this file into egg-info metadata.

this could be useful for generating warnings when we're uninstalling something not installed by pip.

also, we could present this information in pip list

[pfmoore]

this could be useful for generating warnings when we're uninstalling something not installed by pip.

I'd be against a warning if it was going to trigger every time a pip-installed sdist was uninstalled. So maybe leave the warning off till pip stops supporting sdist installs.

[qwcode]

So maybe leave the warning off till pip stops supporting sdist installs.

but I did say "for consistency, we could also start writing this file into egg-info metadata."
or for sdists we could just look for the pip-only installed-files.txt

[dstufft]

Maybe we don't want to start respecting it immediately, but we should start writing it when we can so that there's a corpus of them out there.

[pfmoore]

Agree entirely re writing INSTALLER for wheels and respecting it when present. What I want to avoid is someone doing pip install X; pip uninstall X and getting told pip didn't install X, is it OK? That could happen if it's a sdist install and setuptools starts writing its own INSTALLER file or something, but we want to be careful to avoid it because it's a horrible user experience (it calls into question the reliability of the warning).

That one proviso aside, I'm +1 on the idea.

[qwcode]

What I want to avoid [...] getting told pip didn't install X, is it OK?
That could happen if it's a sdist install and setuptools starts writing its own INSTALLER file

agreed, but like I said, I would only ever consider turning on the warning, if either pip was writing INSTALLER to the egg-info install, or pip was checking for it's own installed-files.txt

[pfmoore]

I think we're violently agreeing :-)

[szepeviktor]

Could you estimate the date of this decision?

[dstufft]

I'm gonna kick this out of 6.0, I think the original installer file doesn't contain enough information to actually do the kinds of things we want to be able to do with it, and given nobody else implements it I think we'd be better off moving to something better.

[ncoghlan]

Now that most sdist installs also go through a wheel archive, just implementing this for wheels seems like a potentially useful path forward. I'm getting some work time to contribute to pip now, so I'll tackle creating a PR for that as my first task.

[ncoghlan]

PR #3284 implements an initial version of this that generates the INSTALLER file just before generating the RECORD file when installing from a wheel archive.

Even if we decide to define a different format for the file itself (e.g. by using something more inherently extensible, like JSON), it's possible to iterate on the tests and where we want to put this in the code.

The initial location I chose means the new file will be generated whenever pip installs from a wheel archive, whether a pre-existing one, or one it created from a source format.

[ncoghlan]

One limitation of this initial attempt is that a system build script that uses pip to handle the installation at package build time will still claim that "pip" installed the package. I'm not sure what the UX should be for overriding that - specifier an "installer metadata file" is the most future-proof, but a bit harder to work with in a package build process, while a command line argument for the installer name would mean that every new piece of installer metadata would require a new command line argument.

[Ivoz]

How about a single line that constructs what to text to write, that say a distro could patch in their system pip, if they wanted?

I was thinking that specifying the version of pip that installed a package would be very nice to have as well as just saying it's pip. This might be helpful in the distant future; where a future pip could then know how a package got installed, because it was installed using pip version X.

[xavfernandez]

Cf @ncoghlan response: #3284 (comment) on that subject.
I'd be in favor of closing this ticket as implemented by #3284.

[ncoghlan]

The version of pip shouldn't matter - if the way something needs to be uninstalled changes, then that means a change in the dist-info format.

The distutils-sig discussion I mention in the PR comment is at https://mail.python.org/pipermail/distutils-sig/2016-January/028080.html

It kinda fizzled without a decision, so I think this issue is still valid - pip setting INSTALLER to "pip" is only step 1, the next step is defining what pip does on uninstall when it finds something that isn't "pip" in that file.

[ncoghlan]

As far as distros patching their pip to use a different tag goes, that wouldn't be desirable - the distinction we want to make is between "pip used as a build tool" (e.g. creating an RPM from an SRPM) and "pip as a system management tool" (i.e. running pip directly), which fits with a command line flag or configuration file setting, not with a patch to the system pip package.

[pradyunsg]

@ncoghlan Can we close this issue?

[pradyunsg]

(content: #5346 is happening now)

[ncoghlan]

@pradyunsg Even with #5346, I believe that sudo pip uninstall systemd-python would still try to uninstall systemd-python, even though that doesn't have a valid INSTALLER file saying pip.

[ncoghlan]

Closing this as pip populates INSTALLER, and has done for a while.

#5605 covers getting pip to skip updating packages installed by system package managers.