pip install with redirection does not hit the cache

[tibortakacs]

Description

If the pypi repository (specified in --index-url) uses URL redirections with changing URLs, the cache does not provide the previously downloaded and stored wheels from the cache but provides "No cache entry available" message and downloads the wheel again.

One example is Azure ADO feeds that use URL redirection in a certain way: the actual wheels are stored in a blob storage whose URLs changes in each call. It looks like that pip stores the responses specifically to the URLs, the cache entries are never used again.

Expected behavior

I would expect that the wheel the is downloaded after the chain of redirections are referenced to the original URL in the cache.

pip version

Latest main (a057c9b) and 22.3.1

Python version

Python 3.8

OS

Ubuntu 20.04

How to Reproduce

I executed the following installation command twice:
pip install black -vv --index-url=https://microsoft.pkgs.visualstudio.com/MyProject/_packaging/my_repository/pypi/simple/

Output

Here is the relevant part of the output of the first run:

  Looking up "https://microsoft.pkgs.visualstudio.com/<ID_A>/_packaging/<ID_B>/pypi/download/black/22.12/black-22.12.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl" in the cache
  No cache entry available
  https://microsoft.pkgs.visualstudio.com:443 "GET /<ID_A>/_packaging/<ID_B>/pypi/download/black/22.12/black-22.12.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl HTTP/1.1" 303 0
  Status code 303 not in (200, 203, 300, 301, 308)
  Looking up "https://vsblob<BLOB>.vsblob.vsassets.io/b-<LONG_URL_X>black-22.12.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl%<...>" in the cache
  No cache entry available
  Starting new HTTPS connection (1): vsblob<BLOB>.vsblob.vsassets.io:443 "GET ...

And, from the second run:

  Looking up "https://microsoft.pkgs.visualstudio.com/<ID_A>/_packaging/<ID_B>/pypi/download/black/22.12/black-22.12.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl" in the cache
  No cache entry available
  https://microsoft.pkgs.visualstudio.com:443 "GET /<ID_A>/_packaging/<ID_B>/pypi/download/black/22.12/black-22.12.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl HTTP/1.1" 303 0
  Status code 303 not in (200, 203, 300, 301, 308)
  Looking up "https://vsblob<BLOB>.vsblob.vsassets.io/b-<LONG_URL_Y>black-22.12.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl%<...>" in the cache
  No cache entry available
  Starting new HTTPS connection (1): vsblob<BLOB>.vsblob.vsassets.io:443 "GET ...
  https://vsblob<BLOB>.vsblob.vsassets.io:443

It shows that the redirected URLs (the actual blob URLs, LONG_URL_X and LONG_URL_Y) are different in the two runs. I think this is the reason why there is no cache hit.

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[uranusjr]

See discussions in #10694.