Spread the Windows test load between Azure & Appveyor

[xavfernandez]

Cf #6791 for the rationale

[pradyunsg]

Appveyor is angry at you Xavier.

[pfmoore]

This looks suspiciously like pypa/virtualenv#1339

You need to be very careful to get a completely up to date virtualenv when using 3.7, as behaviour changes within the (internal details of) minor versions of 3.7 repeatedly broke virtualenv. I think we got all the edge cases dealt with, but if anyone finds a new one, it'll need addressing (although @gaborbernat is rewriting virtualenv to use the core venv mechanisms, so hopefully this whole mess will be behind us soon).

[xavfernandez]

Appveyor is angry at you Xavier.

Appveyor seems to be quite sensitive indeed.

This looks suspiciously like pypa/virtualenv#1339

yup, it looks like it. I moved 3.7 build to azure to avoid this (or get the same error on Azure ^^).

[pradyunsg]

get the same error on Azure ^^

Ah, optimism. ;)

---

@pfmoore this was the issue with the behavior varying for patch versions of CPython 3.7, right?

FWIW, pip "officially" only supports the latest patch versions of each non-EOL minor version. Maybe we should use the newest CPython minor version on CI (via pyenv or equivalent), with caching to optimize performance?

[xavfernandez]

Well Azure seems more sympathetic and tests seem to go smoothly.

• Appveyor build time did not increase dramatically https://ci.appveyor.com/project/pypa/pip/history
• Azure build time did decrease slightly https://dev.azure.com/pypa/pip/_build?definitionId=6&_a=summary&view=buildsHistory
• Integration tests are now run on Python3.7-x64

[pfmoore]

@pradyunsg Looks like the issue is resolved. But yes, in theory we could force use of the latest version. Seems like a lot of bother though. Making sure we use the latest virtualenv should be enough.

Personally, I'm a little frustrated that this stuff got changed in CPython minor releases without proper deprecation - but it is internal details, so it's hard to get too upset (it's just that there's no supported way to implement virtualenv, using internals is necessary - I'm so looking forward to the rewrite when we can finally put all of this rubbish behind us) .

[pradyunsg]

Seems like a lot of bother though. Making sure we use the latest virtualenv should be enough.

Fair enough. 🙃

[pradyunsg]

Merging since no one is opposed to doing this.