Deprecate setup.py install

[astrojuanlu]

The good old setup.py install invokation is deprecated. Paul Ganssle described it in great depth here: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html

Despite the fact that no one will read more than the next 3 paragraphs, I hope that this article can be useful when you want to advocate against the use of setup.py: when you make a PR or a comment in a slack channel, you can link to this Proustian monstrosity and hope that your audience pales before the prospect of reading through the whole thing and just assents to whatever you're asking them to do.

It has already caused us some issues due to the recent docutils 0.18 bug, because it turns out that, for projects using method: setuptools in their configuration, pinning docutils<0.18 in setup.py doesn't actually get docutils 0.17, as described here #8639

Similarly to #8623, we can start by removing mentions of this from the documentation. However, if we want to proactively act on this, we should warn all projects using this method to either switch to method: pip, or cap setuptools somehow.

We know though that the projects that don't follow this advice will have broken builds, and this will result in a massive stream of support issues again. Therefore, what we can do is to replace this line:

readthedocs.org/readthedocs/doc_builder/python_environments.py

Line 200 in 3c4b495

'setuptools',

by setuptools<58.3.0, because from that version onwards, the invocation might be removed at any time: https://github.com/pypa/setuptools/blob/main/CHANGES.rst#v5830

Thoughts @readthedocs/backend @readthedocs/advocacy?

[agjohnson]

That is a great way to ensure the issue doesn't immediately pop up on us. I'd agree that's our first step, though we also don't want to linger around with the pinned dependency for too long. But this at least buys some time for us to figure out a solution with good UX.

Alternatively, we could also only pin this dependency if the project is using the method: setuptools option

We have talked about increased error messages and notifications emitted from builds, which I very much want. This might be a good application for detecting incompatibilities with dependencies and giving some helpful errors to users, however we probably also want to be more proactive and message the user before their project starts failing. I agree we should be doing what we can to avoid issues like the docutils 0.18 issue.

I believe the most simple version is to use the stored build configuration on Build to pull up a list of projects using the setup.py install configuration option and notify them with some guidance and what to expect.

[stsewd]

Also, we have this option exposed in the UI

[astrojuanlu]

I tried to send a small pull request removing mentions of setup.py, but it is everywhere in the FAQ because of the aforementioned UI option. I think we should start there.

For example, we could show it as grayed out/blocked/disabled with whatever value it has for the current project, like this:

[astrojuanlu]

I went ahead and opened #8711. The breakage can happen any time now, and it is good for our users that we read deprecation warnings for them and provide a happy path for those that couldn't care less.

This issue can remain open to discuss deprecation of setup.py install invokations on our platform.

[kif]

The doc of pyFAI built with RTD is broken since pygment upgraded to 2.13 (15/8/2022) which switched to importlib_metadata with python3.7.
Apparently, the fact that RTD uses python setup.py install --force breaks the entry-points listing in importlib_metadata.

I found several work-around:

• Upgrade CI to use python 3.8+, since importlib_metadata is used only for python3.7
• replace python setup.py install --force with pip install . in the pipeline of RTD.

[humitos]

I created a Metabase question to know how many projects are using this feature python.install.method == "setuptools" over time. https://ethicalads.metabaseapp.com/question/255-projects-using-setuptools-for-python-install-method?days=30. Currently, we have 406 projects using this feature in the last 60 days.

We should encourage them to migrate to python.install.method == "pip" as confirmed by @kif that works fine on Read the Docs. This can be done via https://github.com/readthedocs/readthedocs-assistant for those projects using GitHub. Besides, we should send an email to all of them communicating an EOL support for this feature.

[benjaoming]

The UI update mentioned in #8659 (comment) would still be nice to have. Rendering the option readonly is not a dead-end, since all projects can both change/enable/disable it in their .readthedocs.yml.

[humitos]

Currently, we have 406 projects using this feature in the last 60 days.

I checked this today and we have 315.