reST Primer duplicates the Sphinx version; should link to it instead

[CAM-Gerlach]

Similar to PEP 12, the reST primer in the devguide duplicates the same information in the Sphinx reST primer, along with some bits specific to its particular context.

In fact, unlike PEP 12, they are mostly word for word identical and thus clearly derive from a common source, with the Sphinx version being close to a strict superset of the CPython version, containing some additional and updated sections but also still mentioning many CPython-specific conventions (e.g. section heading format).

This near-duplication is clearly not very DRY, and has several main problems:

• Additions, improvements and updates/changes to one are not reflected in the other. Clearly, the more appropriate place for this is the Sphinx docs as it is generally applicable to all Sphinx users, and indeed they seem to have been kept much more up to date there, and it also links the full reST reference for each section for readers looking for more information.
• Readers already familiar with reST or just looking for the CPython-specific bits must either dig through each section, or may skip it entirely and miss them

Therefore, I propose for the Devguide reST primer:

• Recommending and linking the Sphinx reST primer at the top of the Devguide reST primer section
• Linking each top-level devguide reST primer section to the corresponding Sphinx section via Intersphinx
• Eliminating all but the CPython-specific guidance/convention/recommendations in each section

There are also some non-specific bits that could be trimmed from the Additional Markup Constructs section, and each role/directive should be linked to its canonical full Sphinx documentation if present, but that can be addressed separately on a case by case basis.

If we agree this is desirable, I'm willing to take this on, unless @ezio-melotti would prefer to do it.

Somewhat related: #916

[CAM-Gerlach]

This was brought up again at the December docs community meeting, where this seems to be a reasonable step to move forward with. @Mariatta @encukou or anyone (it will have to be one of the few people with write permission on this repo), could you transfer this issue to the devguide repository, as it is more appropriate there? Thanks!

[CAM-Gerlach]

Thanks @encukou ! Funny enough, it's issue 1000 for the devguide, created within 24 hours of issue 100000 for the CPython repo.

python/peps#2337 is the equivalent for PEP 12 in the PEPs repo, for reference.

[hugovk]

Therefore, I propose for the Devguide reST primer:

• Recommending and linking the Sphinx reST primer at the top of the Devguide reST primer section
• Linking each top-level devguide reST primer section to the corresponding Sphinx section via Intersphinx

Good idea, I sometimes want to find out more info. Occasionally I need to dip into https://docutils.sourceforge.io/docs/user/rst/quickref.html and https://docutils.sourceforge.io/docs/ref/rst/restructuredtext.html.

• Eliminating all but the CPython-specific guidance/convention/recommendations in each section

I'm generally fine with some duplication, I find the devguide page very convenient as a one-stop reference guide. In fact, I'm also using it when doing non-CPython docs too. The Sphinx page is good though.

[ezio-melotti]

I agree with @hugovk. I find the devguide page very convenient, since I know where to find it, and I know that (almost) everything I need is in that page. Docutils and Sphinx docs have several pages each and finding what you need is generally more time-consuming.

• Recommending and linking the Sphinx reST primer at the top of the Devguide reST primer section

I would also add https://docutils.sourceforge.io/docs/user/rst/quickref.html. This already has links to the docutils reST specification, like you suggested in:

• Linking each top-level devguide reST primer section to the corresponding Sphinx section via Intersphinx

However it doesn't include some additional Sphinx constructs.

• Eliminating all but the CPython-specific guidance/convention/recommendations in each section

What do you mean exactly? For example the Lists and quotes section includes both examples and explanations. If you just leave the examples it will end up looking like https://docutils.sourceforge.io/docs/user/rst/quickref.html#bullet-lists and we might end up duplicating the quickref instead. If you leave an empty section with just a link to the Sphinx reST primer it will have roughly the same content but will need an extra click.

Another option would be to just download the Sphinx reST primer and .. include:: it after the CPython-specific sections. It's not very elegant but solves the DRY problem and doesn't require extra clicks for the user. Would this be an acceptable solution?

There might also be some extension that does that automatically without having to download it manually, but I haven't looked.

[CAM-Gerlach]

Sorry if it was unclear; to clarify, right now I'm specifically focused on the reStructuredText Primer section of the reStructuredText Markup pag, rather than the rest of the page.

I have investigated it further, and the TL;DR is that aside from a few minor tweaks, I've confirmed that this specific section is a nearly exact copy of the Sphinx reStructuredText Primer...from before it was initially committed to the Sphinx repository nearly 15 years ago.

FULL INVESTIGATION RESULTS

I created a branch on my fork containing the four versions as the contents of the documentation/markup.rst file in four separate commits, for your ease of comparison. Other than removing the other top-level sections from the devguide page, I've made no changes to the document content. These are:

• CAM-Gerlach@b01e53e contains the first version committed in 3160c41 to the devguide repository in Jan 2012, when it was apparently moved from https://docs.python.org/dev/documenting/
• CAM-Gerlach@e3f9871 contains the current version on the devguide
• sphinx-doc/sphinx@ccfd92e contains the very first version committed in sphinx-doc/sphinx@ccfd92e to the Sphinx repository in March 2008, along with the rest of the initial Sphinx documentation
• CAM-Gerlach@e088dc4 contains the current version in Sphinx

If we compare the first-ever March 2008 Sphinx version against the original devguide version, only a few lines are different, encompassing two trivial wording tweaks due to not being a part of CPython's own documentation and two modest additions. Given that, it seems likely that the devguide copy is from an even older version of the document before it was checked in to the Sphinx repository, as it is missing the additions/clarifications while having no changes of its own.

Now comparing that original devguide version against the current version, we only see a small handful of changes in the 11 years since then, and even fewer of them substantive—adding a few ref target labels and a couple ref links, changing a heading level style, updating a URL, and a couple small wording clarifications.

By contrast, comparing the original Sphinx version against the current one, we can see there has been substantial updates, clarifications improvements and additions over the same time period, including incorporating or superseding all but a couple of those above.

Taking the results and feedback above into account, my revised proposal is to:

• Replace the nearly 15 year old static copy of this document in the reStructuredText Primer section with a prominent link to the up-to-date, improved and expanded version in the Sphinx docs (and update the handful of refs accordingly, either via URLs or Intersphinx)
• Ensure the very few minor tweaks not already reflected there are contributed upstream, and the one short section describing the convention for heading styles (which is also included upstream, but is CPython-specific) is moved to the Style Guide (where the Sphinx reST primer actually links it to)
• Leave the rest of the document exactly as it, aside from eliminating the now redundant layer of nesting all the sections under Additional Markup Constructs

[CAM-Gerlach]

I'm generally fine with some duplication, I find the devguide page very convenient as a one-stop reference guide. In fact, I'm also using it when doing non-CPython docs too.

My further investigation has confirmed for the reST primer, the entire section is essentially a duplicate of the Sphinx version from ≈15 years ago, with a few minor tweaks since. The fact that people are still relying on it for other docs, rather than the updated, improved and extended official version, seems a little unfortunate and is IMO a good reason to point users to the canonical version instead of keeping around a 1 1/2 decade old copy.

I would also add docutils.sourceforge.io/docs/user/rst/quickref.html. This already has links to the docutils reST specification

To note, the revised proposal eliminates that part and just links to the Sphinx version of the primer directly, which has abundant links to the reST spec throughout and also links it and the old docutils primer and quickref (which the Sphinx primer supercedes both of) at the top.

Another option would be to just download the Sphinx reST primer and .. include:: it after the CPython-specific sections. It's not very elegant but solves the DRY problem and doesn't require extra clicks for the user. Would this be an acceptable solution?

Hmm, well this would make what is by far the longest page on the devguide even longer and would deny a clear separation between the general reST/Sphinx content and the sections focused specifically on CPython. It would also need to be investigated from a licensing perspective, as it would mean this repo is no longer cleanly licensed PSF-2.0 and fully covered by the CLA.

And of course, it would also add non-trivial complexity, fragility and maintenance issues, as we'd need to:

• Design, write and test the tooling to do that, and ensure it runs regularly
• Add and keep up to date all the Sphinx extensions the page uses
• Add and maintain the conf.py config (e.g. extlinks, custom directives) that affects it
• Add the Sphinx docs as an Intersphinx target, including for the disabled by default :doc: role (and risk breakage, collisions and confusion accordingly)
• Hope they don't use any Sphinx roles/directives/syntax newer than what our version can support

Ultimately, I'm having trouble understanding the justification for all this—I'm not aware of where else in the devguide, CPython docs or elsewhere where we straight copied a whole page of the existing documentation of an external language or tool (e.g. C/++, GCC/Clang, MSVC, GDB, etc) into ours rather than just discussing CPython-specific guidance and linking to an appropriate canonical resource for the rest. The closest I can see is the Git cheatsheet, but even then the content there is focused specifically on using Git in the context of CPython (as the rest of the docs content is here), as opposed to being copied from the Git manual. I guess I'm just not understanding what is so special about this particular case that justifies this approach?

[ezio-melotti]

I guess I'm just not understanding what is so special about this particular case that justifies this approach?

It's just a matter of convenience/habit, since some of us visit this page semi-regularly. If this page didn't exist in the first place, we would have just gotten used to the Sphinx primer instead.

The points you raised are valid though, so I guess we can handle an extra click and/or update our habits :)

If we still end up finding it inconvenient, we can always update the quick reference or add more sections/links.