bpo-44010: IDLE: colorize pattern-matching soft keywords

[taleinat]

This implements highlighting of the match, case and _ soft keywords, following the syntax defined in the docs to the extent reasonably possible within the framework of the current colorization method.

https://bugs.python.org/issue44010

[terryjreedy]

Code review only so far.

[bedevere-bot]

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

[terryjreedy]

I now agree that this is much more important than f-string field expressions. Highlighting helps one read complicated code and also to understand some errors. Field expressions tend to be short and simple. Most would get no markup other than 'not string'.

[taleinat]

I've refactored the code somewhat and made it mark all lone underscores in case <...>: as keywords (after going down several wrong roads...). I think it'll need more tweaking to handle guards, though.

[taleinat]

I still haven't done away with the duplicated test code fragment, but I'll make sure to do so if we move forward with this.

[terryjreedy]

'_' is only and always a keyword in patterns, but not in guards or anywhere else. Simple rule, fortunately. The guards are standard python, as are the actions and match line. Doc has been patched to say do. The example from one of the outdated comments 2 day ago:

>>> _ = 'a'
>>> match _:
...     case _ if _: print(_, 'ok')
...  
a ok

[E-Paine]

I am not sure this quite works, sadly. IMO, we need to recursively colour all content in the guard:

case _ if ("a" if _ else set()): pass

[taleinat]

'_' is only and always a keyword in patterns, but not in guards or anywhere else.

A bare _ is also explicitly, specifically disallowed in pattern-matching "capture groups", i.e. case <pattern> as _: is disallowed.

The latest commits do mark a bare _ as a keyword in this case, which I think we agree is the desired behavior.

[taleinat]

I am not sure this quite works, sadly. IMO, we need to recursively colour all content in the guard:

case _ if ("a" if _ else set()): pass

Fixed, and I've added that example to the tests.

[taleinat]

@terryjreedy, I've made the doc changes you suggested.

I've also updated help.html. Note that it will require manual backporting to continue showing correct version numbers, so I'm removing the auto-backport label.

[taleinat]

I have made the requested changes; please review again.

[bedevere-bot]

Thanks for making the requested changes!

@terryjreedy: please review the changes made to this pull request.

[terryjreedy]

The version numbers and other extraneous material in help.html are outside of the part of the file displayed, which is <div class='section>, lines 103 to 878. They have always been the same in all active versions so that there is never an issue with backporting.

This is the first time we have not backported a doc change to all versions. So if we do nothing, there would be merge conflicts the next time we backport a doc change to 3.9. To avoid this, help.html should be copied, after merging, to the 3.9 branch. The new paragraph, lines 584 to 588, should be deleted. The change on 693, which should have been incorporated into the file previously, and all other changes, should be merged into 3.9.

I have considered moving the 'section' extraction mechanism from display to copying. The result would have to be tested and might be less robust against sphinx formatting changes, but the diffs and the code would be simpler.

[taleinat]

@terryjreedy, do you want anything else done here? Are you okay with this being merged in its current form?

To avoid this, help.html should be copied, after merging, to the 3.9 branch. The new paragraph, lines 584 to 588, should be deleted. The change on 693, which should have been incorporated into the file previously, and all other changes, should be merged into 3.9.

Alright, we can take care of that in a followup PR.

[terryjreedy]

News entries look fine, so go ahead. I will handle idlelib/NEWS separately. Also 3.9 doc patch if you don't.

[miss-islington]

Thanks @taleinat for the PR 🌮🎉.. I'm working now to backport this PR to: 3.10.
🐍🍒⛏🤖

[bedevere-bot]

GH-26237 is a backport of this pull request to the 3.10 branch.

[taleinat]

Alright, we can take care of that in a followup PR.

See PR GH-26239.