Add more granularity to _pyrepl syntax theming for "keywords"

[treyhunner]

Feature or enhancement

Proposal:

I made a custom _colorize theme to try making my REPL syntax highlighting look similar to my text editor's.

This is the theme I used:

theme_with_dim_operators = default_theme.copy_with(
    syntax=Syntax(
        keyword=ANSIColors.GREEN,
        builtin=ANSIColors.BLUE,
        comment=ANSIColors.INTENSE_YELLOW,
        string=ANSIColors.CYAN,
        number=ANSIColors.CYAN,
        definition=ANSIColors.BLUE,
        soft_keyword=ANSIColors.BOLD_GREEN,
    ),
)

I then used this example code to check the highlighting between the two:

from math import pi
print(f"Welcome to Python {pi:.2f}")
name = None
if name is None:
    print("It's", True)

Pasting that code in my text editor (vim), looks like this:

Pasting the same code in the REPL, looks like this:

The granularity with syntax highlighting needs to stop somewhere, but I feel it would be nice to add a bit more granularity than is currently supported by _colorize.

In particular, it would be nice to distinguish between:

1. Syntax keywords in general (if, is, def, for, with, etc.)
2. Keywords that are variables (True, False, None)
3. Imports in particular (from, import)

I feel that distinguishing between 1 and 2 is more important than between 1 and 3. While None is a keyword, seeing it appear in the same color as is and other keywords feels odd.

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Linked PRs

• gh-134953: Expand theming for True/False/None #135000

[treyhunner]

I looked into how Pygments highlights and how MagicPython (used by VS Code) highlights. Both have much more granularity than I expect _colorize will ever support, but I do think it's worth giving _colorize slightly more granularity in syntax highlighting.

Pygments uses:

• Keyword.Constant for True, False, and None
• Keyword.Namespace for import and from
• Keyword for if and most other keywords
• Operator.Word for is, and, or, and not

It seems like MagicPython uses:

• constant.language.python for True, False, and None
• keyword.control.import.python for import and from
• keyword.control.import.python for if and most other keywords
• keyword.operator.logical.python for is, and, or, and not

Both of these syntax highlighters also seem to support a hierarchy (if custom highlighting is not defined for and/or/is then the same highlighting used for if and other keywords will be used.