python: Enhance syntax highlighting for type hints

[ClanEver]

Release Notes:

• Python: Enhance syntax highlighting for type hints

Before

After

---

Why manual recursion?

• Due to tree-sitter grammar not supporting recursion in query (Recursive nodes are difficult to match in queries tree-sitter-grammars/tree-sitter-lua#24), currently only manual recursion is possible (refer to https://github.com/projekt0n/github-nvim-theme/pull/250/files).

Unable to highlight when simple structures appear before complex structures, example:

def t() -> str | dict[int, dict[int, dict[int, str]]]:
    pass

Because complex structures are parsed as subscript rather than generic_type by tree-sitter in this case ☹

Related:

• Python Syntax Highlighting for t | None is incorrect #14715
• Union Type (Python Doc)
• Type parameter lists (Python Doc)

[zed-industries-bot]

	Messages
📖	This PR includes links to the following GitHub Issues: #tree-sitter-grammars/tree-sitter-lua#24, #14715 If this PR aims to close an issue, please include a Closes #ISSUE line at the top of the PR body.

Generated by 🚫 dangerJS against 1a2db3f

[maxdeviant]

@maxbrunsfeld Do you have thoughts on this approach?

[maxbrunsfeld]

I don't think we should land this code. I get that it's working around a limitation in the query system right now, but I think we just need to fix that limitation.

I worry this is going to have a negative performance impact, and it's also just hard to understand and maintain.

[maxbrunsfeld]

@ClanEver Can you back out the one complex pattern from the query (sorry that it's not possible to highlight union types cleanly with our current scheme) so that we can land the other changes in the query?

[ClanEver]

@maxbrunsfeld Would it be acceptable to reduce the depth to 3?

; Union type X | Y (up to 3 types)
(type
  (binary_operator
    left: [
      (binary_operator
        left:  (_) @type
        right: (_) @type
      ) @type
      (_) @type
    ]
    right: (_) @type
  ) @type
)