Question: Are DiagnosticTags deprecated in favor of semantic tokens?

[jfly]

I've seen a some conversations that imply that semantic tokens obsolete DiagnosticTag:

• feat(diagnostic): add support for tags neovim/neovim#22747 (comment): "the diagnostics themselves are just a precursor to semantic tokens (which came after)."
• reportUnusedVariable ignore pattern pyright#1118 (comment): "Pyright's implementation of this functionality does indeed predate the introduction of semantic tokens. [...] If you'd like to build your own language server on top of pyright (e.g. to convert "unreferenced code" tagged hints into semantic tokens), you're welcome to do so."
• reportUnusedVariable ignore pattern pyright#1118 (comment): "for Pyright to switch these tagged diagnostics to semantic tokens"
• clangd even has a (currently undocumented) extension to the Language Server Protocol that adds a new textDocument/inactiveRegions request: Document the textDocument/inactiveRegions extension llvm/clangd-www#104. I think this is implemented using the semantic tokens api, but I haven't checked.
• SemanticTokensModifier: Add unused token modifier vscode-languageserver-node#604

But the discussion over on microsoft/vscode#92157 has a comment from @alexdima that says:

I am not in favor of adding competing API that gives two distinct ways to achieve the same effect.

So I'm confused. @HighCommander4, maybe you know the answer? Why does clangd need an extension to the Language Server Protocol to do this? Is it related to marking large regions of code as unused?

[HighCommander4]

So I'm confused. @HighCommander4, maybe you know the answer? Why does clangd need an extension to the Language Server Protocol to do this? Is it related to marking large regions of code as unused?

Inactive regions are a feature for dealing with conditional preprocessor branches in C/C++ code, for example:

#ifdef WINDOWS
// windows-specific declarations
#else
// declarations specific to other platforms
#endif

If the active build configuration is Windows, the region between the #else and the #endif is inactive (not compiled in this configuration), and we want to visually identify that in the editor in some way.

Diagnostics don't seem appropriate here, as a region being inactive does not indicate anything wrong with the code, or any action that should potentially be taken by the programmer (unlike, say, unused code, where the programmer may want to remove it).

We decided to go with our own protocol rather than using semantic tokens for the following reasons:

• Semantic tokens did not make it easy for the editor to apply a whole-line style to the region (e.g. grey background color). We hacked it together in vscode with middleware, but it was messy and having a dedicated protocol made it much cleaner.
• Semantic tokens would overwrite client-side highlighting tokens (e.g. TextMate tokens in VSCode), but it was actually useful to apply client-side highlighting to inactive regions, because it made them more readable to have the individual tokens within an inactive region colored differently.

[jfly]

Thanks for the quick feedback!

Diagnostics don't seem appropriate here, as a region being inactive does not indicate anything wrong with the code, or any action that should potentially be taken by the programmer (unlike, say, unused code, where the programmer may want to remove it).

I think this is common confusion about the spec. As I understand it (and this is really just me parroting @erictraut here), diagnostic hints are not necessarily something actionable. This confusion seems to partially be due to an unfortunate choice of words in the spec: "unnecessary" implies something is wrong, whereas (IMO), it really should be "unused". I filed a separate issue about this: #2025

I have no idea if this actually would address the issues you raised about implementing the client side of this.

EDIT: Nevermind, after chatting with Neovim, I'm convinced this is just underspecified in the spec. I've filed #2026 to try to address this.