Skip to content

Add default semantic token rules for C# (Roslyn) token types - #60027

Closed
tatsudo wants to merge 1 commit into
zed-industries:mainfrom
tatsudo:semantic-tokens-csharp-roslyn-rules
Closed

Add default semantic token rules for C# (Roslyn) token types#60027
tatsudo wants to merge 1 commit into
zed-industries:mainfrom
tatsudo:semantic-tokens-csharp-roslyn-rules

Conversation

@tatsudo

@tatsudo tatsudo commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

Roslyn reports an extended token-type legend (field, controlKeyword, recordClass, extensionMethod, ...) that the default rules did not cover, so those tokens were silently dropped and C# fell back to tree-sitter-only highlighting. Map them to existing theme styles.

Objective

Companion to #60015. This PR adds highlighting support for Roslyn's (C#) semantic token types, so C# code can be colored correctly instead of being limited to the standard LSP token set.

Zed's default semantic token rules only cover the ~24 standard LSP token types, and there's no catch-all rule. Roslyn reports a much larger, C#-specific legend (field, controlKeyword, recordClass, extensionMethod, ...), so any token whose type isn't one of those 24 is silently dropped — leaving C# fields, constants, control-flow keywords, records, etc. without semantic highlighting (a private field stays indistinguishable from a local variable).

Solution

Add default rules in default_semantic_token_rules.json mapping Roslyn's C# token types to existing theme styles (with fallback chains, in the existing // C# section).

Testing

  • Added a test asserting the default rules parse and cover the added C# token types (guards against accidental removal).
  • Verified manually on a C# project: fields, constants, control-flow keywords, records and extension methods now get semantic highlighting consistent with the theme.

Self-Review Checklist:

  • I've reviewed my own diff for quality, security, and reliability
  • Unsafe blocks (if any) have justifying comments
  • Tests cover the new/changed behavior
  • Performance impact has been considered and is acceptable

Showcase

Click to view showcase

Before — Roslyn's C# token types aren't mapped, so fields/keywords fall back to tree-sitter (a field looks like a plain variable):

Screenshot 2026-06-28 at 10 06 41

After — fields, constants, control-flow keywords and records get semantic colors:

Screenshot 2026-06-28 at 09 58 15

Release Notes:

Improved C# semantic highlighting by mapping Roslyn's token types (fields, constants, records, control-flow keywords, and more) to theme styles.

Roslyn reports an extended token-type legend (field, controlKeyword,
recordClass, extensionMethod, ...) that the default rules did not cover, so
those tokens were silently dropped and C# fell back to tree-sitter-only
highlighting. Map them to existing theme styles.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@cla-bot cla-bot Bot added the cla-signed The user has signed the Contributor License Agreement label Jun 28, 2026
@zed-community-bot zed-community-bot Bot added the first contribution the author's first pull request to Zed. NOTE: the label application is automated via github actions label Jun 28, 2026
xfight-eleven pushed a commit to xfight-eleven/zed that referenced this pull request Jun 28, 2026
…ries#60015)

In C# files nothing gets semantic highlighting — class fields and other
identifiers are colored by tree-sitter only, so a private field looks
the same as a plain local variable.

The reason: Roslyn (the C# language server) doesn't declare
`semanticTokensProvider` statically in its `initialize` response. It
registers it **dynamically** (`client/registerCapability`), and only
when the client advertises
`textDocument.semanticTokens.dynamicRegistration = true`. Zed advertised
`false` and didn't handle such a registration, so Roslyn never offered
semantic tokens at all. (rust-analyzer/gopls are unaffected — they
declare the capability statically.)

This is the first of two PRs. This one makes Roslyn actually **send**
semantic tokens. The companion PR (zed-industries#60027) maps Roslyn's C#-specific
token types to theme styles — without it the tokens arrive but most are
dropped, since their types aren't in Zed's default rules.

## Solution

- Advertise `textDocument.semanticTokens.dynamicRegistration = true`.
- Handle the `textDocument/semanticTokens` registration and
unregistration so the capability is stored, following the existing arms
for `documentLink`, diagnostics, etc.

## Testing

- Added a test that dynamically registers and unregisters
`textDocument/semanticTokens` and checks the stored capability appears
and is cleared.
- Verified manually against Roslyn on a C# project: Zed now sends
`textDocument/semanticTokens/full` and gets tokens back; before this
change there was no semantic-token traffic at all.

## Self-Review Checklist:

- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable

---

Release Notes:

- Support dynamic registration of the `textDocument/semanticTokens`
capability.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@MrSubidubi

Copy link
Copy Markdown
Member

Thanks for this!

I see no reason to not just ship this with the C# extension, hence am gonna close this at this time. Feel free to inform me here should I be missing something. Thanks!

@MrSubidubi MrSubidubi closed this Jun 28, 2026
@MrSubidubi MrSubidubi self-assigned this Jun 28, 2026
MrSubidubi pushed a commit to zed-extensions/csharp that referenced this pull request Jul 17, 2026
Roslyn reports an extended semantic-token legend (`field`,
`controlKeyword`, `recordClass`, `extensionMethod`, ...) beyond the
standard LSP types. Without rules for them, those tokens are dropped and
C# falls back to tree-sitter-only highlighting (e.g. a private field
looks like a local variable).

This adds `languages/csharp/semantic_token_rules.json` mapping the
C#-specific types to theme styles. Standard types stay covered by Zed's
built-in defaults, so only the Roslyn-specific ones are listed here
(same approach the built-in rust/cpp/go/python rules use).

Fixes #85. Moved here from zed-industries/zed#60027 per @MrSubidubi's
suggestion. Needs zed-industries/zed#60015 (dynamic registration of
`textDocument/semanticTokens`) to actually deliver the tokens — together
they resolve #85.

## Showcase

<details>
  <summary>Click to view showcase</summary>

**Before** — Roslyn's C# token types aren't mapped, so fields/keywords
fall back to tree-sitter (a field looks like a plain variable):

<img width="979" height="1089" alt="Screenshot 2026-06-28 at 10 06 41"
src="https://github.com/user-attachments/assets/26a79bc9-f7c6-4700-8b39-0505af75180c"
/>

**After** — fields, constants, control-flow keywords and records get
semantic colors:

<img width="934" height="1032" alt="Screenshot 2026-06-28 at 09 58 15"
src="https://github.com/user-attachments/assets/def44b05-85fa-4b39-9ea7-74ceb667bf6d"
/>

</details>

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
jolutz pushed a commit to jolutz/zed that referenced this pull request Aug 8, 2026
…ries#60015)

In C# files nothing gets semantic highlighting — class fields and other
identifiers are colored by tree-sitter only, so a private field looks
the same as a plain local variable.

The reason: Roslyn (the C# language server) doesn't declare
`semanticTokensProvider` statically in its `initialize` response. It
registers it **dynamically** (`client/registerCapability`), and only
when the client advertises
`textDocument.semanticTokens.dynamicRegistration = true`. Zed advertised
`false` and didn't handle such a registration, so Roslyn never offered
semantic tokens at all. (rust-analyzer/gopls are unaffected — they
declare the capability statically.)

This is the first of two PRs. This one makes Roslyn actually **send**
semantic tokens. The companion PR (zed-industries#60027) maps Roslyn's C#-specific
token types to theme styles — without it the tokens arrive but most are
dropped, since their types aren't in Zed's default rules.

## Solution

- Advertise `textDocument.semanticTokens.dynamicRegistration = true`.
- Handle the `textDocument/semanticTokens` registration and
unregistration so the capability is stored, following the existing arms
for `documentLink`, diagnostics, etc.

## Testing

- Added a test that dynamically registers and unregisters
`textDocument/semanticTokens` and checks the stored capability appears
and is cleared.
- Verified manually against Roslyn on a C# project: Zed now sends
`textDocument/semanticTokens/full` and gets tokens back; before this
change there was no semantic-token traffic at all.

## Self-Review Checklist:

- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable

---

Release Notes:

- Support dynamic registration of the `textDocument/semanticTokens`
capability.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed The user has signed the Contributor License Agreement first contribution the author's first pull request to Zed. NOTE: the label application is automated via github actions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants