zed: Add automatic language detection for pasted code

[florian-trehaut]

Self-Review Checklist:

• I've reviewed my own diff for quality, security, and reliability
• Unsafe blocks (if any) have justifying comments
• The content is consistent with the UI/UX checklist
• Tests cover the new/changed behavior
• Performance impact has been considered and is acceptable

Closes #4868

Transparency

First OSS contribution in Rust. First time working on a desktop app codebase (my background is data/backend). The code was generated with Claude Code under close supervision. Flagging so reviewers pay extra attention to Rust idioms, GPUI patterns, and desktop conventions.

Summary

Adds automatic language detection for code pasted into Plain Text buffers or received via stdin. Parses pasted content against each native tree-sitter grammar and picks the best-scoring language based on the ratio of valid AST nodes to total nodes.

New language_detection module with init() called at startup. Observes EditorEvent::Edited on Plain Text buffers with 500ms debounce. Grammar list is built dynamically from LanguageRegistry via available_language_for_modeline_name() — no hardcoded language mapping. Hidden languages and a blacklist of meta-grammars that cause false positives are filtered out. Scoring runs on a background thread via cx.background_executor().spawn().

Shows an auto-dismissing toast with an Undo action on detection. New auto_detect_language editor setting (default: true), independent of disable_ai. Also handles stdin content detection on editor creation.

Adds grammars and tree-sitter as direct dependencies to the zed crate (both already in the workspace, no new external dependencies).

Architecture

LanguageDetector is a GPUI Global that tracks user overrides and which buffer is currently being updated by detection. LanguageDetectionAddon is a per-editor Addon holding DebouncedDelay state. build_detection_grammars(registry) queries the LanguageRegistry to dynamically map grammar names to LanguageName, filtering hidden languages and a META_GRAMMAR_BLACKLIST. detect_language(content, grammars) scores each grammar using a single reused Parser and returns the best match above threshold. count_error_nodes() does an iterative tree traversal with TreeCursor instead of recursion.

Follows the edit_prediction_registry pattern (cx.observe_new) and conflict_view pattern (Editor Addon).

Grammar selection

12 of 20 native grammars participate in scoring. 8 are excluded:

• Hidden languages (filtered dynamically via AvailableLanguage::hidden()): jsdoc, regex, markdown-inline
• Meta-grammar blacklist — these parse almost any input as valid, identified by manual testing: diff, markdown, gomod, gowork, gitcommit

Why tree-sitter scoring

PR #43057 attempted this feature with Magika (Google's ML content type detection). It was closed due to latency and architecture concerns. Tree-sitter scoring was suggested by @SomeoneToIgnore on that PR:

"we can run all Zed's languages' tree-sitter queries on the document range and score the matches within the same timeframe"

No new external dependencies (tree-sitter and grammars already in workspace), no binary size increase (no ONNX Runtime), and it uses the same grammars Zed already uses for syntax highlighting.

Performance

Measured in two ways: unit tests (isolated scoring) and runtime logs (temporary log::info! with Instant::now() deltas around the full pipeline).

Unit tests (debug build, cargo test): Rust ~1KB in ~24ms, Python ~950B in ~24ms, both for 12 grammars.

Runtime logs (debug build): build_detection_grammars() takes ~70-84µs, detect_language() scoring takes ~15-17ms for 12 grammars. Total end-to-end ~17ms.

All measurements on debug build only. I did not measure release builds. The methodology is basic — reviewers should flag if something more rigorous is needed.

Design decisions open for discussion

1. Scoring algorithm: (descendant_count - error_count) / descendant_count. Simple ratio.

2. Minimum score threshold (0.5): In my tests, correct languages score >0.9 and incorrect ones <0.3. The 0.5 threshold is arbitrary but sits between the two clusters.

3. Meta-grammar blacklist: 5 non-hidden grammars excluded statically, identified by manual testing — they parse almost any input as valid.

4. Minimum content threshold (16 bytes): Arbitrary floor before attempting detection.

5. Debounce duration (500ms): Tested manually. No data on what the right value is.

6. JS/TS ambiguity: JavaScript uses the tsx grammar in Zed (crates/grammars/src/javascript/config.toml), so there is no separate javascript native grammar. JS code will be detected as TSX.

7. Buffer text cloning: The full buffer text is cloned after debounce fires (inside fire_new closure), not on every keystroke. For very large buffers this could be expensive — I don't know if partial content would give reliable results with tree-sitter scoring.

Test Plan

• Unit: Rust detected as Rust, Python detected as Python
• Unit: Plain English text not detected as a structured language
• Unit: Meta-grammars excluded from scoring set
• Unit: Guard logic (plain text, setting enabled/disabled, user override)
• Unit: count_error_nodes iterative correctness (valid and invalid inputs)
• Performance: 12 grammars on ~1KB < 100ms (measured ~24ms debug)
• Performance: runtime grammar building < 1ms (measured ~80µs)
• Performance: runtime scoring < 100ms (measured ~16ms debug)
• Manual: paste Rust, Python, Go — correct detection with toast
• Manual: Undo toast reverts to Plain Text
• Manual: language selector override blocks re-detection
• Manual: short content (< 16 bytes) does not trigger detection
• Manual: stdin pipe detected correctly
• Manual: LSP starts after auto-detection
• Manual: auto_detect_language: false disables detection
• clippy, fmt, cargo test pass

Release Notes:

• Added automatic language detection for pasted code using tree-sitter grammar scoring

[SomeoneToIgnore]

Thank you, I do not think 500ms "debounce" makes any sense given Zed's alignment to be fast.

We should dig down to that and find ways to speed things up before doing anything else, as the last comment in the PR you've linked to explicitly states: #43057 (comment)