editor: Add language detection for untitled buffers by amtoaer · Pull Request #61201 · zed-industries/zed

amtoaer · 2026-07-17T14:20:25Z

Objective

Closes #4868.

Untitled buffers start as Plain Text and require users to select a language manually before receiving syntax highlighting. Add lightweight automatic language detection for code entered or pasted into untitled buffers.

Solution

This builds on Max Stevens's earlier language-detection work, replacing Magika with Betlang.

While researching smaller and faster alternatives to Magika, I came across Betlang, a recently introduced language detection library developed by DioxusLabs for dioxus-code. The fact that it comes from DioxusLabs gave me more confidence in evaluating this relatively new dependency for Zed. Betlang embeds an approximately 50 KB model, is MIT-licensed, and depends only on fearless_simd, making it well suited to Zed's cross-platform embedding requirements.

Detection runs on the background executor with bounded input sampling. It is restricted to untitled buffers, skips content shorter than 20 bytes, and requires at least 50% confidence. These limits have worked well in local testing. In release builds on Linux with an Intel Core i5-13600KF, the betlang::detect call alone typically completes within 3 ms when processing the maximum sampled input.

Testing

I added a test for language detection in untitled buffers that covers both manually entered and pasted content. The test passes successfully. I also manually tested the feature to verify that the overall experience works well.

Self-Review Checklist:

I've reviewed my own diff for quality, security, and reliability
Unsafe blocks (if any) have justifying comments
The content adheres to Zed's UI standards (UX/UI and icon guidelines)
Tests cover the new/changed behavior
Performance impact has been considered and is acceptable

Showcase

2026-07-17_21-06-03.mp4

Release Notes:

N/A

ChristopherBiscardi

The dependency here is new, but maintained by dioxus and the transitive dependency is maintained by linebender which have good reputations. So this doesn't feel like taking on any additional risk.

The functionality works, thanks!

@amtoaer

…stries#61201)" (zed-industries#61351) This reverts commit 690c7fa. # Objective language detection is firing very often in channel notes, often picking YAML instead of markdown, causing some language-detection flickering. cc/ @amtoaer (author of the original PR)

…1201) # Objective Closes zed-industries#4868. Untitled buffers start as Plain Text and require users to select a language manually before receiving syntax highlighting. Add lightweight automatic language detection for code entered or pasted into untitled buffers. ## Solution This builds on [Max Stevens's earlier language-detection work](zed-industries#43057), replacing Magika with [Betlang](https://github.com/DioxusLabs/betlang). While researching smaller and faster alternatives to Magika, I came across Betlang, a recently introduced language detection library developed by DioxusLabs for dioxus-code. The fact that it comes from DioxusLabs gave me more confidence in evaluating this relatively new dependency for Zed. Betlang embeds an approximately 50 KB model, is MIT-licensed, and depends only on `fearless_simd`, making it well suited to Zed's cross-platform embedding requirements. Detection runs on the background executor with bounded input sampling. It is restricted to untitled buffers, skips content shorter than 20 bytes, and requires at least 50% confidence. These limits have worked well in local testing. In release builds on Linux with an Intel Core i5-13600KF, the `betlang::detect` call alone typically completes within 3 ms when processing the maximum sampled input. ## Testing I added a test for language detection in untitled buffers that covers both manually entered and pasted content. The test passes successfully. I also manually tested the feature to verify that the overall experience works well. ## Self-Review Checklist: - [x] I've reviewed my own diff for quality, security, and reliability - [x] Unsafe blocks (if any) have justifying comments - [x] The content adheres to Zed's UI standards ([UX/UI](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist) and [icon](https://github.com/zed-industries/zed/blob/main/crates/icons/README.md) guidelines) - [x] Tests cover the new/changed behavior - [x] Performance impact has been considered and is acceptable ## Showcase https://github.com/user-attachments/assets/96e28ad7-2968-4325-9aff-37fe813a2da7 --- Release Notes: - Added automatic language detection for untitled buffers. --------- Co-authored-by: Max Stevens <maxstevens2708@gmail.com>

@amtoaer

…stries#61201)" (zed-industries#61351) This reverts commit 55b1b59. # Objective language detection is firing very often in channel notes, often picking YAML instead of markdown, causing some language-detection flickering. cc/ @amtoaer (author of the original PR)

…1412) # Objective Closes zed-industries#4868. Reintroduce automatic language detection for user-created untitled buffers without allowing it to override language choices made by users or owning surfaces. More details: + zed-industries#61201 + zed-industries#61351 ## Solution VS Code distinguishes between inferred and explicitly selected languages, and only continues automatic detection in the former state. This change introduces a buffer-local opt-in flag with the same purpose. It defaults to disabled and is enabled only for user-created untitled buffers and restored untitled buffers that did not persist a language. Choosing a language through the language picker disables detection, while automatically detected languages remain eligible for later redetection. Buffers whose language is assigned by their owning surface, such as Channel Notes, never opt in. Rather than adding persisted state or changing the database schema, restoration reuses the existing optional serialized language. An opted-in buffer that remains Plain Text is serialized without a language and resumes detection after restoration. Once automatic detection has selected a language, that language is serialized normally and restoration leaves detection disabled, effectively treating it as an explicit selection. This also matches VS Code's effective default after restoration. Candidate selection adopts VS Code's confidence-group filtering. Results below the minimum confidence are discarded, and candidate groups are accepted only after a clear confidence gap. The detector then selects the first available Zed language without eagerly loading every candidate. I then added a separate switching rule: Zed keeps the current language unless another candidate has a clear confidence advantage, reducing language changes as model scores fluctuate during editing. Model inference runs off-thread on a bounded sample, and stale results are discarded. I also evaluated penalizing YAML and using stricter global switching thresholds, but neither reliably resolves the ambiguity between Markdown and YAML. In the reproduction from zed-industries#61351, Betlang assigns YAML more than 80% confidence during editing and peaks around 90%. This is reasonable from the content alone: `#` denotes a YAML comment, while `-` denotes a sequence item. Adding ordinary prose or a fenced code block shifts the model strongly toward Markdown. A fixed penalty would overfit this example and risk suppressing legitimate YAML detection. Since known-language buffers do not opt in and manual language selection takes precedence, I consider this level of ambiguity acceptable and prefer not to introduce language-specific score tuning. Model inference itself is fast. As noted in zed-industries#61201: > In release builds on Linux with an Intel Core i5-13600KF, the `betlang::detect` call alone typically completes within 3 ms when processing the maximum sampled input. The main concern is the work surrounding inference. Without debouncing, rapid typing creates a snapshot and launches a background detection task on every keystroke, while the detected language may change before the user has finished typing. This change therefore adds a 200 ms debounce delay, allowing detection to run and update the language only after the user pauses. ## Testing Automated coverage verifies that short inputs remain Plain Text, sufficient Rust content is detected as Rust, replacing it with Go content triggers redetection, and manually selecting Plain Text prevents subsequent automatic changes. Restoration coverage verifies that a stored language keeps detection disabled, while an empty language-less untitled buffer resumes detection. Beyond the automated tests, I manually tested automatic language detection while editing untitled buffers. ## Self-Review Checklist: - [x] I've reviewed my own diff for quality, security, and reliability - [x] Unsafe blocks (if any) have justifying comments - [x] The content adheres to Zed's UI standards ([UX/UI](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist) and [icon](https://github.com/zed-industries/zed/blob/main/crates/icons/README.md) guidelines) - [x] Tests cover the new/changed behavior - [x] Performance impact has been considered and is acceptable ## Showcase https://github.com/user-attachments/assets/103fc731-e12e-48d6-ae89-03978f1c9d74 --- Release Notes: - Added automatic language detection for untitled buffers. --------- Co-authored-by: Max Stevens <maxstevens2708@gmail.com> Co-authored-by: Christopher Biscardi <chris@christopherbiscardi.com>

…1201) # Objective Closes zed-industries#4868. Untitled buffers start as Plain Text and require users to select a language manually before receiving syntax highlighting. Add lightweight automatic language detection for code entered or pasted into untitled buffers. ## Solution This builds on [Max Stevens's earlier language-detection work](zed-industries#43057), replacing Magika with [Betlang](https://github.com/DioxusLabs/betlang). While researching smaller and faster alternatives to Magika, I came across Betlang, a recently introduced language detection library developed by DioxusLabs for dioxus-code. The fact that it comes from DioxusLabs gave me more confidence in evaluating this relatively new dependency for Zed. Betlang embeds an approximately 50 KB model, is MIT-licensed, and depends only on `fearless_simd`, making it well suited to Zed's cross-platform embedding requirements. Detection runs on the background executor with bounded input sampling. It is restricted to untitled buffers, skips content shorter than 20 bytes, and requires at least 50% confidence. These limits have worked well in local testing. In release builds on Linux with an Intel Core i5-13600KF, the `betlang::detect` call alone typically completes within 3 ms when processing the maximum sampled input. ## Testing I added a test for language detection in untitled buffers that covers both manually entered and pasted content. The test passes successfully. I also manually tested the feature to verify that the overall experience works well. ## Self-Review Checklist: - [x] I've reviewed my own diff for quality, security, and reliability - [x] Unsafe blocks (if any) have justifying comments - [x] The content adheres to Zed's UI standards ([UX/UI](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist) and [icon](https://github.com/zed-industries/zed/blob/main/crates/icons/README.md) guidelines) - [x] Tests cover the new/changed behavior - [x] Performance impact has been considered and is acceptable ## Showcase https://github.com/user-attachments/assets/96e28ad7-2968-4325-9aff-37fe813a2da7 --- Release Notes: - Added automatic language detection for untitled buffers. --------- Co-authored-by: Max Stevens <maxstevens2708@gmail.com>

@amtoaer

…stries#61201)" (zed-industries#61351) This reverts commit 690c7fa. # Objective language detection is firing very often in channel notes, often picking YAML instead of markdown, causing some language-detection flickering. cc/ @amtoaer (author of the original PR)

…1412) # Objective Closes zed-industries#4868. Reintroduce automatic language detection for user-created untitled buffers without allowing it to override language choices made by users or owning surfaces. More details: + zed-industries#61201 + zed-industries#61351 ## Solution VS Code distinguishes between inferred and explicitly selected languages, and only continues automatic detection in the former state. This change introduces a buffer-local opt-in flag with the same purpose. It defaults to disabled and is enabled only for user-created untitled buffers and restored untitled buffers that did not persist a language. Choosing a language through the language picker disables detection, while automatically detected languages remain eligible for later redetection. Buffers whose language is assigned by their owning surface, such as Channel Notes, never opt in. Rather than adding persisted state or changing the database schema, restoration reuses the existing optional serialized language. An opted-in buffer that remains Plain Text is serialized without a language and resumes detection after restoration. Once automatic detection has selected a language, that language is serialized normally and restoration leaves detection disabled, effectively treating it as an explicit selection. This also matches VS Code's effective default after restoration. Candidate selection adopts VS Code's confidence-group filtering. Results below the minimum confidence are discarded, and candidate groups are accepted only after a clear confidence gap. The detector then selects the first available Zed language without eagerly loading every candidate. I then added a separate switching rule: Zed keeps the current language unless another candidate has a clear confidence advantage, reducing language changes as model scores fluctuate during editing. Model inference runs off-thread on a bounded sample, and stale results are discarded. I also evaluated penalizing YAML and using stricter global switching thresholds, but neither reliably resolves the ambiguity between Markdown and YAML. In the reproduction from zed-industries#61351, Betlang assigns YAML more than 80% confidence during editing and peaks around 90%. This is reasonable from the content alone: `#` denotes a YAML comment, while `-` denotes a sequence item. Adding ordinary prose or a fenced code block shifts the model strongly toward Markdown. A fixed penalty would overfit this example and risk suppressing legitimate YAML detection. Since known-language buffers do not opt in and manual language selection takes precedence, I consider this level of ambiguity acceptable and prefer not to introduce language-specific score tuning. Model inference itself is fast. As noted in zed-industries#61201: > In release builds on Linux with an Intel Core i5-13600KF, the `betlang::detect` call alone typically completes within 3 ms when processing the maximum sampled input. The main concern is the work surrounding inference. Without debouncing, rapid typing creates a snapshot and launches a background detection task on every keystroke, while the detected language may change before the user has finished typing. This change therefore adds a 200 ms debounce delay, allowing detection to run and update the language only after the user pauses. ## Testing Automated coverage verifies that short inputs remain Plain Text, sufficient Rust content is detected as Rust, replacing it with Go content triggers redetection, and manually selecting Plain Text prevents subsequent automatic changes. Restoration coverage verifies that a stored language keeps detection disabled, while an empty language-less untitled buffer resumes detection. Beyond the automated tests, I manually tested automatic language detection while editing untitled buffers. ## Self-Review Checklist: - [x] I've reviewed my own diff for quality, security, and reliability - [x] Unsafe blocks (if any) have justifying comments - [x] The content adheres to Zed's UI standards ([UX/UI](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist) and [icon](https://github.com/zed-industries/zed/blob/main/crates/icons/README.md) guidelines) - [x] Tests cover the new/changed behavior - [x] Performance impact has been considered and is acceptable ## Showcase https://github.com/user-attachments/assets/103fc731-e12e-48d6-ae89-03978f1c9d74 --- Release Notes: - Added automatic language detection for untitled buffers. --------- Co-authored-by: Max Stevens <maxstevens2708@gmail.com> Co-authored-by: Christopher Biscardi <chris@christopherbiscardi.com>

maxstevens-nl and others added 12 commits July 17, 2026 13:03

first pass

92199ca

make language detector global

9568db3

Only run when language is PLAIN_TEXT

5e6d20f

remove prints

2c8e66e

remove global

fce977e

remove unused imports

716945d

small cleanup

2f5968f

fix clippy

1036cfb

fix language mappings

a96c9b7

Make license-GPL a symlink

68f6867

remove unused crates

7742ef7

editor: Add language detection for untitled buffers

c8b4165

cla-bot Bot added the cla-signed The user has signed the Contributor License Agreement label Jul 17, 2026

zed-community-bot Bot added community champion Issues filed by our amazing community champions! 🫶 guild Pull requests by someone in Zed Guild. NOTE: the label application is automated via github actions labels Jul 17, 2026

ChristopherBiscardi approved these changes Jul 20, 2026

View reviewed changes

ChristopherBiscardi added this pull request to the merge queue Jul 20, 2026

Merged via the queue into zed-industries:main with commit 690c7fa Jul 20, 2026
58 checks passed

ChristopherBiscardi self-assigned this Jul 20, 2026

amtoaer deleted the language-detection-betlang branch July 20, 2026 05:52

amtoaer mentioned this pull request Jul 21, 2026

editor: Add language detection for untitled buffers #61412

Merged

5 tasks

Navigation Menu

Uh oh!

Uh oh!

editor: Add language detection for untitled buffers - #61201

editor: Add language detection for untitled buffers#61201
ChristopherBiscardi merged 12 commits into
zed-industries:mainfrom
amtoaer:language-detection-betlang

amtoaer commented Jul 17, 2026 •

edited by MrSubidubi

Loading

Uh oh!

ChristopherBiscardi left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Uh oh!

Conversation

amtoaer commented Jul 17, 2026 • edited by MrSubidubi Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Objective

Solution

Testing

Self-Review Checklist:

Showcase

Uh oh!

ChristopherBiscardi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

amtoaer commented Jul 17, 2026 •

edited by MrSubidubi

Loading