Skip to content

feat(platform): add shiki skipLangs option for analog v3 - #2239

Merged
brandonroberts merged 1 commit into
analogjs:alphafrom
benpsnyder:fix/2029-mermaid-shiki-oom
Apr 13, 2026
Merged

feat(platform): add shiki skipLangs option for analog v3#2239
brandonroberts merged 1 commit into
analogjs:alphafrom
benpsnyder:fix/2029-mermaid-shiki-oom

Conversation

@benpsnyder

Copy link
Copy Markdown
Contributor

PR Checklist

Closes #2029

Affected scope

  • Primary scope: platform
  • Secondary scopes:

Recommended merge strategy for maintainer [optional]

  • Squash merge
  • Rebase merge
  • Other

Commit preservation note [optional]

What is the new behavior?

This adds shikiOptions.highlighter.skipLangs to the platform content Shiki integration.

When a fenced code block language is listed in skipLangs, the block bypasses Shiki and is rendered as a plain fenced code block instead of going through Shiki's highlighter pipeline.

That gives users a supported way to exclude languages like mermaid from Shiki while still keeping Shiki enabled for the rest of the markdown pipeline. This follows the maintainer direction in the issue thread that skipLangs looked like the right option.

Test plan

  • nx format:check
  • pnpm build
  • pnpm test
  • Manual verification

Commands run:

  • pnpm nx test platform --runTestsByPath packages/platform/src/lib/content/shiki/index.spec.ts
  • pnpm nx build platform

Does this PR introduce a breaking change?

  • Yes
  • No

Other information

The behavior is intentionally narrow:

  • skipped languages are rendered as plain fenced code blocks
  • existing Mermaid handling stays unchanged when mermaid is not listed in skipLangs

@netlify

netlify Bot commented Apr 5, 2026

Copy link
Copy Markdown

Deploy Preview for analog-blog ready!

Name Link
🔨 Latest commit 1e6ea9c
🔍 Latest deploy log https://app.netlify.com/projects/analog-blog/deploys/69dc7f0bb9842a0008e48dae
😎 Deploy Preview https://deploy-preview-2239--analog-blog.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@netlify

netlify Bot commented Apr 5, 2026

Copy link
Copy Markdown

Deploy Preview for analog-app ready!

Name Link
🔨 Latest commit 1e6ea9c
🔍 Latest deploy log https://app.netlify.com/projects/analog-app/deploys/69dc7f0b50c8dd0008b70796
😎 Deploy Preview https://deploy-preview-2239--analog-app.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@coderabbitai

coderabbitai Bot commented Apr 5, 2026

Copy link
Copy Markdown

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds a highlighter.skipLangs?: BundledLanguage[] option and plumbing to exclude specific languages from Shiki. getShikiHighlighter normalizes additionalLangs/skipLangs, ensures highlighter.langs is initialized, merges additionalLangs, removes skipLangs entries, computes mermaid support, and passes skipLangs into ShikiHighlighter. ShikiHighlighter gains a skipLangs field and escapes + returns a plain <pre><code> block when a fenced code block’s language is in skipLangs (mermaid handling remains). Adds Vitest tests and docs demonstrating the new option.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Description check ✅ Passed The PR description is well-structured and directly addresses the feature addition, including context, test results, and behavior clarification.
Linked Issues check ✅ Passed The PR implements the core skipLangs feature addressing issue #2029, including option addition, test coverage, and documentation updates.
Out of Scope Changes check ✅ Passed All changes are directly aligned with the skipLangs feature scope: new option type, implementation in shiki-highlighter, integration in index.ts, tests, and documentation.
Title check ✅ Passed The title 'feat(platform): add shiki skipLangs option for analog v3' follows Conventional Commit style with type(scope): summary format, uses a supported package scope (platform), and clearly describes the main change.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions github-actions Bot added the scope:platform Changes in @analogjs/platform label Apr 5, 2026
@benpsnyder
benpsnyder marked this pull request as ready for review April 5, 2026 08:18
@benpsnyder

Copy link
Copy Markdown
Contributor Author

I checked this against analogjs/analog#2029, the issue discussion, and the current implementation in this branch. skipLangs is the right direction, but a few things still look incomplete before this fully resolves the issue:

  1. The current implementation still pushes additionalLangs into Shiki's langs, and the new test explicitly expects mermaid to remain there. If the goal is to avoid the Mermaid grammar being loaded by Shiki to prevent the build-time OOM reported in analogjs/analog#2029, this likely still needs adjustment.

  2. The skipped-language output shape does not match Analog's existing Mermaid path. Right now skipped Mermaid returns <pre class="language-mermaid"><code class="language-mermaid">..., while the current Mermaid handling expects <pre class="mermaid">.... That means skipLangs: ['mermaid'] would bypass Shiki, but it also appears to bypass the client-side Mermaid rendering path.

  3. The issue explicitly called out a documentation gap as well as the missing API. The docs still show additionalLangs: ['mermaid'] in the Shiki example and do not mention skipLangs or the constrained-memory Mermaid guidance, so the documented user path is still the one that can trigger the original problem.

I did verify that the targeted tests and build pass locally:

  • pnpm nx test platform --runTestsByPath packages/platform/src/lib/content/shiki/index.spec.ts
  • pnpm nx build platform

So this looks close, but I don't think it fully closes analogjs/analog#2029 yet without the behavior/docs follow-up above.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/platform/src/lib/content/shiki/shiki-highlighter.ts`:
- Around line 57-59: The skip-branch in the highlight callback returns raw code
into HTML when this.skipLangs includes(lang), which allows unescaped HTML to be
rendered; update the highlight callback in shiki-highlighter (the place using
this.skipLangs and returning `<pre
class="language-${lang}">...${code}...</pre>`) to HTML-escape the code before
interpolation (e.g., replace & < > " ' with entities or use an existing escape
utility) so skipped languages render as safe text; ensure the escaped string is
used only for the skip branch while keeping normal codeToHtml handling for other
languages.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 61aefa5a-b3af-4e79-a4e4-3821b6217c6a

📥 Commits

Reviewing files that changed from the base of the PR and between cf86db8 and 1ae8554.

📒 Files selected for processing (4)
  • packages/platform/src/lib/content/shiki/index.spec.ts
  • packages/platform/src/lib/content/shiki/index.ts
  • packages/platform/src/lib/content/shiki/options.ts
  • packages/platform/src/lib/content/shiki/shiki-highlighter.ts

Comment thread packages/platform/src/lib/content/shiki/shiki-highlighter.ts Outdated
@github-actions github-actions Bot added the scope:docs Documentation changes label Apr 5, 2026
@benpsnyder

Copy link
Copy Markdown
Contributor Author

Follow-up pushed on fix/2029-mermaid-shiki-oom.

Addressed the skipped-language HTML injection path in packages/platform/src/lib/content/shiki/shiki-highlighter.ts by escaping the fallback <pre><code> output before interpolation, while leaving normal codeToHtml() handling unchanged.

Also added a regression test in packages/platform/src/lib/content/shiki/index.spec.ts that verifies skipped code containing HTML characters is rendered as escaped text.

Verification re-run:

  • pnpm nx test platform --runTestsByPath packages/platform/src/lib/content/shiki/index.spec.ts

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
packages/platform/src/lib/content/shiki/shiki-highlighter.ts (1)

66-68: Pre-existing: Mermaid path uses unescaped code.

The Mermaid rendering path at line 67 interpolates raw code without escaping. This is existing behavior (not introduced by this PR) and intentionally preserved per the PR objectives. However, for completeness, Mermaid diagram syntax can include user-defined labels that might contain HTML entities.

Consider addressing this in a follow-up if Mermaid content could be user-supplied:

         if (this.hasLoadMermaid && lang === 'mermaid') {
-          return `<pre class="mermaid">${code}</pre>`;
+          return `<pre class="mermaid">${escapeHtml(code)}</pre>`;
         }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/platform/src/lib/content/shiki/shiki-highlighter.ts` around lines 66
- 68, The Mermaid branch in the ShikiHighlighter unconditionally interpolates
raw code (the check this.hasLoadMermaid and branch returning `<pre
class="mermaid">${code}</pre>`), which can inject HTML entities; modify the
branch to HTML-escape or sanitize the `code` value before interpolation (e.g.,
call an existing escapeHtml / sanitizeMarkdown function or add a small utility
that replaces &, <, >, " and '), and use the escaped/sanitized string in the
returned `<pre class="mermaid">...</pre>` so user-supplied Mermaid labels cannot
introduce HTML markup.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@packages/platform/src/lib/content/shiki/shiki-highlighter.ts`:
- Around line 66-68: The Mermaid branch in the ShikiHighlighter unconditionally
interpolates raw code (the check this.hasLoadMermaid and branch returning `<pre
class="mermaid">${code}</pre>`), which can inject HTML entities; modify the
branch to HTML-escape or sanitize the `code` value before interpolation (e.g.,
call an existing escapeHtml / sanitizeMarkdown function or add a small utility
that replaces &, <, >, " and '), and use the escaped/sanitized string in the
returned `<pre class="mermaid">...</pre>` so user-supplied Mermaid labels cannot
introduce HTML markup.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 915e1907-d172-407d-815c-b70224c1497f

📥 Commits

Reviewing files that changed from the base of the PR and between 1ae8554 and 908367b.

📒 Files selected for processing (4)
  • apps/docs-app/docs/features/routing/content.md
  • packages/platform/src/lib/content/shiki/index.spec.ts
  • packages/platform/src/lib/content/shiki/index.ts
  • packages/platform/src/lib/content/shiki/shiki-highlighter.ts
🚧 Files skipped from review as they are similar to previous changes (2)
  • packages/platform/src/lib/content/shiki/index.spec.ts
  • packages/platform/src/lib/content/shiki/index.ts

@brandonroberts

Copy link
Copy Markdown
Member

This needs to be validated against the reproduction that it actually fixes the OOM issue.

@brandonroberts brandonroberts added the blocked Blocked label Apr 13, 2026
@brandonroberts

Copy link
Copy Markdown
Member

Beta branch

@benpsnyder

benpsnyder commented Apr 13, 2026

Copy link
Copy Markdown
Contributor Author

[On alpha] Validated against the reproduction at dmorfav/analyzing_oom (linked from analogjs/analog#2029 (comment)).

Test harness

Exercised @analogjs/platform's real getShikiHighlighter() + getHighlightExtension() through a marked instance against two module trees, same config in both runs (additionalLangs: ['mermaid'], plus skipLangs: ['mermaid'] in the fixed run):

  • Baseline: npm pack @analogjs/platform@3.0.0-alpha.29 — pristine published alpha, no fix.
  • Fixed: packages/platform/dist/ built from fix/2029-mermaid-shiki-oom, swapped into the reproduction repo's node_modules/@analogjs/platform.

Results

Assertion Baseline Fixed
shiki.getLoadedLanguages().includes('mermaid') — OOM root cause true false
Loaded-lang count 27 (includes mermaid + mmd) 25 (neither)
Mermaid fence renders as <pre class="mermaid">…</pre> yes yes — byte-identical (verified by diff)
ts fence receives full shiki highlighting yes yes

Baseline loaded langs included mermaid, mmd. Fixed run stripped both — createHighlighter(...) never loads the mermaid TextMate grammar (or its embedded grammars) into the oniguruma registry, eliminating the OOM path described in the issue.

What this proves

  1. Root cause confirmed — the published alpha loads mermaid's TextMate grammar into shiki, matching the issue's diagnosis.
  2. Fix worksskipLangs: ['mermaid'] filters mermaid out of langs before createHighlighter(...), so shiki never loads it.
  3. No regression — mermaid fences still emit <pre class="mermaid">…</pre> (downstream mermaid rendering preserved) and non-skipped languages still get full shiki highlighting. Output diff for the mermaid block is byte-identical between baseline and fixed.

Full Vite build on the reproduction completes successfully with the fix applied.

@benpsnyder
benpsnyder force-pushed the fix/2029-mermaid-shiki-oom branch from 3e984df to badafac Compare April 13, 2026 04:11
@benpsnyder
benpsnyder changed the base branch from alpha to beta April 13, 2026 04:11
@github-actions github-actions Bot added the scope:ci GitHub workflow changes label Apr 13, 2026
@netlify

netlify Bot commented Apr 13, 2026

Copy link
Copy Markdown

Deploy Preview for analog-docs ready!

Name Link
🔨 Latest commit f68a87b
🔍 Latest deploy log https://app.netlify.com/projects/analog-docs/deploys/69dc7cb6d8895b0008dbe86c
😎 Deploy Preview https://deploy-preview-2239--analog-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@github-actions github-actions Bot removed the scope:ci GitHub workflow changes label Apr 13, 2026
@benpsnyder
benpsnyder force-pushed the fix/2029-mermaid-shiki-oom branch from 6114608 to 8e9de4f Compare April 13, 2026 04:35
@github-actions github-actions Bot added the scope:content Changes in @analogjs/content label Apr 13, 2026
@benpsnyder
benpsnyder force-pushed the fix/2029-mermaid-shiki-oom branch from 8e9de4f to 5d9ad4d Compare April 13, 2026 05:01
@github-actions github-actions Bot added the scope:router Changes in @analogjs/router label Apr 13, 2026
@benpsnyder
benpsnyder force-pushed the fix/2029-mermaid-shiki-oom branch 2 times, most recently from b72e152 to f68a87b Compare April 13, 2026 05:18
@benpsnyder
benpsnyder changed the base branch from beta to alpha April 13, 2026 05:23
@benpsnyder benpsnyder changed the title feat(platform): add shiki skipLangs option feat(platform): add shiki skipLangs option for analog v3 Apr 13, 2026
@github-actions github-actions Bot added scope:astro-angular Changes in @analogjs/astro-angular scope:ci GitHub workflow changes labels Apr 13, 2026
@github-actions github-actions Bot added scope:create-analog Changes in create-analog scope:nx-plugin Changes in @analogjs/nx-plugin scope:repo Repository metadata and tooling scope:storybook-angular Changes in @analogjs/storybook-angular scope:vite-plugin-angular Changes in @analogjs/vite-plugin-angular scope:vite-plugin-nitro Changes in @analogjs/vite-plugin-nitro scope:vitest-angular Changes in @analogjs/vitest-angular labels Apr 13, 2026
@benpsnyder
benpsnyder force-pushed the fix/2029-mermaid-shiki-oom branch from f68a87b to d32b012 Compare April 13, 2026 05:25
@benpsnyder
benpsnyder force-pushed the fix/2029-mermaid-shiki-oom branch from d32b012 to 1e6ea9c Compare April 13, 2026 05:28
@github-actions github-actions Bot removed scope:astro-angular Changes in @analogjs/astro-angular scope:content Changes in @analogjs/content scope:create-analog Changes in create-analog scope:nx-plugin Changes in @analogjs/nx-plugin scope:router Changes in @analogjs/router scope:storybook-angular Changes in @analogjs/storybook-angular scope:vite-plugin-angular Changes in @analogjs/vite-plugin-angular scope:vite-plugin-nitro Changes in @analogjs/vite-plugin-nitro scope:vitest-angular Changes in @analogjs/vitest-angular scope:ci GitHub workflow changes scope:repo Repository metadata and tooling labels Apr 13, 2026
@benpsnyder

Copy link
Copy Markdown
Contributor Author

@brandonroberts this got very "un-fun" for beta branch so I am reverting to target this for alpha. I opened a new PR that can be worked on for targeting beta here #2282

@brandonroberts brandonroberts left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@brandonroberts
brandonroberts merged commit ec6bc0f into analogjs:alpha Apr 13, 2026
22 checks passed
@brandonroberts
brandonroberts deleted the fix/2029-mermaid-shiki-oom branch April 13, 2026 14:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

blocked Blocked scope:docs Documentation changes scope:platform Changes in @analogjs/platform

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ability to exclude Mermaid blocks from Shiki to avoid OOM in CI environments

2 participants