[Memory Leak]: Builds incorporating Shiki Twoslash crash with JavaScript heap out of memory errors

[IMax153]

Describe the bug

Hello 👋 from the Effect-TS community :)

We have recently been experimenting with using different frameworks for building our documentation website. I decided to try building our documentation with Vitepress because of your native integration with Shiki Twoslash (via @shikijs/vitepress-twoslash), which we make extensive use of in our docs.

However, when I attempt to build our documentation with Vitepress and Shiki Twoslash, I consistently receive a JavaScript heap out of memory error. I have attempted to increase the --max-old-space-size to ~8GB and still encounter this issue.

With Twoslash Integration

In my .vitepress/config.ts, if I remove the transformerTwoslash from the markdown.codeTransformers array, the build succeeds without issue in ~20 seconds.

Without Twoslash Integration

Also, please let me know if it is more appropriate to move this issue to the Shiki project. If so, I have no problem closing this issue and moving it 👍 Thank you for your time!

Reproduction

1. Clone our docs experiment

   git clone https://github.com/IMax153/vitepress-memory-leak-repro.git

2. Install dependencies

   pnpm install

3. Attempt to build

   pnpm build

Expected behavior

My expected behavior would be for Vitepress to be able to build our docs with Shiki Twoslash integrated without issue.

System Info

System:
    OS: macOS 14.6.1
    CPU: (10) arm64 Apple M1 Pro
    Memory: 1.99 GB / 16.00 GB
    Shell: 5.9 - /nix/store/w53pgqwh5dm38skdd7n7wn0k6cw72jzj-zsh-5.9/bin/zsh
Binaries:
    Node: 20.17.0 - /nix/store/hxl1k8qgmrm1vfq5f419iv4wybz9szqq-nodejs-20.17.0/bin/node
    npm: 10.8.2 - /nix/store/hxl1k8qgmrm1vfq5f419iv4wybz9szqq-nodejs-20.17.0/bin/npm
    pnpm: 9.10.0 - /nix/store/ddjsxwcpw09llisgvw7drvr92dbkg4mi-corepack-nodejs-20.17.0/bin/pnpm
Browsers:
    Chrome: 129.0.6668.70
    Safari: 17.6
npmPackages:
    vitepress: ^1.3.4 => 1.3.4

Note: I use Nix for provisioning my system, hence the strange looking binary paths.

Additional context

I've included a script in the package.json that reduces the available NodeJS heap memory (to force an OOM earlier) and takes a heap snapshot right before an OOM error occurs in case that is useful for you folks.

pnpm build:snapshot

Validations

• Check if you're on the latest VitePress version.
• Follow our Code of Conduct
• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.

[brc-dd]

Yeah https://github.com/shikijs/shiki/issues might be better place to post this. I'm fine with keeping this open if it's indeed a memory leak with vitepress.

[IMax153]

https://github.com/shikijs/shiki/issues might be better place to post this

No problem, makes sense to me - I've gone ahead and opened an issue upstream. Linking here for tracking purposes: shikijs/shiki#796

[jasonkuhrt]

I started experiencing these issues today with Graffle's website as well. https://github.com/jasonkuhrt/graffle/actions/runs/11129348920/job/30926231362?pr=1151. Nothing more to add than what @IMax153 has already said.

[brc-dd]

@jasonkuhrt If possible, do you have the exact commit which started giving these? If it's recent then it might have something to do with newer shiki releases. 👀

[jasonkuhrt]

Hey @brc-dd, it started failing on this commit ade5a3e61ad536484968880d9218bdcdf4a66ac1. It's a huge commit unfortunately. You can see under the website directory no version of any dependencies changed in it fwiw.