CI: Disable test-runner for Svelte 4 sandboxes

[JReinhold]

What I did

Test runner in CI has been failing for a couple of days now with a flaky, hard to track error.

This solution just disables the test runner for the two offending Svelte sandboxes, and leaves it enabled for the 3 others. What's noticeable about the two failing is that they are using Svelte 4 + SvelteKit.

The timing of the flaky failures aligns with the release of SvelteKit 2.17.0 so that could be the reason, but I haven't confirmed this.

FYI @benmccann, not something you need to worry about, but might worth knowing for you if you get other user reports about similar issues. The failures indicate that for some reason the browser sometimes navigates when/after rendering a story, which causes the Playwright tests to fail. But only for SvelteKit+Svelte 4.

Checklist for Contributors

Testing

The changes in this PR are covered in the following automated tests:

• stories
• unit tests
• integration tests
• end-to-end tests

Manual testing

This section is mandatory for all contributions. If you believe no manual test is necessary, please state so explicitly. Thanks!

Documentation

• Add or update documentation reflecting your changes
• If you are deprecating/removing a feature, make sure to update
  MIGRATION.MD

Checklist for Maintainers

• When this PR is ready for testing, make sure to add ci:normal, ci:merged or ci:daily GH label to it to run a specific set of sandboxes. The particular set of sandboxes can be found in code/lib/cli-storybook/src/sandbox-templates.ts

• Make sure this PR contains one of the labels below:

  Available labels

  • bug: Internal changes that fixes incorrect behavior.
  • maintenance: User-facing maintenance tasks.
  • dependencies: Upgrading (sometimes downgrading) dependencies.
  • build: Internal-facing build tooling & test updates. Will not show up in release changelog.
  • cleanup: Minor cleanup style change. Will not show up in release changelog.
  • documentation: Documentation only changes. Will not show up in release changelog.
  • feature request: Introducing a new feature.
  • BREAKING CHANGE: Changes that break compatibility in some way with current major version.
  • other: Changes that don't fit in the above categories.

🦋 Canary release

This PR does not have a canary release associated. You can request a canary release of this pull request by mentioning the @storybookjs/core team here.

core team members can create a canary release here or locally with gh workflow run --repo storybookjs/storybook canary-release-pr.yml --field pr=<PR_NUMBER>

name	before	after	diff	z	%
createSize	0 B	0 B	0 B	-	-
generateSize	78.4 MB	80.4 MB	2.1 MB	13.73	2.6%
initSize	78.4 MB	80.4 MB	2.1 MB	-1.9	2.6%
diffSize	97 B	97 B	0 B	-2	0%
buildSize	7.22 MB	7.25 MB	29.6 kB	2.87	0.4%
buildSbAddonsSize	1.87 MB	1.87 MB	-1.17 kB	1.43	-0.1%
buildSbCommonSize	195 kB	195 kB	0 B	-	0%
buildSbManagerSize	1.88 MB	1.88 MB	0 B	1.53	0%
buildSbPreviewSize	0 B	0 B	0 B	-	-
buildStaticSize	0 B	0 B	0 B	-	-
buildPrebuildSize	3.95 MB	3.94 MB	-1.17 kB	1.46	0%
buildPreviewSize	3.27 MB	3.3 MB	30.8 kB	9.41	0.9%
testBuildSize	0 B	0 B	0 B	-	-
testBuildSbAddonsSize	0 B	0 B	0 B	-	-
testBuildSbCommonSize	0 B	0 B	0 B	-	-
testBuildSbManagerSize	0 B	0 B	0 B	-	-
testBuildSbPreviewSize	0 B	0 B	0 B	-	-
testBuildStaticSize	0 B	0 B	0 B	-	-
testBuildPrebuildSize	0 B	0 B	0 B	-	-
testBuildPreviewSize	0 B	0 B	0 B	-	-

name	before	after	diff	z	%
createTime	14.5s	14.8s	326ms	-0.17	2.2%
generateTime	20s	22s	1.9s	1.3	🔺8.9%
initTime	4.7s	5s	327ms	-1.68	🔺6.5%
buildTime	8.5s	9.7s	1.1s	0.5	12.2%
testBuildTime	0ms	0ms	0ms	-	-
devPreviewResponsive	4.8s	4.9s	112ms	-0.54	2.2%
devManagerResponsive	3.6s	3.6s	-5ms	-0.83	-0.1%
devManagerHeaderVisible	705ms	716ms	11ms	-0.58	1.5%
devManagerIndexVisible	713ms	797ms	84ms	-0.28	10.5%
devStoryVisibleUncached	3.7s	3.9s	116ms	-0.36	3%
devStoryVisible	734ms	798ms	64ms	-0.32	8%
devAutodocsVisible	784ms	691ms	-93ms	-0.61	-13.5%
devMDXVisible	676ms	694ms	18ms	-0.34	2.6%
buildManagerHeaderVisible	660ms	978ms	318ms	1.48	🔺32.5%
buildManagerIndexVisible	795ms	1s	210ms	0.97	20.9%
buildStoryVisible	646ms	843ms	197ms	0.94	23.4%
buildAutodocsVisible	512ms	666ms	154ms	-0.03	23.1%
buildMDXVisible	589ms	655ms	66ms	0.24	10.1%

Greptile Summary

Based on the provided files and context, I'll create a concise summary of the changes in this PR:

Disabled test runner for specific Svelte 4 sandboxes and adjusted CI parallelism to address flaky test failures.

• Disabled test runner for svelte-kit/skeleton-js and svelte-kit/skeleton-ts sandboxes in code/lib/cli-storybook/src/sandbox-templates.ts
• Reduced test-runner-production parallelism from 9 to 8 in .circleci/config.yml normal workflow
• Reduced parallelism in merged workflow (15 to 14) and daily workflow (32 to 30)
• Issue appears related to browser navigation problems specific to SvelteKit+Svelte 4 combinations

[nx-cloud]

View your CI Pipeline Execution ↗ for commit 53c996a.

Command	Status	Duration	Result
nx affected -t check -c production --parallel=7	✅ Succeeded	6s	View ↗
nx run-many -t build -c production --parallel=3	✅ Succeeded	12s	View ↗

---

☁️ Nx Cloud last updated this comment at 2025-02-06 08:51:00 UTC

[greptile-apps]

LGTM

_{2 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings | Greptile}

[JReinhold]

Self-merging

[benmccann]

The timing of the flaky failures aligns with the release of SvelteKit 2.17.0 so that could be the reason, but I haven't confirmed this.

FYI @benmccann, not something you need to worry about, but might worth knowing for you if you get other user reports about similar issues. The failures indicate that for some reason the browser sometimes navigates when/after rendering a story, which causes the Playwright tests to fail. But only for SvelteKit+Svelte 4.

FYI @dummdidumm @eltigerchino in case you think any of the recent changes could cause this test failure