Skip to content

QVAC-18481 test[skiplog]: pre-load multi-file model companion sets on bootstrap#1899

Merged
Victor-Rodzko merged 6 commits into
mainfrom
test/preload-companion-sets
May 7, 2026
Merged

QVAC-18481 test[skiplog]: pre-load multi-file model companion sets on bootstrap#1899
Victor-Rodzko merged 6 commits into
mainfrom
test/preload-companion-sets

Conversation

@Victor-Rodzko

@Victor-Rodzko Victor-Rodzko commented May 5, 2026

Copy link
Copy Markdown
Contributor

Note: be concise and prefer bullet points.

🎯 What problem does this PR solve?

  • tts-chatterbox-short-text flakes on Android smoke (first test cold-loads ~5 ONNX files + ONNX Runtime init, can exceed the 600s test watchdog)
  • bootstrap() only pre-downloaded the main constant for multi-file models (chatterbox, supertonic, parakeet, vision); companion files were lazily fetched during the first test
  • Mobile chatterbox needs referenceAudioSrc resolved from a bundled RN asset URI — can't be expressed as a static config object
  • A few iOS transcribe() call sites slipped past the existing ^transcription- skip regex (config-reload-then-transcribe, error-transcription-failed)

📝 How does it solve it?

  • Switch multi-file model resources to preLoadUnload: true on mobile + desktop. loadModel() triggers downloadCompanionSetFromRegistry() which fetches the entire set; the subsequent unloadModel() keeps the file cache + warm native paths
  • Drop redundant skipPreDownload: true so models download in parallel during the download phase, then pre-load runs sequentially after
  • Add async config resolver support in ResourceManager (cached per-dep). Mobile chatterbox uses it to resolve referenceAudioSrc from expo-asset at bootstrap time
  • Remove obsolete patchChatterboxReferenceAudio() workaround in MobileTtsExecutor — config now carries everything before the first test runs
  • Extract shared resolveBundledAssetUri helper (mobile/asset-uri.ts) — model-asset-executor.ts:resolveAsset() and the chatterbox audio lookup now delegate to it (was a duplicated 12-line block in two places)
  • Extend iOS transcribe() skip list to cover config-reload-then-transcribe and error-transcription-failed

🧪 How was it tested?

  • Local smoke on macOS: 89/89 passed, 4:20 min, all 8 preLoadUnload models pre-cached cleanly
  • Local smoke on iPhone 16e (iOS): 76/89 passed + 13 skipped, 0 failed, 4:58 min, peak memory 2.45 GB; tts-chatterbox-short-text green at 15.8s
  • Verified bootstrap log shows full companion-set fetches: chatterbox pulls all 5 ONNX files at pre-load (vs only tokenizer.json previously)

Switch chatterbox/supertonic/parakeet/vision/diffusion to preLoadUnload
so loadModel() at bootstrap fetches every companion file (encoder,
decoder, vocab, projection, etc.) — otherwise they were lazily fetched
inside the first test, which caused the tts-chatterbox-short-text
Android flake (5 ONNX files + ONNX Runtime cold init blew through the
600s test watchdog).

Add async config resolver to ResourceManager so chatterbox can resolve
its referenceAudioSrc from the bundled RN asset registry at bootstrap
time; cached per-dep. Remove the now-obsolete
patchChatterboxReferenceAudio workaround from MobileTtsExecutor.

Extend the iOS transcribe() skip list to catch the call sites that
slipped past the ^transcription- regex
(config-reload-then-transcribe, error-transcription-failed).

Co-authored-by: Cursor <cursoragent@cursor.com>
@Victor-Rodzko Victor-Rodzko requested a review from a team as a code owner May 5, 2026 11:52
Pull asset resolution + file:// stripping out of the duplicated copies in
mobile/consumer.ts and mobile/executors/model-asset-executor.ts into a
single mobile/asset-uri.ts helper. Both sites now delegate, so future
changes to expo-asset handling live in one place.

Tighter idioms in the helper itself: regex strip instead of substring(7),
?? instead of ||, no mutable let.

Co-authored-by: Cursor <cursoragent@cursor.com>
@Victor-Rodzko Victor-Rodzko changed the title test[notask]: pre-load multi-file model companion sets on bootstrap test[skiplog]: pre-load multi-file model companion sets on bootstrap May 5, 2026
Add `clean:sdk-snapshot` to wipe the cached @qvac/sdk copies left over
in tests-qvac/node_modules and the iOS/Android consumer build dirs by
previous `npm install --install-links` runs. Wire it into
`install:build:full` so a full rebuild always pulls a fresh SDK
snapshot after `prepare:sdk` rebuilds the SDK itself.

Co-authored-by: Cursor <cursoragent@cursor.com>
@github-actions

github-actions Bot commented May 5, 2026

Copy link
Copy Markdown
Contributor

QVAC E2E — ios — ✅ all tests passed (76/89, 890s)

Config: suite=smoke · filter=(none) · exclude=(none)
View run · Artifacts

@github-actions

github-actions Bot commented May 5, 2026

Copy link
Copy Markdown
Contributor

QVAC E2E — windows⚠️ no results

Config: suite=smoke · filter=(none) · exclude=(none)
View run · Artifacts

The test job did not produce a results artifact. Check the run for job-level failures.

@github-actions

github-actions Bot commented May 5, 2026

Copy link
Copy Markdown
Contributor

QVAC E2E — linux — ✅ all tests passed (89/89, 465s)

Config: suite=smoke · filter=(none) · exclude=(none)
View run · Artifacts

@github-actions

github-actions Bot commented May 5, 2026

Copy link
Copy Markdown
Contributor

QVAC E2E — macos — ✅ all tests passed (89/89, 313s)

Config: suite=smoke · filter=(none) · exclude=(none)
View run · Artifacts

@github-actions

github-actions Bot commented May 5, 2026

Copy link
Copy Markdown
Contributor

QVAC E2E — android — ✅ all tests passed (83/89, 2311s)

Config: suite=smoke · filter=(none) · exclude=(none)
View run · Artifacts

Comment thread packages/sdk/tests-qvac/tests/mobile/consumer.ts
@github-actions

github-actions Bot commented May 6, 2026

Copy link
Copy Markdown
Contributor

Tier-based Approval Status

**PR Tier:** TIER1

**Current Status:** ✅ APPROVED

**Requirements:**
- 1 Team Member approval ✅ (1/1)
- 1 Team Lead OR Management approval ✅ (1/1)



---
*This comment is automatically updated when reviews change.*

Mobile MobileTtsExecutor used a string-prefix heuristic that mapped
every `tts-supertonic-*` test to `tts-supertonic`, so the new
`tts-supertonic-multilingual` resource was preloaded but never
exercised by `tts-supertonic-multilingual-text`. Switch to
`test.metadata?.dependency` to match the desktop TtsExecutor.
@Victor-Rodzko

Copy link
Copy Markdown
Contributor Author

/review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test-e2e-smoke Triggers smoke e2e test suite [Currently SDK-only] tier1 verify

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants