QVAC-14392 chore: scope down DataLoader cleanup to packages/rag#1754
Conversation
…392)
- Drop the `overrides: { @qvac/dl-hyperdrive: ^0.2.0 }` pin in
packages/sdk/package.json. Without the artificial pin, dl-hyperdrive
resolves to the version the addons' peer deps actually request
(0.1.1), and dl-filesystem and stale infer-base versions fall out
of the SDK install entirely.
- Migrate packages/rag examples and integration tests off
@qvac/dl-hyperdrive: replace HyperDriveDL-based loader plumbing
with the new files-based addon constructor shape
({ files, config, logger, opts }). Models are downloaded over
HTTPS via a small downloadModel helper, mirroring the addon
package examples.
- Bump rag's devDependencies on @qvac/embed-llamacpp (^0.7.6 ->
^0.14.0) and @qvac/llm-llamacpp (^0.5.7 -> ^0.16.0) to versions
that ship the files-based API. Add bare-https for the model
download helper.
- Regenerate packages/sdk/NOTICE and packages/rag/NOTICE to reflect
the updated dependency trees.
Addon-side cleanup (deleting dl-* packages, dropping the
@qvac/dl-hyperdrive peer dep from @qvac/infer-base, and migrating
remaining addon test/example deps) is tracked separately and will
land in the addon pod.
…AC-14392)
Replace the hand-rolled HTTPS downloader and bare-https devDependency in
@qvac/rag examples and integration tests with @qvac/registry-client. This
matches the pattern already used by @qvac/ocr-onnx (examples/utils.js +
scripts/download-ocr-models.js) and means the rag examples now fetch models
through the same QVAC registry the SDK consumes at runtime.
- examples/utils.js: rewrite as a thin wrapper around QVACRegistryClient.
Single client instance, opens once, downloads all missing models, closes.
Exposes RAG_MODELS catalog (gte-large_fp16.gguf and Llama-3.2-1B-Instruct-Q4_0.gguf)
with their canonical { path, source } registry tuples lifted from
packages/sdk/models/registry/models.ts.
- examples/quickstart.js, examples/chunking.js: switch to ensureModels()
helper; drop bare-path / explicit URL constants.
- test/integration/rag.test.js: same migration; modelName now sourced from
RAG_MODELS to avoid duplication.
- package.json: add @qvac/registry-client@^0.4.1, remove bare-https.
No runtime / public API surface change.
…ng (QVAC-14392) The override pinning @qvac/dl-hyperdrive ^0.2.0 was load-bearing for typing in the SDK: it transitively bumps @qvac/infer-base to 0.4.x, which exposes the narrower Loader/file types the SDK relies on. Without the override, @qvac/dl-hyperdrive resolves to 0.1.1 (the peer dep declared by infer-base@0.1.x), which broadens the receiver types of several SDK utilities and causes 39 @typescript-eslint/no-unnecessary-type-assertion errors in lint (CI's "check (sdk)" job fails as a result). Restore the override and regenerate bun.lock + NOTICE. The dl-* package deletion will follow once the addons drop their @qvac/dl-hyperdrive peer-dep entirely.
|
Reverted the Without the Pushed Local verification:
|
Tier-based Approval Status |
|
Opened the addon-side prerequisite as #1761 — once |
…r-audio (QVAC-14392) (#1761) Two manifest-only changes that together unblock the SDK-side removal of @qvac/dl-* packages tracked in QVAC-14392. @qvac/infer-base - Drop the vestigial peerDependencies."@qvac/dl-hyperdrive": "^0.1.0" declaration. The runtime moved off DataLoaders in #1688 (Loader interface lives in infer-base, ready()/close() optional), so this peer-dep is no longer required by anything in the package. Lint, test:dts, and brittle-bare unit tests (118/118) all pass with the declaration removed. @qvac/decoder-audio - Bump "@qvac/infer-base" from "^0.1.0" to "^0.4.0". This stops pulling the published infer-base@0.1.1 (which still carries the old dl-hyperdrive peer-dep on its 0.1.x line) into consumers' install trees. Local install resolves @qvac/infer-base@0.4.0, decoder-audio's BaseInference subclassing in index.js continues to work unchanged, lint and brittle-bare unit tests (9/9) pass. Effect on the SDK: once both packages are republished (infer-base @ 0.4.1, decoder-audio @ 0.3.8 -- or whichever versions ship), the "overrides": { "@qvac/dl-hyperdrive": "^0.2.0" } block in packages/sdk/package.json can be removed and @qvac/dl-hyperdrive plus @qvac/dl-base fall out of the SDK's dep tree entirely. That cleanup is tracked in #1754. NOTICE files regenerated for both packages.
|
GPR-publish PRs opened off #1761:
Once both merge into their |
….1 is on npm (QVAC-14392) @qvac/infer-base@0.4.1 (released via tetherto#1777) drops the vestigial peerDependencies."@qvac/dl-hyperdrive": "^0.1.0" declaration that was forcing the override. With it on npm, the SDK no longer needs the override block in package.json — `bun install` resolves cleanly and the type-narrowing regression we saw before tetherto#1761 is gone. Verified locally with override dropped: - bun install: clean, 1315 packages - bunx eslint . --max-warnings=0: clean (was 39 errors pre-tetherto#1761) - bun run typecheck: clean - bun run build: clean - bun run test:unit: 10/10 pass NOTICE regenerated. @qvac/dl-base and @qvac/dl-hyperdrive still appear in the install tree (now at 0.1.1 instead of 0.2.1) because @qvac/decoder-audio@0.3.7 still pins @qvac/infer-base: ^0.1.0 — once @qvac/decoder-audio@0.3.8 (tetherto#1778) clears its integration tests and publishes to npm, those will fall out entirely and a final NOTICE regen will close out QVAC-14392 end-to-end. Refs: tetherto#1761, tetherto#1777, tetherto#1778
|
Pushed the override drop now that Local verification on this commit (override removed, fresh
The lint regression that originally forced the override is gone — Caveat — NOTICE still lists
Status of #1778 npm publish: GPR merged, but the npm publish job is gated behind integration tests (linux/darwin/win32 + iOS/Android mobile e2e) that are sitting in Plan: once Backmerge PRs for the trunk version sync are open at #1781 (infer-base) and #1782 (decoder-audio). |
…o@0.3.8 is on npm (QVAC-14392) @qvac/decoder-audio@0.3.8 (released via tetherto#1778) flips its @qvac/infer-base direct dep from ^0.1.0 to ^0.4.0. Combined with @qvac/infer-base@0.4.1 (no peer-dep on @qvac/dl-hyperdrive), nothing in the SDK install tree peer-deps or directly-deps @qvac/dl-* anymore — they fall out entirely. NOTICE now shows: - @qvac/decoder-audio@0.3.8 (was 0.3.7) - @qvac/infer-base@0.4.1 (was 0.1.1 + 0.4.0 dual) - @qvac/dl-base REMOVED - @qvac/dl-hyperdrive REMOVED QVAC-14392 closeable end-to-end. Refs: tetherto#1761, tetherto#1777, tetherto#1778
|
QVAC-14392 fully cleared — Final NOTICE delta on this commit:
Status now: Backmerge PRs for trunk version sync: #1781 ( |
|
/review |
…r-audio (QVAC-14392) (#1761) Two manifest-only changes that together unblock the SDK-side removal of @qvac/dl-* packages tracked in QVAC-14392. @qvac/infer-base - Drop the vestigial peerDependencies."@qvac/dl-hyperdrive": "^0.1.0" declaration. The runtime moved off DataLoaders in #1688 (Loader interface lives in infer-base, ready()/close() optional), so this peer-dep is no longer required by anything in the package. Lint, test:dts, and brittle-bare unit tests (118/118) all pass with the declaration removed. @qvac/decoder-audio - Bump "@qvac/infer-base" from "^0.1.0" to "^0.4.0". This stops pulling the published infer-base@0.1.1 (which still carries the old dl-hyperdrive peer-dep on its 0.1.x line) into consumers' install trees. Local install resolves @qvac/infer-base@0.4.0, decoder-audio's BaseInference subclassing in index.js continues to work unchanged, lint and brittle-bare unit tests (9/9) pass. Effect on the SDK: once both packages are republished (infer-base @ 0.4.1, decoder-audio @ 0.3.8 -- or whichever versions ship), the "overrides": { "@qvac/dl-hyperdrive": "^0.2.0" } block in packages/sdk/package.json can be removed and @qvac/dl-hyperdrive plus @qvac/dl-base fall out of the SDK's dep tree entirely. That cleanup is tracked in #1754. NOTICE files regenerated for both packages.
…r-audio (QVAC-14392) (#1761) Two manifest-only changes that together unblock the SDK-side removal of @qvac/dl-* packages tracked in QVAC-14392. @qvac/infer-base - Drop the vestigial peerDependencies."@qvac/dl-hyperdrive": "^0.1.0" declaration. The runtime moved off DataLoaders in #1688 (Loader interface lives in infer-base, ready()/close() optional), so this peer-dep is no longer required by anything in the package. Lint, test:dts, and brittle-bare unit tests (118/118) all pass with the declaration removed. @qvac/decoder-audio - Bump "@qvac/infer-base" from "^0.1.0" to "^0.4.0". This stops pulling the published infer-base@0.1.1 (which still carries the old dl-hyperdrive peer-dep on its 0.1.x line) into consumers' install trees. Local install resolves @qvac/infer-base@0.4.0, decoder-audio's BaseInference subclassing in index.js continues to work unchanged, lint and brittle-bare unit tests (9/9) pass. Effect on the SDK: once both packages are republished (infer-base @ 0.4.1, decoder-audio @ 0.3.8 -- or whichever versions ship), the "overrides": { "@qvac/dl-hyperdrive": "^0.2.0" } block in packages/sdk/package.json can be removed and @qvac/dl-hyperdrive plus @qvac/dl-base fall out of the SDK's dep tree entirely. That cleanup is tracked in #1754. NOTICE files regenerated for both packages.
…r-audio (QVAC-14392) (#1761) Two manifest-only changes that together unblock the SDK-side removal of @qvac/dl-* packages tracked in QVAC-14392. @qvac/infer-base - Drop the vestigial peerDependencies."@qvac/dl-hyperdrive": "^0.1.0" declaration. The runtime moved off DataLoaders in #1688 (Loader interface lives in infer-base, ready()/close() optional), so this peer-dep is no longer required by anything in the package. Lint, test:dts, and brittle-bare unit tests (118/118) all pass with the declaration removed. @qvac/decoder-audio - Bump "@qvac/infer-base" from "^0.1.0" to "^0.4.0". This stops pulling the published infer-base@0.1.1 (which still carries the old dl-hyperdrive peer-dep on its 0.1.x line) into consumers' install trees. Local install resolves @qvac/infer-base@0.4.0, decoder-audio's BaseInference subclassing in index.js continues to work unchanged, lint and brittle-bare unit tests (9/9) pass. Effect on the SDK: once both packages are republished (infer-base @ 0.4.1, decoder-audio @ 0.3.8 -- or whichever versions ship), the "overrides": { "@qvac/dl-hyperdrive": "^0.2.0" } block in packages/sdk/package.json can be removed and @qvac/dl-hyperdrive plus @qvac/dl-base fall out of the SDK's dep tree entirely. That cleanup is tracked in #1754. NOTICE files regenerated for both packages.
* chore(sdk,rag): remove DataLoader package usage from SDK pod (QVAC-14392)
- Drop the `overrides: { @qvac/dl-hyperdrive: ^0.2.0 }` pin in
packages/sdk/package.json. Without the artificial pin, dl-hyperdrive
resolves to the version the addons' peer deps actually request
(0.1.1), and dl-filesystem and stale infer-base versions fall out
of the SDK install entirely.
- Migrate packages/rag examples and integration tests off
@qvac/dl-hyperdrive: replace HyperDriveDL-based loader plumbing
with the new files-based addon constructor shape
({ files, config, logger, opts }). Models are downloaded over
HTTPS via a small downloadModel helper, mirroring the addon
package examples.
- Bump rag's devDependencies on @qvac/embed-llamacpp (^0.7.6 ->
^0.14.0) and @qvac/llm-llamacpp (^0.5.7 -> ^0.16.0) to versions
that ship the files-based API. Add bare-https for the model
download helper.
- Regenerate packages/sdk/NOTICE and packages/rag/NOTICE to reflect
the updated dependency trees.
Addon-side cleanup (deleting dl-* packages, dropping the
@qvac/dl-hyperdrive peer dep from @qvac/infer-base, and migrating
remaining addon test/example deps) is tracked separately and will
land in the addon pod.
* chore(rag): use @qvac/registry-client for example model downloads (QVAC-14392)
Replace the hand-rolled HTTPS downloader and bare-https devDependency in
@qvac/rag examples and integration tests with @qvac/registry-client. This
matches the pattern already used by @qvac/ocr-onnx (examples/utils.js +
scripts/download-ocr-models.js) and means the rag examples now fetch models
through the same QVAC registry the SDK consumes at runtime.
- examples/utils.js: rewrite as a thin wrapper around QVACRegistryClient.
Single client instance, opens once, downloads all missing models, closes.
Exposes RAG_MODELS catalog (gte-large_fp16.gguf and Llama-3.2-1B-Instruct-Q4_0.gguf)
with their canonical { path, source } registry tuples lifted from
packages/sdk/models/registry/models.ts.
- examples/quickstart.js, examples/chunking.js: switch to ensureModels()
helper; drop bare-path / explicit URL constants.
- test/integration/rag.test.js: same migration; modelName now sourced from
RAG_MODELS to avoid duplication.
- package.json: add @qvac/registry-client@^0.4.1, remove bare-https.
No runtime / public API surface change.
* chore(sdk): restore @qvac/dl-hyperdrive override for SDK type narrowing (QVAC-14392)
The override pinning @qvac/dl-hyperdrive ^0.2.0 was load-bearing for
typing in the SDK: it transitively bumps @qvac/infer-base to 0.4.x,
which exposes the narrower Loader/file types the SDK relies on.
Without the override, @qvac/dl-hyperdrive resolves to 0.1.1 (the peer
dep declared by infer-base@0.1.x), which broadens the receiver types
of several SDK utilities and causes 39 @typescript-eslint/no-unnecessary-type-assertion
errors in lint (CI's "check (sdk)" job fails as a result).
Restore the override and regenerate bun.lock + NOTICE. The dl-* package
deletion will follow once the addons drop their @qvac/dl-hyperdrive
peer-dep entirely.
* chore(sdk): drop @qvac/dl-hyperdrive override now that infer-base@0.4.1 is on npm (QVAC-14392)
@qvac/infer-base@0.4.1 (released via #1777) drops the vestigial
peerDependencies."@qvac/dl-hyperdrive": "^0.1.0" declaration that was
forcing the override. With it on npm, the SDK no longer needs the
override block in package.json — `bun install` resolves cleanly and
the type-narrowing regression we saw before #1761 is gone.
Verified locally with override dropped:
- bun install: clean, 1315 packages
- bunx eslint . --max-warnings=0: clean (was 39 errors pre-#1761)
- bun run typecheck: clean
- bun run build: clean
- bun run test:unit: 10/10 pass
NOTICE regenerated. @qvac/dl-base and @qvac/dl-hyperdrive still appear
in the install tree (now at 0.1.1 instead of 0.2.1) because
@qvac/decoder-audio@0.3.7 still pins @qvac/infer-base: ^0.1.0 — once
@qvac/decoder-audio@0.3.8 (#1778) clears its integration tests and
publishes to npm, those will fall out entirely and a final NOTICE
regen will close out QVAC-14392 end-to-end.
Refs: #1761, #1777, #1778
* chore(sdk): regenerate NOTICE — drop @qvac/dl-* now that decoder-audio@0.3.8 is on npm (QVAC-14392)
@qvac/decoder-audio@0.3.8 (released via #1778) flips its @qvac/infer-base
direct dep from ^0.1.0 to ^0.4.0. Combined with @qvac/infer-base@0.4.1
(no peer-dep on @qvac/dl-hyperdrive), nothing in the SDK install tree
peer-deps or directly-deps @qvac/dl-* anymore — they fall out entirely.
NOTICE now shows:
- @qvac/decoder-audio@0.3.8 (was 0.3.7)
- @qvac/infer-base@0.4.1 (was 0.1.1 + 0.4.0 dual)
- @qvac/dl-base REMOVED
- @qvac/dl-hyperdrive REMOVED
QVAC-14392 closeable end-to-end.
Refs: #1761, #1777, #1778
Summary
Scopes down the SDK-side DataLoader (
@qvac/dl-*) cleanup tracked in QVAC-14392 to just thepackages/ragmigration off@qvac/dl-hyperdrive. The original attempt to also drop thedl-hyperdriveoverride inpackages/sdk/package.jsonis reverted in this PR — see Why the override stays below.What changed
packages/rag/{examples,test/integration}— migrate off@qvac/dl-hyperdrive. Replace the oldHyperDriveDL+ loader-based addon construction with the new files-based shape ({ files, config, logger, opts }). Model fetching for examples and the integration test now goes through@qvac/registry-client(mirroringpackages/ocr-onnx/examples/utils.jsandscripts/download-ocr-models.js), so the rag examples consume models from the same QVAC registry the SDK uses at runtime instead of hand-rolling HTTPS downloads. The integration test still exercises the same RAG flows; only the model setup has changed.packages/rag/package.json— drop@qvac/dl-hyperdrivefromdevDependencies, add@qvac/registry-client@^0.4.1, bump@qvac/embed-llamacpp ^0.7.6 → ^0.14.0and@qvac/llm-llamacpp ^0.5.7 → ^0.16.0to versions that ship the files-based API.packages/sdk/package.json—overrides: { @qvac/dl-hyperdrive: ^0.2.0 }is kept (initially removed, then restored in commit0fd2e96feafter a reviewer flagged the regression).packages/{sdk,rag}/NOTICE— regenerated.Why the override stays
The override is load-bearing for typing inside the SDK. With it in place,
@qvac/dl-hyperdriveresolves to0.2.1, which transitively pulls@qvac/infer-base@0.4.xinto the dep tree — and that is the version exposing the narrowerLoader/ file types the SDK relies on.Removing the override drops
dl-hyperdriveto0.1.1(the version@qvac/infer-base@0.1.xdeclares as a peer dep). The receiver types of several SDK utilities then become broader, and ESLint flags 39@typescript-eslint/no-unnecessary-type-assertionerrors acrossserver/rpc/profiling/*,server/rpc/rpc-utils.ts,server/utils/formatting.ts,utils/global-singleton.ts, etc. CI'scheck (sdk)job fails on this.The full removal of
@qvac/dl-*from the SDK depends on the addons-side cleanup landing first:@qvac/infer-basestill declarespeerDependencies: { @qvac/dl-hyperdrive: ^0.1.0 }, so the addons (llm,embed,tts-onnx,transcription-*,translation-nmtcpp,diffusion-cpp,ocr-onnx,decoder-audio) still pulldl-hyperdriveanddl-basetransitively.That follow-up belongs to the addon pod and will land in a separate PR.
Test plan
bun installinpackages/sdk—dl-hyperdriveresolves to0.2.1,infer-base@0.4.0present, no peer warnings.bunx eslint . --max-warnings=0inpackages/sdk— clean.bun run typecheck(tsc --noEmit) inpackages/sdk— clean.bun run buildinpackages/sdk— clean.bun run test:unitinpackages/sdk— 10/10 passing.npm install+npm run lint+npm run test:unitinpackages/rag— lint clean, 104/104 passing.node .cursor/skills/notice-generate/scripts/generate-notice.js sdk/… rag— NOTICE files regenerated.tier1+verify).packages/rag/test/integration/rag.test.jsagainst the QVAC registry (integration tests are not in CI for this package).