fix(local-inference): graceful non-speculative fallback for a missing MTP drafter GGUF — #11517 back-compat, never brick an installed model

[NubsCarson]

What

The Gemma-4 MTP cutover (#11517, extended to 4b by #11782) added eliza-1-2b/eliza-1-4b to ELIZA_1_HOSTED_MTP_TIER_IDS, so the shared catalog now declares separate-drafter runtime.mtp for those tiers. resolveLocalInferenceLoadArgs then hard-threw for any installed bundle missing mtp/drafter-<tier>.gguf — which is every bundle downloaded before the cutover — so a previously-working installed model could no longer load at all. Two packages/ui active-model.test.ts cases were red on develop catching exactly this regression.

Why graceful fallback (not a throw)

The drafter is a perf-only speculative-decoding artifact (measured ~1.3-1.6x greedy speedup, #11517 evidence). A missing drafter degrades speed, not correctness — bricking the user's installed, previously-working model over it is the wrong UX. The resolver now:

• loads the text model without MTP (no draft args, no mobileSpeculative), and
• emits a console.warn telling the user to re-download the model to enable the MTP drafter (a fresh download bundles it, since the catalog advertises components.mtp for hosted tiers).

The fallback also closes a pre-existing hole: a no-bundleRoot install (external-scan / single-file GGUF) of a hosted-MTP tier previously leaked a half-configured speculative state (draftMin/draftMax/mobileSpeculative set with NO draftModelPath) into the loader. Now the whole MTP block is skipped whenever the drafter isn't on disk. (The capacitor-bridge bootstrap already behaved this way — plugins/plugin-capacitor-bridge/src/mobile-device-bridge-bootstrap.ts only passes MTP args when the drafter resolves — so this aligns the desktop/UI resolvers with the mobile path.)

Applied to both copies of the resolver: packages/ui/src/services/local-inference/active-model.ts (client mirror) and plugins/plugin-local-inference/src/services/active-model.ts (runtime).

Tests

• packages/ui active-model.test.ts — the two stale pre-cutover tests are replaced with the post-cutover contract: hosted tiers with the drafter bundled enable MTP (path + catalog draft window), unhosted tiers ignore a stray on-disk drafter, and a pre-cutover bundle (no drafter) falls back with the warning asserted. bun run --cwd packages/ui test active-model → 17/17 (was 14 passed / 2 failed).
• plugins/plugin-local-inference mmproj-routing.test.ts — new pre-cutover fallback case → 10/10.
• plugins/plugin-local-inference load-args-drafter.fuzz.test.ts — the throw assertion becomes the fallback assertion; the external-scan case now expects a fully-unset MTP block → 22/22.
• Full related suites: ui src/services/local-inference 84/84, plugin active-model* + text-bundle + ensure-assigned-model-loaded all green.
• typecheck green in both packages; biome clean on all touched files.

Evidence

• Repro on develop tip (c5e1ca53e5): bun run --cwd packages/ui test active-model → 2 failed (AssertionError: expected '/tmp/.../mtp/drafter-2b.gguf' to be undefined and Error: [local-inference] eliza-1-2b declares a separate-drafter MTP but no bundled drafter GGUF was found under /tmp/...).
• After: 17/17. The back-compat scenario (bundle with text GGUF, no mtp/ dir — byte-identical to a pre-cutover install layout) is exercised by real resolver calls against the real shared catalog in three suites, including the warn text.
• Screenshots/video: N/A — no rendered UI change; this is a loader-args resolution fix with deterministic file-system-level tests driving the real resolver + real catalog.
• Live-model trajectory: N/A — no prompt/model behavior change; the fix only decides whether speculative-decoding load args are passed. (On a pre-cutover install, develop currently cannot load the model at all, which is the bug.)

Fixes the 2 red active-model.test.ts cases on develop; back-compat regression from #11517.

Signed: nubs-cloud [cloud-frontdoor]

🤖 Generated with Claude Code

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: c500a3ce-ca05-4cf9-9700-bd3ce23aa3eb

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/11517-mtp-backcompat

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[greptile-apps]

Your trial has ended. Reactivate Greptile to resume code reviews.

[greptile-apps]

Your trial has ended. Reactivate Greptile to resume code reviews.

[claude]

Claude encountered an error —— View job

---

I'll analyze this and get back to you.