QVAC-23752 feat[bc]: remove sliding-context support from the llm-addon

[yingying0906]

🎯 What problem does this PR solve?

• The addon could evict tokens from the middle of the KV cache once the context filled, and tracked firstMsgTokens so the system prompt was never what got dropped. It was opt-in through n_discarded and defaulted to off, so the shipped behaviour was already the non-sliding path while the slide machinery added state every other path had to keep correct.
• Reasoning-block compaction kept shifting the cache even with sliding gone: seq_rm the <think> span, then seq_add the tail down over it. That is context shifting under another name, so it goes too.

📝 How does it solve it?

Sliding removed. ContextSlider and ContextShifter are gone, along with firstMsgTokens / protectedPrefix, slideCapable admission, applySlide, supportsSliding and SequenceStepResult::discarded.

Overflow is now the single path n_discarded=0 already took. A prefill that does not fit throws ContextOverflow. A generation that fills the window stops with stopReason=contextOverflow and still returns what it produced, so a caller can tell a full context from a prediction-limit cutoff. Both prefill guards name the quantity they report.

Compaction replays instead of shifting. Every model rewinds to a boundary anchored before the reasoning span and re-decodes what it keeps, which is what the recurrent path already did. Pure attention does not need that path's full-state snapshot: its cells are positionally indexed, so the boundary is just a position and rewinding is a tail trim. KvCacheOps.hpp/.cpp and compactKvRange are deleted, so no seq_add remains in the addon and the deferred K-shift never runs. Outcome::Kind collapses to one Compacted, and FailedKvIntact goes with both drivers' roll-back-to-pre-request-cursor recovery: compaction rewinds before it replays, so by the time anything can fail only a wipe leaves the cache coherent.

Compaction was pointer math over cells and is now a re-decode of the kept tokens, once per reasoning turn. Measured on Qwen3-0.6B on Metal that is 0.85 ms to replay 30 tokens and 2.2 ms to replay 71, so it scales like prefilling those tokens rather than like generating them.

A multi-token reasoning close marker works now. The policy refused markers that tokenise to several pieces, because replay could only seed the single token that tripped the close detector. Nothing structural is replayed at all now, so marker length decides nothing and the refusal path goes with it.

Pure attention compacts to preamble + answer. The full-state path replays the close marker. Pure attention anchors an absolute position before the span, so a force-open opener is trimmed with everything else and nothing structural is seeded or replayed. Recurrent and hybrid anchor at the end of prefill, which keeps the opener in the restored prefix, so the canonical close is seeded and replayed to balance it. The compacted cache there is preamble + <think></think> + answer, and the reasoning body is gone either way.

Anchoring the full-state boundary before the opener would mean stopping the prefill decode mid-prompt, and that is not free: on Vulkan with coopmat2 the same tokens fed as one decode and as two land on different SSM state, and Qwen3.5 then takes another path from the first generated token. cpu is invariant to the split, and so is Vulkan with GGML_VK_DISABLE_COOPMAT2=1. So no decode is split anywhere: the chunk cap never fires, the batch prefill takes no pause, and multimodal feeds its chunks whole.

Compaction replay runs outside the scheduler mutex. finalizeTerminalDriver reaches compactThinkSpan, so on a reasoning turn the drain now runs a real llama_decode. stepLocked already drops the lock for the main decode and for media eval; drainFinishedLocked did not, so a replay stalled every co-tenant slot and blocked a cross-thread cancel() for its whole length. This window holds a reference into slots_ across the unlock, so the usual reconcile on every reacquisition is wrong here: a cancel recorded during it still passes slotOwnedByLocked and would run onCancel on a driver mid-finalize. TeardownDeferGuard suspends application for the window without dropping anything, and the worker applies the records at its next loop top where the ownership re-check discards the stale ones. The direct teardown paths defer for the window too. cancel, cancelGroupQueued and clear kept a branch that freed the slot outright once the scheduler was stopping, and submitLocked could re-admit a seqId whose SlotState the drain had not freed yet, since the batcher releases its own slot first. Both destroyed a driver mid-finalize. Deferring while stopping is safe because the destructor joins and then clears every slot.

generatedTokens no longer comes from llama's perf counters. llama_perf_context keys on batch size, not meaning: n_queued_tokens == 1 bumps n_eval, larger bumps n_p_eval. Generation decodes singly, so the two agreed by coincidence until replay started decoding in batches. The stat is counted in the generation loops now, where a token is committed. On the single-prompt path that restores the old numbers rather than redefining them. The batch path used to count whatever the sampler returned, which included an EOG that was never decoded, so it now counts at the commit site too and reports one less. A prediction-limit sample is still counted, because the caller received it and the single-prompt loop decodes and counts it as well. See Breaking Changes.

TTFT does not follow that rule. The caller sees the terminal token, since onLogitsReady streams it before the slot is marked finished, so a predict: 1 batch request was returning output while reporting TTFT 0. The first stamp is taken before the terminal filter now, and the rate window still closes on a counted token, so a request whose only token also ended it reports a TTFT with no rate.

Batch path. advance marked a slot finished the moment currentPos hit maxTokensPerSequence, so it was filtered out before the driver's overflow check and ordinary generation reported sequenceLimit for a full window. That limit IS the slot's share of the context, and submit rejects any request that would not fit, so a full window is the one thing it can mean. It now says so.

Shared helpers. contextWindowFull(pos, ceiling) replaces four hand-rolled context-full checks that had drifted between >=, +1 >, raw llama_n_ctx and ctxCeiling(), and the MTMD prefill guards move onto exceedsContextWindow beside the text ones. That last one is a small behaviour change: those guards compared >= llama_n_ctx for every request, so a prefill-only prompt that exactly filled the window was refused. It is accepted now, which is the rule the text path already had, because a request that will not generate needs no free cell. SessionMetadata moves into LlmContext.hpp so the {nPast, nPast, cacheTokens, cacheTokens} layout has one home instead of three.

Session metadata keeps its four-slot width so files written by either build still load. Slots 1 and 3 are retired and this build's readers ignore them, but they are not written as 0: an older build reads slot 1 as its protected prefix and evicts from there, so a 0 would point that at position 0 and silently drop the system prompt and tool definitions. They mirror the live cursors instead, which drives that build's slide guard negative so it refuses the slide and reports a context overflow with the cache intact. That refusal covers its prefill slide only, since the generation slide never carried the guard, so mirroring is the better of the two values to write rather than a guarantee at every slide site.

The SDK half is in #3999, and the two can land in either order: sdk and inference pin @qvac/llm-llamacpp@^0.47.0 from npm, and #3999 still parses the published wordings. A fabric bug found on the way is fixed in tetherto/qvac-fabric-llm.cpp#213; this PR does not depend on it, since it removes the addon's only route into that code.

🧪 How was it tested?

• C++ unit: 842 ran, 840 pass, 2 skipped for missing model fixtures. New coverage pins the compaction contract: a failed restore must not go on to replay, a failed replay wipes, and a span with no boundary is a clean no-op rather than a wipe. Also pinned: an unfinished span does not replay its opener, an EOG sample is not counted while a prediction-limit sample is, and deferred cancels survive the finalize unlock window. Each of those was confirmed to fail with its fix reverted, since two tests this PR inherited passed only because they never set up the state production has. Separately, the compacted cache shape is pinned on forced-open pure attention, generated-opener pure attention and the recurrent path, asserting the exact replayed sequence rather than its length.- Desktop integration on Metal: reasoning 16/16 with real compactions through the replay path, api-behavior 14/14, cache-state-machine 21/21, generation-params 2/2.
• JS unit: 108/108, including the generationParams key guard, every documented key still admitted, and a key reachable only through the prototype chain never reaching the addon.
• api-behavior.test.js covers both prefill guards, including a cached follow-up that only the second guard can reject, and pins each guard's wording since the SDK parser matches them separately. test/integration/_context-overflow.js owns the tokenizer-sensitive sizing both files share.
• sliding-context.test.js and mrope-sliding-context.test.js deleted, with all three mobile registries updated so a run does not abort on an unregistered test.
• clang-format and clang-tidy clean on every changed addon/src translation unit.
• CI on d3071e090, cpp and desktop: https://github.com/tetherto/qvac/actions/runs/33367447657
• Mobile, dispatched on demand against the branch build from GitHub Packages: Android 14/14 devices and 42/42 tests, https://github.com/tetherto/qvac/actions/runs/33352083170, iOS 24/24 devices and 72/72 tests, https://github.com/tetherto/qvac/actions/runs/33352089835

💥 Breaking Changes

n_discarded is no longer consumed, so it reaches llama's own argument parser and fails model load as an unknown option.

BEFORE:

await model.load({ ctx_size: 2048, n_discarded: 256 })

AFTER:

await model.load({ ctx_size: 2048 })

A batched sequence that fills its window reports the full context instead of the per-sequence cap.

BEFORE:

// parallel: 4, predict: -1, slot window fills
stats.stopReason // 'sequenceLimit', same value a predict cutoff gives

AFTER:

stats.stopReason // 'contextOverflow'

A batched request that stops on EOG reports one less generated token. That sample is never decoded, so it never reached the cache and is no longer counted. This matches what the single-prompt path has always reported. TPS shifts with it.

BEFORE:

// parallel: 4, sequence stops on EOG after 64 committed tokens
stats.generatedTokens // 65, counting the EOG that was never decoded

AFTER:

stats.generatedTokens // 64

A batched request that stops on the prediction limit reports one more. That sample is ordinary content the caller already received, and the single-prompt loop decodes and counts it, so the two paths now agree at both boundaries instead of only at EOG. predict: 1 used to report 0 next to non-empty output.

BEFORE:

// parallel: 4, predict: 64
stats.generatedTokens // 63

AFTER:

stats.generatedTokens // 64

generationParams with a key the addon does not read now throws instead of ignoring it. The binding pulls each param by name, so a near miss like n_predict for predict used to run with the load-time default and report nothing. Only own keys count, and only own keys are forwarded: the binding's named get walks the prototype chain, so a value reachable only through the prototype used to reach sampler config without ever being checked.

BEFORE:

await model.run(prompt, { generationParams: { n_predict: 24 } })
// ran with the load-time predict value

AFTER:

// TypeError: generationParams has unknown key: n_predict.
// Valid keys are temp, top_p, top_k, predict, seed, frequency_penalty,
// presence_penalty, repeat_penalty, grammar, json_schema, reasoning_budget,
// remove_thinking_from_context

🔌 API Changes

contextSlides is gone from the runtime stats snapshot and from RuntimeStats in the type declarations.

const stats = inference.runtimeStats()
// stats.contextSlides no longer exists

---

• To see the specific tasks where the Asana app for GitHub is being used, see below:
  • https://app.asana.com/0/0/1217557716472343

[github-actions]

Review Status

Current Status: ❌ PENDING
Approvals so far: none

Pending reviews: Needs 1 Management or Team Lead, and 1 more from Management, Team Lead, or Member.

[github-actions]

License compliance — clean

No new dependency license findings in this PR.

Warn-only (shadow) mode — this check does not block merges yet.

Updated automatically by the canonical license compliance workflow.

NOTICE presence (advisory)

Missing NOTICE (advisory, does not block):

• ./.github/actions/release-merge-guard
• ./docs/website
• ./packages/ggml-coload-smoke
• ./packages/fabric/test/integration
• ./packages/inference-addon-cpp/mobile
• ./packages/sdk/e2e
• ./packages/llm-llamacpp/benchmarks/performance
• ./packages/llm-llamacpp/benchmarks/server
• ./packages/vla-ggml/sim/server
• ./packages/embed-llamacpp/benchmarks/performance
• ./packages/embed-llamacpp/benchmarks/server
• ./packages/asr-ggml/benchmarks/server

[yingying0906]

/review