Cover asymmetric KV cache width pricing - #594
Merged
Merged
Conversation
ndizazzo
requested changes
May 19, 2026
IvGolovach
force-pushed
the
codex/kv-context-slot-auto-planning
branch
from
May 19, 2026 20:53
4cda985 to
b02caf0
Compare
IvGolovach
force-pushed
the
codex/kv-context-slot-auto-planning
branch
from
May 19, 2026 23:48
b02caf0 to
4fadfeb
Compare
Collaborator
|
@IvGolovach thanks for pushing a fix! |
ndizazzo
approved these changes
May 20, 2026
Collaborator
|
@IvGolovach had to make some mega changes to this based on multi node testing I found - might want to update this to main, and check how it performs (I did consult this one but had to simplify it), am still a little confused on how to avoid things overcommitting and falling over, but glad this is looked at |
Validation * Validation tier: Tier 2R - post-review conflict repair narrowed to a model-artifact KV pricing regression after current main absorbed the runtime planner changes and kept the conservative 4-slot auto ceiling. * git fetch --no-tags origin main:refs/remotes/origin/main codex/kv-context-slot-auto-planning:refs/remotes/origin/codex/kv-context-slot-auto-planning: PASS * git rebase origin/main: PASS, kept current main runtime planning behavior and dropped stale RuntimePlanningProfile/PublicMesh lane escalation. * git diff --check origin/main...HEAD: PASS, no output * git diff --check: PASS, no output * git diff --cached --check: PASS, no output * cargo fmt --all -- --check: PASS * cargo test -p model-artifact gguf --lib: PASS, 16 passed * LLAMA_STAGE_BUILD_DIR=/Users/Funtland/Downloads/mesh-llm/.deps/llama-build/build-stage-abi-metal cargo test -p mesh-llm-host-runtime runtime::context_planning --lib: PASS, 9 passed * cargo check -p model-artifact: PASS * LLAMA_STAGE_BUILD_DIR=/Users/Funtland/Downloads/mesh-llm/.deps/llama-build/build-stage-abi-metal cargo check -p mesh-llm: PASS * /opt/homebrew/bin/cargo-clippy clippy -p model-artifact --all-targets -- -D warnings: PASS * Ledger: not applicable - not required for selected validation tier/change family. * Version: not applicable - test-only regression coverage; no release/version sync required. * Not run: live performance smoke - not required after the final diff no longer changes runtime planning behavior; current main's runtime planner remains covered by targeted context_planning tests and remote CI. Rollback * git revert HEAD
IvGolovach
force-pushed
the
codex/kv-context-slot-auto-planning
branch
from
May 21, 2026 07:02
4fadfeb to
a884363
Compare
michaelneale
added a commit
that referenced
this pull request
May 21, 2026
…able-thinking * origin/main: Cover asymmetric KV cache width pricing (#594)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This is now a narrow regression-test follow-up after current
mainabsorbed and simplified the runtime auto-planning work from the original branch.mainruntime planning behavior, including the conservative unified-KV auto slot ceiling from the recent multi-node fixes.RuntimePlanningProfile/ public-mesh lane escalation from this branch instead of fighting the newer runtime work.Why
The maintainer feedback was to update against current
mainand re-check the shape after the multi-node runtime changes. After rebasing, the right professional scope was not to reintroduce the older runtime planning profile. The remaining useful piece is the model-artifact regression coverage for asymmetric K/V metadata, which protects the capacity math used by the planner without changing runtime behavior.Diff scope
crates/model-artifact/src/gguf.rsCompatibility
Validation
Local validation:
git fetch --no-tags origin main:refs/remotes/origin/main codex/kv-context-slot-auto-planning:refs/remotes/origin/codex/kv-context-slot-auto-planning- PASSgit rebase origin/main- PASSgit diff --check origin/main...HEAD- PASS, no outputgit diff --check- PASS, no outputgit diff --cached --check- PASS, no outputcargo fmt --all -- --check- PASScargo test -p model-artifact gguf --lib- PASS, 16 passedLLAMA_STAGE_BUILD_DIR=/Users/Funtland/Downloads/mesh-llm/.deps/llama-build/build-stage-abi-metal cargo test -p mesh-llm-host-runtime runtime::context_planning --lib- PASS, 9 passedcargo check -p model-artifact- PASSLLAMA_STAGE_BUILD_DIR=/Users/Funtland/Downloads/mesh-llm/.deps/llama-build/build-stage-abi-metal cargo check -p mesh-llm- PASS/opt/homebrew/bin/cargo-clippy clippy -p model-artifact --all-targets -- -D warnings- PASSRemote validation:
Not run:
mainruntime planning remains covered by targetedruntime::context_planningtests and remote CI.Rollback
Revert this PR.
Related: #359