Fix split GGUF stage inventory discovery - #1199
Conversation
|
This pull request is currently a draft. Reviews will not take place until the PR is marked as ready for review. |
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
* Split Skippy by functional boundary * Rebuild llama patch queue by functional boundary * Add explicit standard library includes to Skippy modules * Add explicit tokenization limit dependency * Address Skippy queue review findings * Advertise Skippy chat feature support * Fix split GGUF stage inventory discovery (#1199) * feat: pass reasoning effort through Skippy templates * feat: advertise reasoning efforts in model listings * docs: require Skippy ABI inventories in PRs * docs: clarify Skippy ABI lockstep policy * refactor: remove legacy Skippy chat ABI * fix: stop advertising reasoning effort values * fix: preserve Skippy no-thinking override * docs: publish Skippy native API reference * fix: address Skippy API doc review comments * Split Skippy by functional boundary * Rebuild llama patch queue by functional boundary * Add explicit standard library includes to Skippy modules * Add explicit tokenization limit dependency * Address Skippy queue review findings * Advertise Skippy chat feature support * Fix split GGUF stage inventory discovery (#1199) * feat: pass reasoning effort through Skippy templates * feat: advertise reasoning efforts in model listings * docs: require Skippy ABI inventories in PRs * docs: clarify Skippy ABI lockstep policy * refactor: remove legacy Skippy chat ABI * fix: stop advertising reasoning effort values * fix: preserve Skippy no-thinking override
* Split Skippy by functional boundary * Rebuild llama patch queue by functional boundary * Add explicit standard library includes to Skippy modules * Add explicit tokenization limit dependency * Address Skippy queue review findings * Advertise Skippy chat feature support * Fix split GGUF stage inventory discovery (#1199)
* Split Skippy by functional boundary * Rebuild llama patch queue by functional boundary * Add explicit standard library includes to Skippy modules * Add explicit tokenization limit dependency * Address Skippy queue review findings * Advertise Skippy chat feature support * Fix split GGUF stage inventory discovery (#1199) * feat: pass reasoning effort through Skippy templates * feat: advertise reasoning efforts in model listings * docs: require Skippy ABI inventories in PRs * docs: clarify Skippy ABI lockstep policy * refactor: remove legacy Skippy chat ABI * fix: stop advertising reasoning effort values * fix: preserve Skippy no-thinking override * docs: publish Skippy native API reference * fix: address Skippy API doc review comments * Split Skippy by functional boundary * Rebuild llama patch queue by functional boundary * Add explicit standard library includes to Skippy modules * Add explicit tokenization limit dependency * Address Skippy queue review findings * Advertise Skippy chat feature support * Fix split GGUF stage inventory discovery (#1199) * feat: pass reasoning effort through Skippy templates * feat: advertise reasoning efforts in model listings * docs: require Skippy ABI inventories in PRs * docs: clarify Skippy ABI lockstep policy * refactor: remove legacy Skippy chat ABI * fix: stop advertising reasoning effort values * fix: preserve Skippy no-thinking override
* feat: pass reasoning effort through Skippy templates * feat: advertise reasoning efforts in model listings * docs: require Skippy ABI inventories in PRs * docs: clarify Skippy ABI lockstep policy * refactor: remove legacy Skippy chat ABI * fix: stop advertising reasoning effort values * fix: preserve Skippy no-thinking override * docs: publish Skippy native API reference (#1217) * Split Skippy by functional boundary * Rebuild llama patch queue by functional boundary * Add explicit standard library includes to Skippy modules * Add explicit tokenization limit dependency * Address Skippy queue review findings * Advertise Skippy chat feature support * Fix split GGUF stage inventory discovery (#1199) * feat: pass reasoning effort through Skippy templates * feat: advertise reasoning efforts in model listings * docs: require Skippy ABI inventories in PRs * docs: clarify Skippy ABI lockstep policy * refactor: remove legacy Skippy chat ABI * fix: stop advertising reasoning effort values * fix: preserve Skippy no-thinking override * docs: publish Skippy native API reference * fix: address Skippy API doc review comments * Split Skippy by functional boundary * Rebuild llama patch queue by functional boundary * Add explicit standard library includes to Skippy modules * Add explicit tokenization limit dependency * Address Skippy queue review findings * Advertise Skippy chat feature support * Fix split GGUF stage inventory discovery (#1199) * feat: pass reasoning effort through Skippy templates * feat: advertise reasoning efforts in model listings * docs: require Skippy ABI inventories in PRs * docs: clarify Skippy ABI lockstep policy * refactor: remove legacy Skippy chat ABI * fix: stop advertising reasoning effort values * fix: preserve Skippy no-thinking override
Summary
Fixes direct split-GGUF stage inventory discovery for models whose first shard is metadata-only, stacked on #1194.
Root cause
Stage inventory discovery inferred the layer count only by scanning tensor names. DeepSeek-V4 split GGUFs can place metadata in the first shard while storing all tensors in later shards, so the tensor scan returned no layers and the peer advertised an empty inventory. Split readiness then waited indefinitely even though the mesh connection itself was healthy.
Fix
Validation
just buildcargo test -p mesh-llm-host-runtime --lib inventory_source_resolves_metadata_only_first_shardcargo fmt --all --checkcargo check -p mesh-llm-host-runtimecargo check -p mesh-llmcargo clippy -p mesh-llm-host-runtime --all-targets -- -D warnings -A unfulfilled-lint-expectationscargo clippy -p mesh-llm --all-targets -- -D warnings -A unfulfilled-lint-expectationsStrict clippy remains blocked by 28 pre-existing
unfulfilled_lint_expectationserrors in the parent PR branch; none are in this diff.