Improve GLM MTP parity and batched verify sampling

[i386]

Summary

This PR carries the GLM-4.7 native MTP path for Skippy split serving and restores the winning two-stage MTP condition after the review cleanup regressed it.

Goal: improve Skippy two-stage GLM MTP n=1 decode tok/s toward vanilla llama.cpp zero-MTP. Decode tok/s is the hard optimization target.

Current Carried Condition

Skippy 2-stage split22
native GLM MTP n=1
batched VerifySpan([current, draft])
direct prediction return when available
fallback-lane reply when direct return is unavailable
SKIPPY_NATIVE_MTP_DEFER_REJECT_TRIM=1
SKIPPY_NATIVE_MTP_REJECT_COOLDOWN_TOKENS=6
SKIPPY_NATIVE_MTP_SUPPRESS_COOLDOWN_DRAFT_LIMIT=2
SKIPPY_NATIVE_MTP_GREEDY_SAMPLING_FASTPATH=1

What Broke After Review

The stale post-review row showed Skippy 2-stage MTP at 29.73 tok/s, far below the previous ~40 tok/s result. The restore run found two transport/state regressions:

• PredictedToken direct-return and fail-open fallback writers dropped the predicted_tokens sideband, so native MTP drafts disappeared (drafted=0).
• Stage0 waited only on the direct-return channel for verifier replies. When direct-return failed on the lab and stage1 fell back to the upstream lane, stage0 could deadlock or corrupt the lane.
• The carried condition still set SKIPPY_NATIVE_MTP_DEFER_REJECT_TRIM=1, but the review cleanup had removed that behavior; reject trims then failed on stage1 with cannot trim session beyond current token count.

This branch now preserves sideband tokens, accepts direct-return or fallback-lane replies, and restores the defer-reject-trim behavior for the carried winning condition.

Fresh Benchmark Table

SPEED-Bench slice: qualitative, categories coding,reasoning, limit 4 per category (8 requests total), OSL=512, temperature=0.

Condition	Runtime shape	Decode tok/s	vs vanilla no-MTP	Avg latency	MTP acceptance
vanilla no MTP	llama.cpp full model	48.79	1.00x	11.240s	n/a
vanilla MTP n=1	llama.cpp full model	63.09	1.29x	9.062s	89.8%
2-stage Skippy no MTP	split22: studio54 layers 0..22 -> micstudio layers 22..48	32.72	0.67x	18.208s	n/a
2-stage Skippy MTP n=1 batched	split22 carried condition above	41.07	0.84x	14.936s	79.2% aggregate / 88.5% batched

Important comparison:

• Skippy split22 MTP vs Skippy split22 zero-MTP: 41.07 / 32.72 = 1.26x.
• Skippy split22 zero-MTP vs vanilla zero-MTP: 32.72 / 48.79 = 0.67x.
• Skippy split22 MTP is back in the previous ~40 tok/s band.

Result Artifacts

Lab note:

/Users/jdumay/code/lab-experiments/jianyang/phase-6/iteration-151.md

Final restored result directory:

/Users/jdumay/code/lab-experiments/jianyang/benchmarks/speed-bench/results/skippy-lab-pr858-restore-carried-20260618T024538Z/

Archaeology result directories are also committed in lab-experiments for the failed intermediate restore attempts:

skippy-lab-pr858-restore-carried-20260618T021111Z  # sideband lost, drafted=0
skippy-lab-pr858-restore-carried-20260618T022339Z  # fallback-lane deadlock
skippy-lab-pr858-restore-carried-20260618T023651Z  # reject trim failure
skippy-lab-pr858-restore-carried-20260618T024538Z  # restored row

Validation

Local validation:

cargo test -p skippy-server --lib direct_return
cargo test -p skippy-server --lib native_mtp
cargo test -p skippy-server --lib
cargo clippy -p skippy-server --all-targets -- -D warnings
LLAMA_STAGE_BACKEND=metal LLAMA_STAGE_BUILD_DIR=.deps/llama-build/build-stage-abi-metal cargo build --release --locked -p skippy-server

Lab validation:

• Local release skippy-server built and installed to /private/tmp/skippy-bin/skippy-server on studio54.
• Source rsynced to micstudio:~/src/mesh-llm-codex/.
• Remote release skippy-server built and installed to /private/tmp/skippy-bin/skippy-server on micstudio.
• Fresh two-stage SPEED-Bench rows run through the standard lab split harness.

[ndizazzo]

@i386 I called my guy, and he said:

1. P1: Fix rejected MTP draft trim propagation
- Valid when SKIPPY_NATIVE_MTP_DEFER_REJECT_TRIM=1.
- embedded_generation.rs:849 executes [target, draft] through local and downstream stages, but rejection commits only one token at embedded_generation.rs:917.
- With defer enabled, embedded_generation.rs:963 calls only local trim via embedded_execution.rs:218; full downstream trim is skipped versus embedded_execution.rs:169.
- Address by removing the local-only defer path, or ensuring a downstream TrimSession/queued trim happens before the next stage message.

2. P1: Disable/remove the defer flag until transactional trim exists
- Flag: frontend/native_mtp/env.rs:3, read at env.rs:15.
- Default is off, so this is opt-in, but if used it can desync downstream KV.
- Also review summary/telemetry around embedded_generation.rs:1068.

3. P2: Decide direct-return protocol policy
- PredictionReturnOpen is still accepted at binary_transport.rs:326.
- PredictionReturnHub remains exported/constructed at binary_transport.rs:43, binary_transport.rs:54, binary_transport.rs:238.
- But direct_return.rs:18 drains and drops every reply.
- If direct return is removed, reject/fail fast. If still supported, restore waiter/delivery behavior.

4. P2: Clean up misleading direct-return surface
- Public protocol still decodes PredictionReturnOpen at skippy-protocol/src/binary/types.rs:66 and types.rs:147.
- Either document it as a rejected legacy path or remove/mark it only with an explicit compatibility decision.

5. Coverage gap
- Add a multi-stage rejected-VerifySpan([target, draft]) test with auto-align off and defer-trim on.
- Add direct-return first-message handling coverage proving it either rejects explicitly or delivers replies, never silently discards.

[i386]

@ndizazzo thanks for this

[i386]

build not passing due to HF rate limits, due to different job, but previous failing job is now passing. Merging anyway.