[Router] Vet a peer's snapshot before it may touch the tree (6/13)

[Kangyan-Zhou]

Stack 6 of 13. Base: router-peer-bootstrap-5-tracker-fetch. 5 files changed, 915 insertions(+), 18 deletions(-)

The stack

#	PR	lines	what it adds
1	#40687	962	the sharded tree gains export_snapshot / restore_snapshot; nothing calls them yet
2	#40688	2213	GET /internal/kv_snapshot — the producer half; nothing consumes it yet
3	#40689	536	--kv-peer-selector, the peer registry, the _peers gauges; no behaviour change
4	#40690	1150	the EndpointSlice watch that fills the registry; still no consumer
5	#40691	1177	BootstrapTracker + the fetch client; the consumer's state and transport
6	#40692	933	VettedSnapshot::from_wire — the only bridge from wire bytes to the tree ← this PR
7	#40693	2750	the pump holds a Pending rank's batches and grafts a snapshot handed to it
8	#40694	2478	the sweep: ask siblings, vet, deliver — one sweep per discovered worker
9	#40695	1211	fold a discovery burst into one fleet-wide fetch, plus the gap retry it enables
10	#40696	844	a graft nothing witnessed asks the fleet instead of guessing
11	#40697	646	component test over the real transport: same match_prefix answers as the source
12	#40698	943	kind-cluster proof, the KV-publishing fake worker, and the RBAC/downward-API manifest
13	#40699	869	/readyz holds until bootstrap settles; --kv-bootstrap-seed-required; the metrics

Each PR's base is the branch below it, so every diff shown here is that
PR's own change. Review bottom-up; GitHub retargets each child to main as
its parent merges.

What the series does

A router replica subscribes to each worker's KV topic mid-stream, so every
block already resident in the engine's radix cache is invisible to it — and
engines publish BlockStored only as they insert, so a prefix cached hours
ago is never re-announced. A cache-blind replica then scatters the prefixes the
warm replicas were keeping consolidated, degrading the engines' locality for
the whole fleet; a rolling update does that to every replica in turn. This
series makes a booting replica pull a tree snapshot from a warm sibling over
HTTP and graft it beneath its live delta stream. Off unless --kv-peer-selector
is set.

Supersedes #39750, which carried the same work as one branch on a stale base.

What this change does

The last piece of the consumer's half of the wire contract, still with no
consumer: the type that turns a peer's PeerSnapshot into something graftable.
The pump that uses it lands next.

The trust boundary is enforced by types rather than by discipline. A snapshot
arrives over the network as PeerSnapshot, carrying WireWorker rather than
KvWorkerId; VettedSnapshot::from_wire is the only bridge, and it resolves
every wire identity against the locally discovered worker set, so a peer cannot
introduce a KvWorkerId the local registry does not already know — which would
otherwise let routing resolve to a non-registered endpoint. VettedSnapshot's
fields are private and graft_into is the only route to the tree's restore path
from outside state::kv_events, so format, block-size and parent-bound checks cannot be
skipped by a caller that assembles nodes itself. That is also what retires
restore_snapshot's #[allow(dead_code)].

Vetting rejects rather than repairs: an unknown format, a producer that is
itself cold or not ready, an out-of-range or forward parent reference, tiers
that do not pair with workers, a block-size mismatch, two cursors for one
worker or two worker-table entries naming the same live worker, or a snapshot
that prunes to nothing. Pruning happens at vetting, not at restore: carriers of
workers this replica does not know and carriers held only on tiers this build
does not rank are dropped, and every node left carrying nothing with no
carrying descendant is pruned, so a carrier-less leaf never turns a real hit
into a match with no workers. A rank whose snapshot is rejected runs cold, which is slow and
visible; a snapshot grafted on a hope can hide a missed BlockRemoved as a
permanent false cache hit, which is silent.

Tests

cargo fmt --check, cargo clippy --all-targets -- -D warnings, and the lib +
component + proxy suites all pass on this branch on its own, not only on the
tip of the stack.

Review pass

Reviewed with /code-review --fix and /simplify; fixes were folded into this PR's own commit and the stack was re-verified tier by tier (cargo fmt --check, cargo clippy --all-targets -D warnings, lib + component + proxy tests on every branch).

• Fixed: carriers on tiers this build does not rank are pruned at vetting, and duplicate cursors / duplicate worker identities are rejected.
• File layout: this PR's code lives in state/kv_events/bootstrap/vet.rs (new), plus tree/snapshot.rs; no file the stack creates exceeds ~1,300 lines.

🤖 Generated with Claude Code

https://claude.ai/code/session_016HmJvHV7QDPk3qAjQYzthd

---

CI States

Latest PR Test (Base): ✅ Run #36559683560
Latest PR Test (Extra): ❌ Run #36559682954
Latest PR Test (AMD ROCm 10): ➖ No AMD PR run found for this commit.

[ShangmingCai]

This one is also very straightforward and it looks correct to me.

[ShangmingCai]

/tag-and-rerun-ci