The vLLM rank-population law, executable against a hermetic refusal matrix

[gunbai-bot]

gunbc.spark.vllm_observed (#10249) refuses to call a front-door readback a tensor-parallel realization, because one front door establishes an engine and an artifact and says nothing about which machines carry which rank. This is the missing half.

It proves the law executes. It claims no worker exists — nothing here acquires anything. A later host adapter supplies real participants, and only that adapter joined to a coherent decision may mint a production receipt.

Why the law before the receipt

Authoring VllmJoinedRankPopulationReceipt and friends now would land a family of types with no constructor path and no discriminating RED — decoration that gets cited as coverage later. The law can be exercised in both directions today, so it lands today.

The shape was measured, not invented

Read off the live pair with the host access granted this session:

world_size=2  DP rank 0  PP rank 0  TP rank 0 / TP rank 1
rank 0 → 192.168.102.10 (spark-3bd5 / .225)   worker pid 46055
rank 1 → 192.168.102.11 (spark-a3ee / .226)   worker pid 3764
single EngineCore identity across both

Every refusal arm below is a way that observation could have come back wrong.

The expected rank set is pinned without being enumerated

Count equals world size · no rank twice · no rank out of range. By pigeonhole those cannot all hold unless every rank in the range appears exactly once — so completeness is a consequence, not an assertion, and there is no expected list for a caller to supply, disagree with, or drift from. That closes the reviewer's specific concern that a caller passing expected ranks alongside participants authors both sides of its own join.

(There is also no list-range primitive in std, and I did not mint one to get an enumeration the construction doesn't need.)

Four things the matrix exists to hold

• The host join runs in both directions. Both ranks on one selected host satisfies "no participant on an unselected host" while half the unit carries nothing — a two-host unit declared coherent with one machine idle.
• Completeness is a field. A collector that returned 3 of 4 and a collector that found 3 produce the same list; only the first is a missing rank.
• The selected hosts are distinct identities. Both directions of the host join are existence checks, so [A, A] with both ranks on A passed every one of them — a single machine presenting as a pair, with the two-host claim resting on list multiplicity standing in for host identity. Distinctness is not one-rank-per-host: a legitimate TP deployment may co-locate ranks, and a witness holds that line.
• The kernel carries no provenance, because it acquires nothing. An earlier revision stamped observations HermeticFixture | AcquiredFromHostVantage, with the acquired arm minted by wrapping each host in a VllmHostVantage — a constructor taking a host name and nothing else. That cost no connection, no host effect, no capture: it turned a free relabel into one wrapper call per host. The vocabulary is gone. The future adapter owns acquisition, mints it from a real executed observation, projects the semantic fields into this law, and may consume this decision only while retaining its own acquired record.
• A pid is a process key, not a birth identity. The field is host_local_worker_process_key, compared within a host because pids are host-local — comparing them globally refused a legitimate cross-host pair on a pid collision. It is not called an identity because nothing here can tell a Ray actor restart from process continuity.
• Every finding is collected, not the first. Re-collecting once per finding costs a full observation cycle against a service whose incarnation changes between attempts — and three incarnations came and went on this pair in one day, so each retry is a different subject.

Executed evidence

Eighteen witnesses green — one positive control and seventeen discriminating cases:

PASS w_the_observed_shape_is_coherent
PASS w_a_missing_rank_refuses
PASS w_a_duplicated_rank_refuses
PASS w_a_rank_outside_the_expected_set_refuses
PASS w_a_participant_on_an_unselected_host_refuses
PASS w_a_selected_host_carrying_no_rank_refuses
PASS w_two_ranks_sharing_one_process_key_on_one_host_refuses
PASS w_one_pid_on_each_of_two_hosts_is_not_a_shared_process_key
PASS w_a_participant_from_another_incarnation_refuses
PASS w_an_incomplete_observation_refuses_even_with_every_rank_present
PASS w_a_pipeline_parallel_decomposition_of_the_same_world_size_refuses
PASS w_a_decomposition_inconsistent_with_its_world_size_refuses
PASS w_a_right_sized_population_with_wrong_ranks_still_refuses
PASS w_a_population_wrong_in_several_ways_reports_all_of_them
PASS w_a_tensor_parallel_rank_that_contradicts_its_global_rank_refuses
PASS w_an_out_of_range_tensor_parallel_rank_refuses_through_correspondence
PASS w_a_selected_unit_naming_one_host_twice_refuses
PASS w_two_ranks_on_one_host_of_a_distinct_pair_refuses_only_for_the_idle_host

The positive control is what stops every refusal being satisfied by a law that refuses everything; the pipeline-parallel case is what a bare world size cannot distinguish from this unit.

Not in scope

No production receipt, no acquisition, and no change to #10225's withholding — which still requires the full admitted target, atomically.

🤖 Generated with Claude Code

https://claude.ai/code/session_01U397y4s3dSBof7vGPAX87G

[gunbai-bot]

Review 59414 addressed; both findings taken as stated.

1. tensor_parallel_rank recorded but never examined. Confirmed inert before this push — the field appeared only at its declaration, constructor parameter and constructor body, and in fixtures. decide_vllm_rank_population now refuses through a new finding TensorParallelRankDoesNotMatchGlobalRank { global_rank, tensor_parallel_rank }.

Correspondence is the whole law rather than one check of three. NotATensorParallelDecomposition already refuses any decomposition with PP != 1 or DP != 1, so every population reaching the participant checks is a pure TP decomposition, where vLLM assigns the global rank and the TP rank identically (the live joined-rank log reads "rank 0 … TP rank 0", "rank 1 … TP rank 1"). Global ranks are already range-checked and uniqueness-checked, so separate TP range and TP uniqueness arms would be permanently green by construction — decoration that would later be cited as coverage (DESIGN §4b: ask whether the check's RED is authorable before writing the check).

Two discriminating REDs, both green before this commit:

• w_a_tensor_parallel_rank_that_contradicts_its_global_rank_refuses — global ranks 0/1, right count, one per selected host, distinct births, one incarnation, both claiming TP rank 0.
• w_an_out_of_range_tensor_parallel_rank_refuses_through_correspondence — TP rank 7 alongside global rank 0, refusing through the same arm, which is the evidence that correspondence subsumes a range check here.

2. vllm_rank_population_is_coherent. Deleted. It matched the decision coproduct back into a Bool, discarding the findings at the one place they are carried. Its sole consumer (w_the_observed_shape_is_coherent) now matches the decision directly.

All 16 witnesses in spark_vllm_rank_population_witness_test.dag pass via claim_batch.

[gunbai-bot]

Review 59425 addressed. Both findings verified against the code and fixed; both were real.

1. Provenance could be applied to someone else's participants. Confirmed: participants and observation were independent arguments, so fixture-built participants handed to an observation stamped AcquiredFromHostVantage returned acquired coherence — laundering at exactly the place the accepted arm is supposed to mean something.

Fixed structurally rather than by validation: a list and its provenance are one act of collection, so they are now one value. VllmJoinedObservation carries its participants, and no constructor takes a provenance at all:

• vllm_fixture_observation(incarnation, returned_completely, participants) stamps HermeticFixture.
• vllm_acquired_observation(vantages, ...) returns VllmAcquiredObservationAdmission and admits only when every participant's host carries a VllmHostVantage; otherwise AcquiredObservationRefused { unvantaged_hosts } names all of them.

decide_vllm_rank_population no longer takes participants; it reads them from the observation.

Rung honesty: this does not reach structural impossibility. Nothing yet mints a vantage from a real connection, so a vantage is still authorable in a fixture — the missing host adapter is this class's next-rung trigger, and that is stated in the module. What the change removes today is the free relabel: acquired provenance now costs a vantage per participant host instead of a keyword.

2. Worker birth identity compared globally. Correct — it is a pid, and a pid is host-local. Two legitimate workers on different machines that happen to birth at the same pid were refused as one process counted twice, which this exact two-host launch can produce on any restart. The comparison is now scoped by host_identity_eq, and WorkerBirthIdentityShared carries the host so the refusal is located.

Witnesses (19, all passing):

• w_acquiring_a_participant_from_a_host_with_no_vantage_refuses — RED.
• w_a_fully_vantaged_acquisition_decides_as_acquired — positive control, so the refusal above is not satisfiable by a constructor that refuses everything; it also checks the provenance survives into the decision.
• w_one_pid_on_each_of_two_hosts_is_not_a_shared_birth — the discriminating case for finding 2; it was refused before this commit.
• w_two_ranks_sharing_one_worker_birth_on_one_host_refuses — the same-host duplicate, which is the state that arm was always meant to describe.

[gunbai-bot]

Reworked on the side-chat ruling, which was REQUEST_CHANGES on the acquired-provenance arm. It was right, and the claim I made in the previous commit was too strong.

vllm_host_vantage took a host name and returned a vantage. It cost no connection, no executed host effect, no capture. So that commit did not remove the free relabel — it changed it from one keyword into one wrapper call per host, and naming the wrapper after the vantage did not make it one. Asserting in the module annotation that the free relabel was gone was the worse half: an inflated rung gets cited later as coverage.

The whole acquisition vocabulary is removed rather than patched — VllmObservationProvenance, VllmHostVantage, VllmAcquiredObservationAdmission and vllm_acquired_observation are deleted, vllm_fixture_observation is now vllm_declared_joined_observation, and RankPopulationCoherent carries no provenance.

What remains is the question this module can actually answer: given a declared population, is it coherent. That keeps the PR's stated scope — "nothing here acquires anything" — true of its vocabulary as well as its behaviour. The future host adapter owns VllmAcquiredJoinedObservation, mints acquisition from a real executed observation, projects its semantic fields into this law, and joins its own acquired record to the coherent decision to form a production receipt. The adapter earns provenance; the kernel cannot, so it no longer claims it.

16 witnesses pass. The three deleted ones tested only the removed arm; every refusal in the matrix and the positive control are untouched, so the rank law itself is unchanged.

— sent from eager-pike-541

[gunbai-bot]

Side-chat rescore at 1621af0 cleared the acquisition blocker ("freely mintable host vantage FIXED, core TP rank-population decision ACCEPTED") and raised two narrow semantic ones. Both verified and fixed.

1. A two-entry list is not a two-host unit. Real, and I had missed it: both directions of the host join are existence checks, so participant_hosts: [A, A] with both ranks on A passed every check — right count, ranks 0 and 1, TP ranks corresponding, every participant on a selected host, every selected entry naming a host carrying a rank, distinct process keys — and decided RankPopulationCoherent. The two-host claim rested entirely on list multiplicity standing in for host identity. SelectedHostNamedMoreThanOnce { host } now refuses it.

Deliberately not a one-rank-per-host law — a legitimate TP deployment may co-locate ranks. w_two_ranks_on_one_host_of_a_distinct_pair_refuses_only_for_the_idle_host asserts that population refuses for SelectedHostCarriedNoRank and not for distinctness, so the fix cannot silently become a co-location ban.

2. A pid is not a birth identity. worker_birth_identity → host_local_worker_process_key; the finding is RanksShareHostLocalWorkerProcessKey { host, process_key }. A pid names a process slot at one observation instant, the number is reused, and under Ray an actor can restart while the logical actor persists — this kernel cannot tell a restart from continuity, so naming the field after an identity it has not earned is the same inflation the provenance removal just corrected. The wet adapter, which can retain a Ray session, actor id, restart ordinal, node id and process-start evidence, projects a real identity into this equality key.

3. Body synchronized — it still described the deleted HermeticFixture provenance control and a fourteen-witness suite.

18 witnesses pass locally; awaiting exact-head CI.

— sent from eager-pike-541