D2 follow-up: role-singleton observation by host self-report (ssh probe deleted)

[gunbai-bot]

Follows #13217 (D2). Implements the operator ruling of 2026-10-05: role-singleton observation across hosts is self-report, and no srv1→srv2 ssh will be granted. Retarget to main once #13217 merges. Grants nothing. Transport option A is pending the operator via merry-tern-58.

What changes

• Reporter. Each host's dashboard /instance.json now carries role_singletons (role_singleton_self_report_of). The report is built from the same on-host read D2 already does (broker unit, fabric units, and the store marker's path, content, mode, owner and group), at (host, deployment, member) grain, with observed_at_unix_millis.

• Reader. self_report_reading_source_with is the only RoleSingletonReadingSource. The applying host fetches every roster host's report, its own included, using GetBoundedReportingPeer (curl %{remote_ip}). It accepts a report only if all four hold:

  • (a) tailscale whois says the serving peer is the roster row's tailnet node (extdeps.tailscale.whois, Node.Name). This binds the report to the host's own WireGuard identity, so no credential is minted.
  • (b) The report names that host.
  • (c) Every sighting in it is of that host.
  • (d) observed_at is within the fetch deadline (curl_localhost_max_time) of the reader's clock, in either direction.

  Anything else is RoleSingletonHostUnobservable and is never read as absent.

• ssh probe: deleted, not kept as a fallback. Its only reachable outcome was a refusal this path already produces. Keeping it would leave a second route (the OverTransport readback) that a later grant could silently revive. assess_and_admit loses its own transport parameter. The fixture probes in D2's sealed-caller claims are updated to the new arity so they still refuse for the sealing reason, not an arity error.

• Producer split. roadmap_serve gains dashboard_instance_served_json, the only member that has effects. dashboard_instance_json stays pure for its existing witnesses.

Owed grants (operator; until both land, prod applies refuse before planning)

1. Tailnet reach from the deploy job to each roster host's dashboard door.
2. Read access on fabric-storage-control for the dashboard service account. Today only the deploy job principal has it. Without it the reporter reports itself unobservable.

REDs, each with a control (dag/test/claim/live_deploy/role_singleton_self_report_witness_test.dag, over supplied whois results and /instance.json bodies):

• srv2's report absent, either unserved or served without the member: refuses. Control: the Noop.
• A stale report (bound + 1 ms) and a future one: refuse. A report exactly at the bound is admitted.
• A report signed as the wrong host (srv1's node, a report naming srv1, srv2 carrying srv1 sightings, an unidentified peer): refuses.
• The srv1 holder self-reports a complete marked realization: Noop.
• A moved role with srv1 still reporting: Refuse as a move. Control: the same move over an empty world installs on srv2.
• Mutants: reading a missing report as absent installs where the real reader refuses; accepting any host's report reads Noop where the real reader refuses.
• Inhabitance:
  • The real producer's report decodes as itself.
  • The real observation, which assess_and_admit_rostered decides over, runs the real fetch and refuses with the self-report cause prefix, so the claim checks the route and not only the verdict.

Remote claim_batch evidence will follow in a comment.

🤖 Generated with Claude Code

[gunbai-bot]

Remote claim_batch at head 0ec7a5c. Runner had EstimatedMemory=24GB, with GUNBC_BIND_MEMORY_CGROUP_BYTES=21474836480 set inside the script (buildbuddy invocation 39e9a967-c846-4792-ae13-b049667fd84e). role_singleton_self_report_witness_test: 12/12 PASS (RC=0), including the 5 REDs with controls, both mutants, and the producer and reader inhabitance claims.

At cd98056, D2's deployment_risk_witness_test claims ran 37 PASS / 0 FAIL before the process was killed. The fixture-compile claims (sealed caller, marker readback) are OOM-killed at the 20GB bind, both batched and alone. I am running a baseline at D2's head c07cf7e to see whether that predates this PR.

[gunbai-bot]

Baseline (buildbuddy, EstimatedMemory=24GB, bind 20GiB): no_caller_supplied_spec_reaches_the_role_singleton_assessment is OOM-killed (RC=137) at D2's own head c07cf7e, exactly as it is at this PR's 0ec7a5c. Both runs PASS every_marker_field_is_read_back_and_any_mismatch_refuses and the_marker_readback_has_three_distinct_outcomes. The kill predates this PR, so the updated fixture probes have no remote verdict either way. Head is now 8dca282 after merging D2's 8e4a0ee (closed MarkerStatField adopted on the local-only stat leg). That merge was resolve-checked locally; I did not re-run claims after it.

[gunbai-bot]

Addressed at 3c0e454:

• review 76301 (flat-scalar unit): observed_at_unix_millis is now EpochMs. The decoder refuses a negative instant (new RED a_report_with_a_negative_observed_at_is_undecodable, with a control). Note that at e7fd328 an as EpochMs cast failed at runtime ("cannot cast Int to EpochMs"); values are now built the way extdeps.clock builds them.
• review 76312 (one clock read across sequential fetches): the clock is no longer read once per observation. fetch_self_report reads the reader's clock when each response arrives and carries it as SelfReportServed.received_at_unix_millis. Freshness is judged against that receipt. New claim a_later_fetched_host_is_judged_against_its_own_receipt: srv2's report three deadlines after srv1's receipt is Noop when judged against its own receipt, and refuses when judged against srv1's instant. The real-observation claim now asserts the self-report: GET cause, so it shows the real fetch ran.
• Merged D2 ffdd800. Its local_only_reading_source is replaced by the self-report source. a_valid_ssh_credential_cannot_make_a_nonlocal_roster_host_observable is deleted, because the source no longer takes any transport input; the absent-report and wrong-host REDs cover what it guarded.
  Remote claim_batch at 3c0e454 is running; results will follow.

[gunbai-bot]

Remote claim_batch at 3c0e454 (EstimatedMemory=24GB runner, head pinned in script):

• role_singleton_self_report_witness_test, bind GUNBC_BIND_MEMORY_CGROUP_BYTES=21474836480: 14/14 PASS, RC=0. Includes the 5 REDs with controls, the per-receipt freshness claim, the negative-instant RED, both mutants, and the producer and real-fetch inhabitance claims.
• deployment_risk_witness_test, all 37 claims, bind 23622320128 (22 GiB: the compile probes peak at 18-22 GiB per sharp-heron-165's census receipt): 37/37 PASS, RC=0. Includes the sealed-caller claims that run my updated fixture probes.

[gunbai-bot]

Head 9cdd356 addresses review 76533. The freshness bound is now second_to_millisecond(curl_localhost_max_time), and the age is a typed ReportAge (aged by or ahead by a Millisecond). Remote claim_batch at 9cdd356 (24GB runner, head pinned; buildbuddy invocation f20d5335-95aa-4df4-8db9-a9f22b35a866): self-report file 14/14 PASS at a 20GiB bind; deployment_risk_witness_test 37/37 PASS at a 22GiB bind; both RC=0.

[gunbai-bot]

Head 84be6eb merges D2 bfb2ee9 (clean merge). Remote claim_batch at this head: self-report file 14/14 PASS at a 20GiB bind (RC=0). deployment_risk_witness_test at a 22GiB bind: 27 PASS, then OOM-killed (RC=137) at the first compile-probe claim. D2's own head bfb2ee9 shows the identical result (27 PASS, killed at the same claim), so this predates this PR. The probe claims have no remote verdict at 22GiB after the main merge.

[gunbai-bot]

Head 2cb52ff addresses review 76947. observed_at is now EpochMs?, sent as null on the wire when the reporter had no clock reading or its instance names no roster host. A report saying its host is unobservable is now checked before freshness, so its own cause is kept. A read report with no instant refuses. New RED: a_report_without_an_instant_keeps_its_own_cause_or_refuses. Remote claim_batch at 2cb52ff (24GB runner, 20GiB bind): 15/15 PASS, RC=0.

[gunbai-bot]

Head 85b435c merges D2 72d6a05 (clean merge). Remote claim_batch at 85b435c (24GB runner, 20GiB bind): self-report file 15/15 PASS, RC=0.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-09T18:25:00.435086Z	85b435c	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[gunbai-bot]

Closed without folding in the v1 closeout bankruptcy (#13641). Stacked on #13217, which is closed. Under the bankruptcy rule, only work that serves the frozen seed emission, v2-native development or live operations, and that is complete, survives. The branch is kept for archaeology; no follow-up obligation is created. — sent from neat-wolf-604