Restore the dashboard deploy ROUTE behind a fail-closed interlock — the capability stays withheld, in code

[gunbai-bot]

Retitled and rescoped after review 56947 (REQUEST_CHANGES), which is correct. The previous title — "Restore the dashboard deploy edge" — promised a capability this PR does not deliver. It restores the route, its serialization and its interlock; the deploying capability stays deliberately unavailable. That is a real distinction and the old framing collapsed it.

What this actually does

#8676 deleted deploy.yml, the only caller of live_deploy_apply_srv1_wet. Since then there has been no route at all to deploy srv1 — the tree is 604 commits behind and the roadmap belt refuses every tick on revision drift.

This restores that route as a dashboard_deploy mode with a dedicated dashboard-deploy job, and then refuses to walk it, in code, for a reason measured on the host.

Why the route is a job and not a step

srv1_dashboard_deploy_concurrency_group requires job-grain concurrency, and concurrency is a job key — a mode-gated step inside the shared converge job would have inherited that job's concurrency: none. Landing the deploy as a step would have restored the operation without the serialization the operation requires.

QueueNotMax rather than QueueMax: QueueMax permits many pending entries, so a burst would install each queued revision in turn, including revisions already superseded before they were installed. cancel_in_progress: false is the half that matters most — a queued deploy must never interrupt the one holding the host mutation window.

The timeout is its own derived tier rather than a borrowed one, and gunbc_ci_fleet_job_backstop_timeout_note is updated to record why this job's 105m bound did not move: dashboard_deploy adds no step to the converge job at all.

Why it refuses, and why that is the deliverable rather than a shortfall

The one deployment operation this route reaches transports .git as files — HEAD, objects, packed-refs, refs copied while index is excluded — alongside an rsync of the working tree. Measured on srv1 2026-08-27, /opt/gunbc/gunbc:

git status --porcelain | wc -l  ->  5650
  1964 "D "  tracked+staged present, ABSENT from the worktree
  1731 "??"  present, unknown to HEAD
  1500 "MM"  differs from HEAD in the index AND from the index in the worktree

Status column one is index-vs-HEAD, column two is worktree-vs-index — both wrong, from one execution with no concurrency involved. So this is not a race the new concurrency group addresses; one isolated run of that operation cannot produce a valid target state.

RepositoryTransitionAdmission therefore refuses LegacyGitFileSync at live_deploy_wet_with_access, before observe_candidate_release, so nothing is observed let alone mutated. Retract deliberately does not consult it: a wall that also blocked retraction would strand an operator with a host they can neither converge nor take down, which is stuck rather than fail-closed.

The alternative to this wall is not a working deploy — it is a deploy that corrupts the tree. The capability was already unavailable before this PR and this PR does not remove it; what changes is that the unavailability is now enforced and explained instead of resting on a sentence in a PR body. A prose warning is not a wall (§5), and I had written one.

It also closes ingress the concurrency group structurally cannot: both live_deploy_apply_srv1_wet and live_deploy_apply_srv1_operator_wet route through the walled launcher, so the refusal covers gunbc.apply -> DashboardSrv1 — the direct local route that never traverses a GitHub concurrency group.

Rung, stated honestly

Mechanically preventable, not structural. The wall executes and refuses on the real path, but the bound realization is a hand-declared row: someone could edit it without doing the work and nothing would notice. Next-rung trigger is in the annotation — derive the bound realization from the emitted tree-sync unit so the declaration cannot disagree with what is installed. It dissolves when the Git-native transition lands, at which point the refusal arm stops being production-reachable and its control is retained as a regression probe (§4b(4)).

Why the transition is not in this PR

#9506 builds it: three cited git operations (compare-and-swap update-ref, reflog read, worktree transition), a pure convergence adjudication whose preservation checks are identity joins, and a wet composition ordered objects → CAS → reset → readback. It currently carries REQUEST_CHANGES from side-chat review, which found real defects in it (untracked residue surviving reset --hard; the primary's own branch reported as a lost ref; a partial transition left unnamed when the reset fails after a successful CAS; a reflog law that any unrelated append satisfied). Those are fixed and it now stands at 16 witnesses, but its evidence is still pure adjudication — no witness executes the wet composition — and a scratch-repository execution matrix is owed. An earlier revision of this line said it was "approved and green on nine witnesses", which was stale in both halves and is corrected here rather than quietly updated: it is the successor this PR points at, so overstating its standing overstates the case for withholding activation here.

Binding it here would mean one PR that restores a route, replaces a production repository transport, deletes both rsync legs and changes what a deploy means — from push the runner's checkout to converge the target to an admitted commit. That last one is a semantics decision I have flagged for the operator and do not have confirmation on. Two reviewed increments are the correct shape; one is not.

Corrections carried in this PR

• The serialization rationale cited blue/green slot selection, deleted at the root on 2026-08-19 (single_deployment_note). The conclusion survived the deletion; its stated reason did not. Rewritten against in-place interleaving, moved from a commentary-only data …: String to a §4c annotation, all three dangling references updated.
• QueueNotMax was described as replacing a pending deploy with "the newest main revision". Under workflow_dispatch over a caller-selected ref, the newest arrival need not be main.
• Two witnesses this change invalidated were updated to the new truth, keeping their exact-partition shape (count and per-name membership): nine dispatch modes, three jobs — with dashboard-deploy's needs: [build] edge asserted explicitly.

Still open, recorded in the group's own annotation rather than carried in prose

Serialization is not ordering (a slow build of an older revision can acquire the lease after a fast one and install backwards); the host-local lease and a durable actuator inhibition are two mechanisms, not one, because a process lease dies with the process; and the candidate is not yet proven equal to the admitted refs/fleet/desired.

[gunbai-bot]

Two updates from a side-chat review, one correcting this PR and one correcting the review.

Fixed here (76712ab0856): the serialization rationale cited a mechanism that no longer exists. srv1_dashboard_deploy_serialization_note justified the concurrency group with blue/green slot selection — "both observe blue active, the first flips green, the second restarts the active green slot". Blue/green was deleted at the root on 2026-08-19 by operator directive (gunbc.live_deploy.apply single_deployment_note); apply now converges one instance in place. The conclusion survived the deletion and its stated reason did not, which is the §3 stale-citation defect, and the load-bearing kind: a reader who checks the cited mechanism finds it gone and correctly concludes the group is obsolete. I introduced it in this PR.

The reason is now stated against the deployment that actually runs — two candidate transactions interleaving writes to one tree, binary, unit, process and readiness subject, so each transaction's readback can observe the other's artifacts. In-place convergence makes that stronger, not weaker. The carrier also moved from a data …: String (commentary with no consumer — §4c "misplaced or dead data") to a module-scope annotation on the concurrency group itself, and all three dangling references to the deleted name were updated, including one that restated the blue/green claim in full.

I also weakened a claim I could not support: QueueNotMax was described as replacing a pending deploy with "the newest main revision". While the trigger is workflow_dispatch over a caller-selected ref, the newest arrival need not be main at all.

Stale in the review: "the generated workflow does not contain the change". That was measured at 7d9375b; the regenerated YAML landed in 096ad5a, the next commit. The committed fleet-converge.yml carries the mode option, the dashboard-deploy job, group: srv1-dashboard-deploy and cancel-in-progress: false.

Not fixed here, and recorded as open rather than quietly carried — three of them are now written into the group's own annotation so a later reader cannot mistake this group for a wall it is not:

• Serialization is not ordering. A slow build of an older revision can acquire the lease after a fast build of a newer one and install backwards. Needs a forward-only check after acquiring the lease.
• Ingress is not complete. gunbc.apply → DashboardSrv1 → live_deploy_apply_srv1_wet never traverses a GitHub concurrency group. The closing construction is a host-local lease the host effect owns; GitHub owns only one transport into the critical section.
• The candidate is not proven to be the admitted one. The deploy observes a clean candidate but not that it equals refs/fleet/desired.
• The .git rsync leg is unchanged. GunbcSourceTree still copies HEAD/refs/**/packed-refs as files.

That last one carries an operational consequence I want stated plainly rather than left in a thread: merging this PR is safe, but the restored mode should not be dispatched against srv1 until the .git rsync leg is replaced. This PR restores a route; it does not make the operation it routes to safe. The job-grain concurrency prevents two of that operation from overlapping — it does not make one of them correct.

— sent from calm-ram-380

(Edited: the SHA above posted unexpanded from a quoted heredoc — a citation naming nothing, in a comment about stale citations. Corrected to the commit that carries the fix.)

[gunbai-bot]

Measured the live consequence of the .git rsync leg on srv1, so the "do not dispatch" note above is a number rather than a worry.

/opt/gunbc/gunbc, the tree this deploy converges, is in three-way divergence — HEAD, index and worktree each disagree:

git status --porcelain | wc -l   ->  5650

   1964  "D "   tracked, staged as present, ABSENT from the worktree
   1731  "??"   present in the worktree, unknown to HEAD
   1500  "MM"   differs from HEAD in the index AND from the index in the worktree
    371  "AD"   added to the index, absent from the worktree
     84  "RD"   renamed in the index, absent from the worktree

HEAD        517fdac4ddf92d0dd636c1f66fdb95b508525067   (the deployed release)
origin/main dabe4c580488f4a312a5bfe76b380df9eab35c4d

The first column is index-vs-HEAD and the second is worktree-vs-index, so both are wrong, not just one. That is what the two ExecStart legs produce together: the first rsyncs the runner's working files from one revision, the second copies HEAD/objects/packed-refs/refs from that checkout as files while index is excluded by the whitelist. Files from one revision, metadata from another, an index describing neither.

This is not latent. It is the current state of the tree the belt spawns dispatch worktrees out of.

One measurement that changes the available options, and it is the useful half: srv1 can fetch natively today.

$ git fetch --dry-run origin main
   dabe4c580488f..9538523a3cf17  main -> origin/main
rc=0

Credentials and network are fine — the same gh-provided auth that makes the belt's ls-remote work. So the target is capable of converging itself through git, and the file-level copy is not compensating for a missing capability.

That also collapses two of the open items above into one. Ordering and admitted-SHA coupling were listed as separate gaps, but a git-native convergence cannot be written without naming a target commit — there is no "fetch whatever the runner happens to have", because the runner's checkout is not reachable from srv1. So the Git-safe transition and the admitted-SHA coupling are the same change, not two that must be sequenced.

I have not built it, because it carries a semantics decision that is not mine to make unilaterally: a git-native convergence would make the deploy mean "converge the target to an admitted commit" rather than "push the runner's checkout", and it would collapse both rsync legs rather than just the .git one — git materializes the tree itself, so the working-file rsync stops being necessary too. That is a replacement migration of a production path (§3 delete-first), and the thing it changes is what a deploy is.

Nothing here changes this PR. It is recorded on it because this PR is what restores the route to that operation, and the note above is the reason not to walk it yet.

— sent from calm-ram-380

[gunbai-bot]

Replaced my own warning with a wall (fd8bc46f523). A side-chat review put it plainly and it is DESIGN §5 aimed at me: a note in a PR body is not a wall. I wrote prose where a construction was available, and the unsafe state stayed writable.

What landed. gunbc.live_deploy.deployed_tree_scope now declares the bound repository transition and adjudicates it:

type RepositoryTransitionRealization = LegacyGitFileSync | GitNativeConvergence
data deployed_tree_repository_transition: RepositoryTransitionRealization = LegacyGitFileSync
type RepositoryTransitionAdmission = RepositoryTransitionAdmitted | LegacyGitFileSyncNotAdmitted { reason: String }

live_deploy_wet_with_access matches on it before observe_candidate_release, so a dispatch of the restored mode refuses with a typed cause and nothing is observed, let alone mutated. The ordering is deliberate: live_deploy_candidate_observed_at_launcher_note calls that position the cheapest place to refuse, and a realization that cannot produce a valid target state is refusable one step earlier still, since it does not depend on which candidate was handed over. Observing first would make the log read as though the candidate were the problem.

Home chosen for acyclicity, not tidiness. deployed_tree_scope is the existing leaf that already owns the working-tree-leg fact, precisely because candidate and emit sit on opposite sides of a dependency chain.

It covers more ingress than the concurrency group can, and I checked the call graph rather than assuming. Both live_deploy_apply_srv1_wet and live_deploy_apply_srv1_operator_wet route through the walled launcher, so the refusal reaches the fleet-converge job, the operator entry, and gunbc.apply -> DashboardSrv1 — the direct local route that never traverses a GitHub concurrency group at all.

Stated precisely, because the stronger result must not be read as retiring work: this closes ingress for the admission question ("may this realization run at all"), not the serialization question ("is another transaction running now"). The host-local lease is still owed, and so is the durable actuator inhibition beside it — a process lease dies with the process, so a deploy that fails after moving the tree and before restarting would leave the next belt tick free to consume mixed state. Those are two mechanisms answering different questions, and I had them fused until review separated them.

Retract deliberately does not consult this. Removing a deployment performs no repository transition, and a wall that also blocked retraction would strand an operator with a host they could neither converge nor take down. That is stuck, not fail-closed.

Rung, honestly: mechanically preventable — not structural. The wall executes and refuses on the real path, but the bound realization is a hand-declared row: someone could edit it without having done the work and nothing would notice. Next-rung trigger is recorded in the annotation — derive the bound realization from the emitted tree-sync unit, so the declaration cannot disagree with what is installed. Dissolves on the Git-native transition landing, at which point the refusal arm stops being production-reachable and its control is retained as a regression probe rather than deleted (§4b(4)).

Evidence — dag/test/claim/repository_transition_admission_witness_test.dag, executed:

PASS the_two_realizations_are_admitted_differently
PASS the_deployment_is_currently_bound_to_the_legacy_realization
PASS the_refusal_names_the_mechanism_rather_than_only_a_verdict
PASS an_admitted_realization_carries_no_refusal_reason
[resolve-summary] 1 resolve(s) in 363ms wall; 4 witness(es) in 0ms wall

The pair is the witness, not either arm. A wall asserted only on the arm that currently refuses cannot distinguish a real decision from a function that returns the refusal unconditionally — so both arms are asserted over one function, and fusing them makes the admitted case red. The third pins the row to legacy, which means flipping that row goes red and forces the author to state that the realization actually moved; the flip cannot be silent.

Also fixed on this PR: the body still carried the stale blue/green rationale after the carrier was corrected. Corrected there too, with the correction stated rather than silently swapped.

Two process notes, since the first run of these witnesses looked green and was not: it failed with name 'and' not found in module 'std.logic' and exited 0. A zero exit from this toolchain establishes nothing — the N witness(es) footer and the PASS lines do.

— sent from calm-ram-380

[gunbai-bot]

CI failure was inherited, and I checked my own change before saying so — because the reasoning I used on #9501 had a hole when applied here.

The hole. The four admission witnesses on this PR import deployed_tree_scope, not apply.dag. So the file where the refusal is actually wired had never been resolved by anything I ran, and a resolve error there would refuse preparation with exactly the signature I was about to attribute to contention.dag. The annotations rule out a witness failure (no disposition artifacts means nothing executed) but they do not rule out a resolve error of mine.

Closed by execution. Ran the witnesses that do reach gunbc.live_deploy.apply:

claim_batch --entry dag/test/claim/live_deploy/apply_witness_test.dag --functions <3>
claim_batch: resolved dag/test/claim/live_deploy/apply_witness_test.dag; running 3 witness(es)
PASS witness_red_control_mutation_without_readiness_cannot_converge
PASS witness_red_control_service_not_ready_reason_surfaces_on_blocked_apply
PASS witness_digest_readback_script_is_bounded_and_post_mutation
[resolve-summary] 1 resolve(s) in 63684ms wall; 3 witness(es) in 389ms wall

apply.dag resolves with the wall in place, and the pre-existing apply witnesses still pass — including the two red controls, which is the part worth having: the wall sits ahead of the launcher, and those controls establish that the refusal path below it is unchanged.

And the blocker is gone. #9488 merged at 19:25Z. I verified the repair on main by content rather than by the merge notice — contention.dag now derives a local grant_duration_observation instead of importing the renamed grant_duration_seconds, and dc76571d90f is an ancestor of main's head. Main has been merged into this branch (a merge commit, not a rebase), so CI is re-running on a repaired base.

— sent from calm-ram-380

[gunbai-bot]

Addressing review 56947 (REQUEST_CHANGES). The finding is correct and I have taken the second of the two remedies it offers — rescope and retitle — rather than the first.

Both facts check out: gunbc.live_deploy.deployed_tree_scope deployed_tree_repository_transition is statically LegacyGitFileSync, and gunbc.live_deploy.apply live_deploy_wet_with_access refuses that realization ahead of every mutation. So every dashboard_deploy dispatch fails before deployment. That is exactly what the code does, and the old title — "Restore the dashboard deploy edge" — claimed a capability it does not deliver. That is semantic dilution and the review is right to call it.

What changed: the PR is retitled "Restore the dashboard deploy ROUTE behind a fail-closed interlock — the capability stays withheld, in code" and the body rewritten to lead with the distinction rather than bury it. Route, serialization and interlock are delivered; the deploying capability is deliberately withheld.

Why not the first remedy — landing the Git-native transition with this caller. Three reasons, and the last is the one I cannot resolve unilaterally:

1. The alternative to the wall is not a working deploy, it is a corrupting one. The single operation this route reaches leaves the target with working files from one revision and git metadata from another — measured on srv1 as 5650 divergent paths with index-vs-HEAD and worktree-vs-index both wrong, from one execution with no concurrency involved. Making the mode dispatchable today means dispatching that. Refusing is strictly better than succeeding at it.

2. This PR does not remove a capability that existed. #8676 deleted deploy.yml eight days ago; there has been no route to deploy srv1 since, and the tree is 604 commits behind. What changes here is that the unavailability is enforced and explained instead of resting on a sentence in a PR body — which is precisely the §5 correction I made after a reviewer pointed out that a warning in a PR body is not a wall. I had written that warning.

3. Binding the transition changes what a deploy means. #9506 builds it and is approved and green on nine witnesses, but binding it deletes both rsync legs and moves the deploy from push the runner's checkout to converge the target to an admitted commit. That is a semantics decision I have flagged for the operator and do not have confirmation on. Folding it in would make one PR that restores a route, replaces a production repository transport, and redefines deployment — three separable subjects, one of them blocked on an external decision.

So the two-increment shape is deliberate, and the review's objection was to the framing rather than to the shape. The framing is fixed.

One correction to the review, on §3 grounds rather than substance: the findings cite deployed_tree_scope.dag:83 and apply.dag:381. Those positions are already stale — this branch has moved since — and the repository's standing rule is to cite the symbol rather than the position, because a line rots without anyone touching either end. The symbols are deployed_tree_repository_transition and live_deploy_wet_with_access; both resolve.

— sent from calm-ram-380