[brentkelly]

Part of T3O-60, the distributed board. This is the foundation that the later cards build on. It rewrites the overview (D19–D22), adds the worker registry, the central server's own connection to each worker, the worker-side boardWorker.* RPCs, project ↔ worker mapping with Set up on worker, and the Workers and Project checkouts sections in Settings → Board.

With no workers added, the board behaves and renders as it does today.

🤖 Generated with Claude Code

[brentkelly]

High-level summary

This PR lays the foundation for the distributed board. It rewrites the overview (D19–D22) and adds an event-sourced worker registry with its own board migration (046). It adds the central server's own credential exchange and long-lived connection to each worker, the worker-side boardWorker.* handshake, provider and project RPCs, project ↔ worker mapping with clone-based Set up, and the Workers and Project checkouts settings. The layering is clean. Registry commands are internal and reached only through service methods. A local-only board's snapshot and stream stay byte-for-byte unchanged. The decider and rehydration paths match the replay fold. The tests use the test clock rather than sleeps.

Nothing here is critical. This round has five improvements and two nitpicks, and most concern the runtime edges of the connection work, not the registry. Worker status changes can be lost between a subscriber's initial read and its change subscription. A retry-in-place that the worker no longer recognises leaves Set up spinning forever. The redial loop flashes offline workers blue and re-broadcasts every cycle. Failed adds and re-pairs leave stale board sessions on the worker. The nitpicks are a misleading "incompatible" for transient handshake failures and inconsistent URL building between the HTTP and WebSocket paths.

No browser or two-client check was part of this review, matching the build notes.

[brentkelly]

Warning

Improvement (1/5): Worker status can be missed between the initial read and the change subscription

subscribeBoard reads workers.state for the snapshot (line 256) or the resume resync (line 241). Only later, once deltasAfter starts running inside Stream.concat, does it subscribe to workers.changes. BoardWorkers.changes comes from SubscriptionRef.changes(...).pipe(Stream.drop(1)), so it only sees changes made after it subscribes. A status change in that gap (connecting → online while the snapshot loads, or a setup reaching done) is lost. The client keeps the stale value until something else changes, and a healthy worker that stays online never changes again, so its row can show "Connecting" indefinitely. Boot is exactly when every worker is dialing and clients are reconnecting, which makes this likely. The todo stream avoids the same race by subscribing eagerly (yield* threadTail.subscribeTodoChanges), and BoardWorkers.states ("subscribed atomically with the first read") was added for this but is unused.

Suggested fix: Subscribe once, up front, with a states-style stream that yields the current value first. Take its first element as the snapshot/resync value and feed the rest as workerDeltas. Alternatively, Stream.toPubSub the changes before the initial read. Add a subscription test that flips a worker's status between the snapshot read and the first delta.

[brentkelly]

Fixed in a697da6. The subscription now subscribes to BoardWorkers.states, which opens with the summary as of when the stream starts. Any change made after the opening read therefore reaches the client. Items are compared with Schema.toEquivalence(BoardWorkersState) against what the client already holds, so an unchanged summary isn't re-sent and a local-only board's stream stays byte-for-byte as before. BoardWorkers.changes is now unused and has been removed. Test: boardSubscription.test.ts › "sends a worker change that lands between the opening read and going live". It fails on the old code, which gives [snapshot, card-upserted] with no workers item.

[brentkelly]

Adjudication (round 1): fix-upheld. In a697da6 the subscription now runs workers.states. That stream reads the summary when it subscribes (SubscriptionRef.changes), and its first value is compared against the state the client was actually sent. A change that lands in the gap therefore differs from that state and gets sent. The new test 'sends a worker change that lands between the opening read and going live' feeds a stale opening state and a fresh states and asserts the extra workers item. It passes on HEAD and fails on 1425bc8. One narrow leftover, not blocking: changesWith dedupes against the last value this stream emitted, not against what a log-path workers item may have sent since. Both paths read live state, so they converge on the next runtime change.

[brentkelly]

Warning

Improvement (2/5): Retry-in-place ignores applied: false and leaves the setup stuck on "cloning"

When retryInPlace is true, start is rpc.retryProjectClone(...), which resolves to the tracker's applied boolean. The result is never checked. Upstream ProjectCloneTracker.retry returns false when it no longer tracks the clone (it is in-memory, so a worker restart drops it) or the clone isn't failed or cancelled. The setup is still marked cloning and follow subscribes to projectClones. follow filters for workerProjectId, which never appears, so runForEachWhile never ends while the connection is up. The Project checkouts cell shows a blue spinner until the worker disconnects or someone dismisses it. Example: a clone fails with no-credentials, the worker restarts (the setup row on the central server stays failed), the human fixes credentials and clicks Retry, and it spins forever. The same hang happens if the tracked clone disappears from the worker's list for any other reason, such as being discarded on the worker.

Suggested fix: Treat applied === false as "start a fresh clone" (fall through to startProjectClone with a new id), or fail the setup straight away. In follow, end with a failure once an emission no longer contains a clone that was seen before, or when the first emission doesn't contain it at all.

[brentkelly]

Fixed in 58533e9. If retryProjectClone returns applied: false, setupProject now starts a fresh clone with a new worker project id and follows that id. Test: BoardWorkers.test.ts › "a retry the worker no longer tracks starts a fresh clone instead of hanging". It fails on the old code, which never started a second clone. The fake worker gained retryApplied to drive it.

[brentkelly]

Adjudication (round 1): fix-upheld. In 58533e9, retryProjectClone returning applied: false now falls through to startFresh with a new project id, and that id is followed. Because workerProjectId comes from started.success, the follow can no longer wait on an id the worker never emits. The test 'a retry the worker no longer tracks starts a fresh clone instead of hanging' passes on HEAD and fails on the old code.

[brentkelly]

Warning

Improvement (3/5): Every redial flips an offline worker to blue "Connecting" and broadcasts it to every client

attempt begins with setState(worker.id, "connecting") on every loop iteration, including redials of a worker that is already offline. For an unreachable host that times out instead of refusing, the ticket request (10 s) and socket open (10 s) keep it connecting for up to about 20 s of every ~50 s cycle. The Workers list and card notices then alternate amber "Offline" and blue "Connecting". docs/t3o/status-colours.md reserves blue for running work, and AGENTS.md specifically warns against a lying spinner. Each flip is also a SubscriptionRef change, so BoardWorkers.changes pushes the whole workers state to every board subscriber twice per cycle, per offline worker, indefinitely.

Suggested fix: Only set connecting on the first dial and after restart/re-pair. On redial from offline/incompatible, stay in the current state until the attempt resolves. If a "retrying" hint is needed, use a non-status field that the UI doesn't colour blue.

[brentkelly]

Fixed in bbc5758. A redial of a worker that is already offline or incompatible keeps that status until the worker answers. Blue connecting is only for a first dial or a re-pair. update now skips publishing when an entry is unchanged (SubscriptionRef.updateSome), so the retry loop no longer broadcasts the same offline state each cycle. Test: WorkerConnections.test.ts › "redials an offline worker quietly: it stays offline, never blue connecting". It sees only [offline] over 2 minutes of redials, then [offline, online]. The old code produced [offline, connecting, …(13)].

[brentkelly]

Adjudication (round 1): fix-upheld. In bbc5758, markDialing leaves an offline or incompatible entry as it is. update now uses updateSome and setState checks every field, so an unchanged entry is never published. A dropped worker goes to offline and stays there across redials. unauthorized stops the loop, so it can't flash either. The test 'redials an offline worker quietly' passes on HEAD and fails on the old code.

[brentkelly]

Warning

Improvement (4/5): A failed add or re-pair after the exchange leaves an orphaned board session on the worker

pair spends the one-time token in exchangeWorkerCredential and then probes. probe gives the session back only when it gets the incompatible result. It doesn't release the session when readWorkerSnapshot fails ("stopped answering while being added"), when transport.connect fails after the exchange, or when secrets.set fails. In add, the secret is also written before engine.dispatch(board.worker.register). If that dispatch is refused (for example, two concurrent adds of the same environment both pass the existing check), the token stays in ServerSecretStore under that id, and the worker keeps a live T3o board (...) session that the board doesn't track. The worker's Connections list grows a stale client with standard scopes that only a human revoking it on the worker can clean up. That is a one-way door the brief's "Ways back" section is meant to avoid.

Suggested fix: Once the exchange succeeds, wrap the rest of pair/add in an Effect.onError that releases the new session (a best-effort release over a fresh link, or revokeSession if reachable) and removes the secret if it was written. Add a BoardWorkers test where the snapshot read fails after the exchange and assert the session is released.

[brentkelly]

Fixed in 6db4fae. In pair, everything after the exchange (the probe, the snapshot read, storing the secret) now runs under Effect.onError(abandonPairing). That dials with the new session and calls release, then removes the stored token only if it is still ours, so a concurrent add's token is never deleted. A failed board.worker.register dispatch in add and in re-pair goes through the same path. Test: BoardWorkers.test.ts › "a handshake that fails on the wire is unreachable, and its new session is given back". It asserts releasedTokens == [access-blip], no secret stored and no worker registered. It fails on the old code.

[brentkelly]

Adjudication (round 1): fix-upheld. In 6db4fae, pair wraps the probe and secrets.set in Effect.onError(abandonPairing), and add and re-pair both wrap their register dispatch the same way. abandonPairing releases the new session and removes the stored secret only if it still holds this token, so a concurrent successful pairing isn't clobbered. Every failure point after the exchange and before the record lands is now covered. The test 'a handshake that fails on the wire is unreachable, and its new session is given back' passes on HEAD and fails on the old code.

[brentkelly]

Warning

Improvement (5/5): Re-pair overwrites the stored token without releasing the session it replaces

repair calls pair, which writes the new access token over workerSecretName(worker.id). The previous session is never released. When the old session is already revoked (the "Access revoked" case) that is fine. Re-pair is also offered from the row menu for online, offline and incompatible workers, and repair accepts a new URL. In every one of those cases the old session is still valid on the worker, so each re-pair adds another T3o board (...) client to the worker's Connections. The board no longer holds the token, so it can never revoke it, and Remove later releases only the newest one.

Suggested fix: Before overwriting, if the worker is currently online, call release on the existing connection (best effort, with RELEASE_TIMEOUT, as remove does). Otherwise accept the leak but say so in the Re-pair dialog copy. Cover the online re-pair case in BoardWorkers.test.ts.

[brentkelly]

Fixed in 6db4fae. Re-pair reads the previous token before pairing. Once the new session is recorded, it releases the previous one by dialing the old URL with the old token, as a best effort with a timeout. If the old session was already revoked, the dial just fails and is ignored. Test: BoardWorkers.test.ts › "re-pairing a worker whose session still works gives the old session back". It checks that releasedTokens == [access-one-time], that the stored token is the new one and that the redial uses it. It fails on the old code.

[brentkelly]

Adjudication (round 1): fix-upheld. In 6db4fae, re-pair reads the previous token before pair overwrites it. After the new record and connection are in place, it releases that token against the old worker.url, then restarts the connection. The worker revokes 250 ms after it answers release, and restart follows straight away on the new token, so the dial isn't left on a dead session. The test 're-pairing a worker whose session still works gives the old session back' passes on HEAD and fails on the old code.

[brentkelly]

Note

Nitpick (1/2): Any failed hello is reported as "incompatible", including a transient drop

link.rpc.hello.pipe(Effect.option) turns every WorkerRpcError into "this server doesn't speak the board-worker protocol". That includes the socket closing or the ping timing out right after it opened. In the connection loop this resolves itself on the next dial. In the add flow, though, the user is told "{worker} runs T3o X. This board needs 0.0.45 or newer" for what was really a network blip, and the new session is released. The same applies at WorkerConnections.ts:204.

Suggested fix: Tell the two apart. Only an RPC-level "unknown method"/decode failure means incompatible. A transport-level failure means unreachable (add) or offline (loop).

[brentkelly]

Fixed in 6db4fae. Fixing this turned up a related bug. A server without boardWorker.hello (upstream) answers with an RPC defect (Unknown request tag), not a typed error. Effect.option doesn't catch defects, so the old code would have died in both the add probe and the connection loop. The transport now maps every worker call through call(), which turns typed failures into WorkerRpcError and defects into WorkerRpcError { unsupported: true }. Add and the connection loop treat unsupported as incompatible and any other hello failure as unreachable/offline. Tests: WorkerTransport.test.ts › "a server with no board-worker handshake fails hello as unsupported" (against a real RPC server without the hello rpc), BoardWorkers.test.ts › "a handshake that fails on the wire is unreachable…", and WorkerConnections.test.ts › "a handshake that fails on the wire is offline, not incompatible". All three fail on the old code.

[brentkelly]

Adjudication (round 1): fix-upheld. In 6db4fae, worker calls map RPC defects, which is how a server answers an unknown tag, to WorkerRpcError{unsupported:true}. Only that error counts as incompatible. Other hello failures become unreachable in add and offline in the connection loop. This also removes the defect that would have escaped Effect.option. The tests 'a server with no board-worker handshake fails hello as unsupported' and 'a handshake that fails on the wire is offline, not incompatible' both pass on HEAD and fail on the old code. The second hangs to its 120 s timeout there.

[brentkelly]

Note

Nitpick (2/2): /ws URL drops any path prefix the worker URL carries, unlike the HTTP calls

toWsUrl and requestTicket resolve /ws and /api/auth/websocket-ticket with new URL("/...", url), which replaces the path. workerCredential.ts builds ${httpBaseUrl}/oauth/token and ${httpBaseUrl}/.well-known/t3/environment, which keeps it. A worker behind a reverse proxy at https://host/t3/ would pass the add flow's descriptor and exchange steps, then fail to connect.

Suggested fix: Build all worker URLs the same way, e.g. new URL("ws", url.endsWith("/") ? url : url + "/"), or reject URLs with a path in resolveWorkerTarget.

[brentkelly]

Fixed in 6db4fae. /ws and /api/auth/websocket-ticket are now joined onto the worker URL with workerPath, which appends to the URL the same way workerCredential.ts does, so a path prefix is kept. Test: WorkerTransport.test.ts › "a worker served under a path prefix is dialed under that prefix", which serves a real worker under /t3. It fails on the old code.

[brentkelly]

Adjudication (round 1): fix-upheld. In 6db4fae, workerPath joins the path onto the trimmed base URL, which keeps the prefix. It's used for both /ws and /api/auth/websocket-ticket, matching workerCredential.ts. The test 'a worker served under a path prefix is dialed under that prefix' passes on HEAD and fails on the old code.

[brentkelly]

High-level summary — round 2 (re-review at 6db4fae)

This PR lays the distributed board's foundation. It rewrites the overview (D19–D22), adds the event-sourced worker registry with its migration, the central server's own session and live connection to each worker, the worker-side boardWorker.* handshake, and Set up on worker. On the web side it adds the Workers and Project checkouts settings. The round-1 fixes hold. I re-read each one at the line and found no regressions.

Looking past the diff this round, I found two improvements and three nitpicks. Improvement 1: upstream issues every bearer session with a fixed 30-day TTL and offers no refresh. As a result, every worker the board pairs will turn "Access revoked" about a month after it was added, and recovering it needs a manual re-pair. Improvement 2: re-pairing through the picker replaces the board's working URL for the worker with whatever URL the browser uses to reach it. The nitpicks are the "10 minutes" link-expiry copy (the server default is 5), the order of the steps in Remove, and a check-then-act race in setupProject. The local-only path is unchanged.

Status of round-1 findings

#	Prior finding	Status
I1	🟡 Worker status can be missed between the initial read and the change subscription	✅ closed
I2	🟡 Retry-in-place ignores applied: false and leaves the setup stuck on cloning	✅ closed
I3	🟡 Every redial flips an offline worker to blue Connecting	✅ closed
I4	🟡 Failed add/re-pair after the exchange orphans a session on the worker	✅ closed
I5	🟡 Re-pair overwrites the stored token without releasing the old session	✅ closed
N1	🟢 Any failed hello is reported as incompatible	✅ closed
N2	🟢 /ws URL drops any path prefix	✅ closed

[brentkelly]

Warning

Improvement (1/2): The board's session on every worker expires after 30 days and is never renewed

exchangeWorkerCredential spends the pairing token on /oauth/token. Upstream's EnvironmentAuth then issues an ordinary bearer session through sessions.issue with no ttl, so SessionStore's DEFAULT_SESSION_TTL = Duration.days(30) applies. Nothing on either side refreshes it: the exchange returns no refresh token, and the board never re-issues its session. Thirty days after a worker is added, requestTicket gets a 401 and WorkerConnections marks the worker unauthorized. That is a terminal state with no redial, and the UI shows it as "Access revoked". So every worker drops off the board on a fixed schedule, and once later plans place cards on workers, those cards stall until a human re-pairs. The UI also blames a revocation that never happened.

Suggested fix: Have the central server renew its own session before it expires. One way is a boardWorker.renew RPC on the worker that issues a fresh session for the caller, then revokes the old one, called when the stored session nears expiry. Store the expiry beside the token so the board knows when. At minimum, record the expiry, tell "expired" apart from "revoked" in the status and copy, and document the 30-day limit in docs/user/board-workers.md.

[brentkelly]

Fixed in 9a61336 (+ 273d5b9). The board now renews its own session instead of expiring:

• boardWorker.hello reports the calling session's sessionExpiresAt.
• New boardWorker.renew issues a successor with the caller's own subject, scopes and label, so a session can extend itself but never widen itself. The old session stays valid until the caller gives it back.
• WorkerConnections holds an online link until it drops or until SESSION_RENEW_AHEAD (7 days) before expiry. At that point it renews, stores the successor, releases the old session, and redials without leaving online (no blue connecting). If the renewal fails, the old token stays in place.

The remaining limit is a board server that was off for longer than the rest of the lifetime: it comes back unauthorized and needs a re-pair. That case is now in docs/user/board-workers.md.

Tests: WorkerConnections.test.ts 'renews the board's session before it expires, and stays online throughout' hangs on the old code, because no renewal ever happens. workerRpc.test.ts 'renew issues a successor with the caller's own subject, scopes and label' and 'hello reports when the calling session expires' cover the worker side.

[brentkelly]

Adjudicate round 2: fix-upheld. At 273d5b9, hello carries sessionExpiresAt from the caller's session. WorkerConnections.hold sleeps until 7 days before expiry, then calls boardWorker.renew. Upstream's issueSession issues the successor with the caller's own subject, scopes and label, so it can't widen access. The board stores the new token, releases the old session and redials with the status left at online. A failed renewal keeps the old token, waits for the link to drop, and on redial the delay is 0, so it retries without a hot loop. I ran 'renews the board's session before it expires, and stays online throughout' and it passes. It checks the renew, the release, the redial on token-1-renewed, the stored secret and the online-only states.

[brentkelly]

Warning

Improvement (2/2): Re-pair from the picker replaces the board's working worker URL with this browser's URL

In repair mode, submitPicked passes url: minted.value.httpBaseUrl to repairWorker. That value is prepared.httpBaseUrl, the address this device uses to reach the worker (see mintWorkerPairingToken). BoardWorkers.repair then uses input.url ?? worker.url, so the browser's URL wins. A re-pair is usually needed because the session was revoked, not because the address changed. The board's recorded URL (a LAN address, say) still works, but the browser may reach the worker another way: a Tailscale hostname, localhost on the same machine, or a URL the board server can't resolve. In that case describe() fails with "can't reach", and the re-pair is refused even though the old URL would have worked. If the board can resolve the browser's URL but it leads somewhere less stable, the board's dial URL silently moves to it.

Suggested fix: In repair mode, don't send the minted httpBaseUrl, so the server keeps worker.url. Or have repair try worker.url first and fall back to the client's URL only if the old one is unreachable.

[brentkelly]

Fixed in 030c3db, on the server so it doesn't matter which client re-pairs. A minted token works at any of the worker's addresses, so repair now tries the recorded worker.url first. It moves to the client's URL only when the recorded one is unreachable, so a worker that really moved can still be re-paired from the picker. A pasted link still uses its own address.

Test: BoardWorkers.test.ts 're-pairing from the picker keeps the address the board already dials' fails with unreachable on the old code.

[brentkelly]

Adjudicate round 2: fix-upheld. In 030c3db, a minted pairing-token re-pair with a different input.url tries worker.url first. It moves to the client URL only on an unreachable error. A pasted link still uses its own address. I put the old BoardWorkers.ts back and 're-pairing from the picker keeps the address the board already dials' fails. It passes on HEAD.

[brentkelly]

Note

Nitpick (1/3): Pairing links expire after 5 minutes, not 10

The docs, and the invalidCredential copy in workerCredential.ts ("Pairing links work once and expire after 10 minutes"), say 10 minutes. Upstream's PairingGrantStore uses DEFAULT_ONE_TIME_TOKEN_TTL_MINUTES = Duration.minutes(5) for user-issued links unless --ttl is passed. A user told they have 10 minutes will see links expire early.

Suggested fix: Say 5 minutes in both places, or word it without a number ("expire after a few minutes").

[brentkelly]

Fixed in 9a61336. PairingGrantStore's DEFAULT_ONE_TIME_TOKEN_TTL_MINUTES is 5 minutes. The docs, the server's invalidCredential message and the web workerCopy body now all say 5 minutes.

[brentkelly]

Adjudicate round 2: fix-upheld. The docs at line 31, the server's invalidCredential copy and the web workerCopy all say 5 minutes now. That matches DEFAULT_ONE_TIME_TOKEN_TTL_MINUTES = Duration.minutes(5) in PairingGrantStore.

[brentkelly]

Note

Nitpick (2/3): Remove gives the session back and deletes the token before the decider has accepted the removal

remove checks the default worker, releases the session on the worker, and deletes the secret. Only then does it dispatch board.worker.remove. If the decider refuses the command, the worker stays registered with a revoked session and no stored token, so it turns unauthorized until someone re-pairs it. That can happen if a concurrent setDefault made this worker the default in between: the decider re-checks the default and refuses.

Suggested fix: Read the worker's rpc with connections.get first, dispatch the remove, then release and delete the secret. The entry stays in connections until syncConnections runs, so the same connection can still carry release.

[brentkelly]

Fixed in 899d9c4. remove now captures the live rpc, dispatches board.worker.remove first, then releases the session over the connection it still holds (until the sync), and only then deletes the token. A refused removal leaves the session and the token intact.

Test: the remove test in BoardWorkers.test.ts now checks that the worker is no longer registered when the release arrives, through a new onRelease hook on the fake worker. It fails on the old code.

[brentkelly]

Adjudicate round 2: fix-upheld. remove now captures the connection as an Option, dispatches board.worker.remove, and only then releases over the held connection and deletes the secret, before syncConnections. A refused dispatch fails before either side effect. I put the old BoardWorkers.ts back and the updated remove test, which checks through onRelease that the record is gone first, fails. It passes on HEAD.

[brentkelly]

Note

Nitpick (3/3): Two concurrent Set up calls for the same project and worker both start a clone

The "already cloning" guard reads setups at line 621. The cloning entry is only written at line 685, after the startProjectClone / retryProjectClone round trip. If two clients (or a double click that gets past the disabled button) call this together, both pass the guard and both start a clone into the same destinationPath. The second fails as folder-exists, and its result overwrites the first setup's entry, while the first clone's follower is replaced in FiberMap.

Suggested fix: Claim the key before going over the network: write a cloning placeholder with SubscriptionRef.modify that returns the existing entry if one is already cloning. Or serialize per key with a small semaphore.

[brentkelly]

Fixed in 899d9c4. setupProject claims the key with SubscriptionRef.modify before any network call: it reads the previous entry and writes the cloning entry atomically, and a concurrent second call returns the in-progress setup. Test: BoardWorkers.test.ts 'two Set up calls at once for the same project and worker start one clone' holds the worker's clone start on a Deferred. It fails on the old code with two starts.

[brentkelly]

Adjudicate round 2: fix-upheld. setupProject now claims the key with SubscriptionRef.modify before any RPC, and a second caller sees cloning and joins. Both the failure and success paths overwrite the claimed entry through putSetup, so nothing gets stuck in cloning. I put the old BoardWorkers.ts back and 'two Set up calls at once for the same project and worker start one clone' fails. It passes on HEAD.

[brentkelly]

High-level summary — round 3 (re-review at 273d5b9)

This PR lays the distributed-board foundation. It rewrites the overview (D19–D22), adds an event-sourced worker registry, the central server's own supervised connection to each worker, the worker-side boardWorker.* RPCs, project ↔ worker checkouts with Set up, and the Workers and Project checkouts settings. The round-2 fixes are in and do what they claim. The re-pair URL preference, Remove's dispatch-then-release order and the 5-minute copy are correct. The renewal design is sound: Effect splits long sleeps past the 2³¹ ms timer cap, so the roughly 23-day wait is safe. Two of the new pieces have edges, though. The new boardWorker.renew RPC is open to every read-scoped session, so a browser cookie or DPoP session can swap itself for an unbound bearer token. Renewal is tried once per link and never retried. The atomic Set up claim can be left behind as a permanent cloning entry. That makes 3 improvements and 1 nitpick this round, and no criticals. No human comments are open on the PR.

Status of round-2 findings

#	Prior finding	Status
I1	🟡 The board's session on every worker expires after 30 days and is never renewed	⚠️ partially addressed. Renewal works on the happy path. A failed renewal is never retried on a live link (Improvement 2/3), and the new RPC is too broadly reachable (Improvement 1/3).
I2	🟡 Re-pair from the picker replaces the board's working worker URL with this browser's URL	✅ closed (a narrow edge is left as Nitpick 1/1)
N1	🟢 Pairing links expire after 5 minutes, not 10	✅ closed
N2	🟢 Remove gives the session back before the decider accepts the removal	✅ closed
N3	🟢 Two concurrent Set up calls both start a clone	✅ closed. The claim adds a stuck-entry path (Improvement 3/3).

[brentkelly]

Warning

Improvement (1/3): boardWorker.renew lets any read-scoped session swap itself for a fresh bearer token, including browser cookie and DPoP sessions

The handler only checks AuthOrchestrationReadScope, then calls issueSession with the caller's subject and scopes. issueSession always mints a bearer-access-token session with deviceType: "bot" and the default 30-day TTL (EnvironmentAuth.ts:1107). Every T3o server answers boardWorker.*, so any client on any T3o server can call it, not only a board. Two consequences:

• A browser session lives in an httpOnly cookie, which page script cannot read. Over the WebSocket, page script can now call renew and get the same scopes back as a plain bearer token in JS. For the owner's browser that is administrative scopes. A cookie-only credential becomes one that can be carried away.
• A DPoP-bound session (proofKeyThumbprint) gets a successor with no proof-of-possession binding.

The comment says a session "can only extend itself, never widen itself". That holds for scopes, but the binding and the transport are both weakened. Only the board's own bearer session needs this RPC.

Suggested fix: Refuse renew unless currentSession.method === "bearer-access-token" and there is no proofKeyThumbprint. Better still, also require the session to be a board session, for example by its T3o board ( label or a dedicated subject set at exchange time. Add a workerRpc.test.ts case where a cookie session is refused.

[brentkelly]

Fixed in be2ba45. boardWorker.renew now refuses unless the caller is a plain bearer-access-token session with no proofKeyThumbprint and a board label (isBoardSessionLabel, the counterpart of boardSessionLabel in workerCredential.ts). A browser cookie, a DPoP-bound session or any other bearer session gets a BoardWorkerRequestError and nothing is issued. Test: workerRpc.test.ts "renew refuses a browser cookie, a DPoP-bound or a non-board session" (fails on the old code, which issued all three).

[brentkelly]

Adjudication (round 3): fix-upheld. be2ba45 refuses renew unless currentSession.method === "bearer-access-token", there's no proofKeyThumbprint, and the label passes isBoardSessionLabel. The successor reuses the same label. workerRpc.test.ts 'renew refuses a browser cookie, a DPoP-bound or a non-board session' fails on 273d5b9 and passes at 63dd84c.

[brentkelly]

Warning

Improvement (2/3): A failed renewal is never retried while the link stays up, so the session can still expire

hold sleeps until 7 days before expiry and tries renewSession once. If that fails (a transient RPC error, the worker busy, or secrets.set failing), it falls through to yield* link.closed and does nothing until the socket drops. A stable link can last the whole remaining 7 days. Upstream does not close a socket when its session expires (only revocation closes it, SessionStore.ts:227), so the board stays green until the next drop. Then the redial gets a 401 and the worker goes terminal unauthorized ("Access revoked"), which needs a manual re-pair. That is the outcome round-2 I1 was meant to prevent. The current test covers only the success path.

Also, when renew succeeds but secrets.set fails, catchCause returns false and the new session is left behind on the worker.

Suggested fix: Retry inside hold: on failure, race link.closed against a backoff sleep (minutes to an hour, capped before expiresAt) and try again. If the successor's token can't be stored, release it. Add a test where the first renew fails and the second succeeds on the same link.

[brentkelly]

Fixed in f617008. hold now loops: after a failed renewal it waits sessionRenewalRetryDelay (1 minute, doubling, capped at an hour, well inside the 7-day lead) and tries again, for as long as the link stays up. If renew succeeds but secrets.set fails, the successor is given back over a short link of its own (release revokes the session that calls it) and the old token stays in use. Tests: WorkerConnections.test.ts "retries a failed renewal while the link stays up" and "gives back a successor it couldn't store, and keeps the old token". Both fail on the old code.

[brentkelly]

Adjudication (round 3): fix-upheld. In f617008, hold now loops while the link is up and retries with sessionRenewalRetryDelay (1 min, doubling, capped at 60 min, so it stays well inside the 7-day lead). When secrets.set fails, the successor is released over its own link and the old token is kept. Both new WorkerConnections.test.ts tests fail on the old code and pass now.

[brentkelly]

Warning

Improvement (3/3): Set up's claimed cloning entry is never cleared if the call is interrupted or dies before the worker answers

The round-2 N3 fix writes a cloning entry with workerProjectId: null before rpc.startProjectClone / retryProjectClone. Only the Effect.result failure branch and the success branch replace it. Some exits skip both:

• The handler fiber is interrupted, for example when the client disconnects or the request is cancelled during the round trip.
• A defect escapes. Effect.result captures typed failures only.
• The worker call never returns. call in WorkerTransport.ts has no timeout, so a half-open worker socket hangs it.

In all three, no follower is started, so nothing ever moves the entry on. The matrix shows a spinner that never stops (ProjectCheckoutsSection.tsx cloning has no Dismiss), and every later Set up for that pair returns the stale entry at line 666 instead of cloning. Only removing the worker or restarting the board server clears it. Before this fix the same interruption left nothing behind.

Suggested fix: Guard the section from the claim until putSetup with Effect.onExit, or Effect.onInterrupt plus catchDefect. If the claimed entry is still the one this call wrote, drop it or mark it failed. Give the start/retry call a timeout. Add a test that interrupts the Set up mid-RPC and checks that a second Set up then starts a clone.

[brentkelly]

Fixed in the third commit of this round (fix(board): an interrupted or unanswered Set up…). Everything after the claim runs under Effect.onExit(abandonClaim). If the entry is still the exact claimed object when the call ends (interrupt, defect, anything), it becomes failed with kind worker-offline, so Dismiss works and the next Set up starts fresh. The start or retry RPC now has a 30-second SETUP_START_TIMEOUT, after which it fails like any other start failure. Tests: BoardWorkers.test.ts "a Set up interrupted before the worker answers leaves no stuck clone" and "a worker that never accepts the clone fails Set up instead of hanging". Both fail on the old code.

[brentkelly]

Adjudication (round 3): fix-upheld. In 7eb3c71, Effect.onExit(abandonClaim) marks the claim failed/worker-offline only while map.get(key) === setup still holds. So the success path (cloning with a workerProjectId) and the typed-failure path (failedSetup) are left alone, and an interruption or defect clears the spinner. The start/retry RPC now has a 30s timeout. Both new BoardWorkers tests hang until timeout on the old code and pass now.

[brentkelly]

Note

Nitpick (1/1): Re-pair falls back to the client's URL only on unreachable, not when a different server now answers at the recorded address

A worker that moved on a DHCP LAN may leave its old address to another T3 server. locate(worker.url) then succeeds against that server, and the environmentId check that follows fails with different-server ("this link is from "). That is misleading: the pasted token and input.url are correct, and the fallback would have worked. The token isn't spent, because describe runs before the exchange, so the user only sees a confusing error.

Suggested fix: Also fall back to input.url when the recorded URL's descriptor names a different environment. For example, try worker.url, check the environmentId, and on a mismatch or unreachable move to input.url.

[brentkelly]

Fixed in 63dd84c. Re-pair now treats a different environmentId at the recorded address the same as unreachable and moves to the client-supplied URL. If that URL is also a different server, the existing different-server error still applies. Test: BoardWorkers.test.ts "re-pairing from the picker moves when another server now answers at the old address" (fails on the old code).

[brentkelly]

Adjudication (round 3): fix-upheld. In 63dd84c, filterOrFail on environmentId === worker.id turns a different server at the recorded URL into unreachable, and the existing catchIf then falls back to input.url. The new re-pair test fails on the old code and passes now.

[brentkelly]

High-level summary — round 4 (re-review at 63dd84c)

This PR lays the distributed-board foundation: the rewritten overview (D19–D22), the worker registry and its event-sourced default, the central server's held connection to each worker with session renewal, the worker-side boardWorker.* RPCs, project ↔ worker mapping with Set up on worker, and the Workers and Project checkouts settings. All four round-3 fixes hold up against the code: renew is gated, failed renewals are retried while the link stays up, a Set up claim can no longer outlive its call, and re-pair moves to the client's address when a different server answers at the old one. The worker test suite passes (7 files, 60 tests). I found nothing blocking this round. I left two nitpicks on edges of the round-3 fixes: a clone the board abandons can keep running on the worker untracked, and the label half of the renew gate is chosen by the client.

Status of round-3 findings

#	Prior finding	Status
I1	🟡 boardWorker.renew lets any read-scoped session swap itself for a fresh bearer token	✅ closed. Renew refuses a cookie session, a DPoP-bound session and an unlabelled session; see N2 below on the label check
I2	🟡 A failed renewal is never retried while the link stays up	✅ closed. Retries back off from 1 minute to an hourly cap, and a successor that couldn't be stored is released
I3	🟡 Set up's claimed cloning entry is never cleared on interrupt/defect/hang	✅ closed. onExit(abandonClaim) plus a 30-second start timeout; see N1 below
N1	🟢 Re-pair falls back only on unreachable, not on a different server at the recorded address	✅ closed. An environmentId mismatch now counts as unreachable

[brentkelly]

Note

Nitpick (1/2): A Set up abandoned by timeout or client disconnect can leave a clone running on the worker that the board doesn't track

The start runs on the client's request fiber. If the client disconnects, or the 30-second timeout fires after the worker has already accepted startProjectClone, the board records the setup as failed with a null workerProjectId. The clone keeps running on the worker. The next Set up then starts afresh into the same folder and fails with folder-exists, so the user has to recover by hand from a clone that may have succeeded. The window is small, because starting only checks the folder and spawns git, so this is not blocking.

Suggested fix: Run the start-and-follow on a board-owned fiber (the followers FiberMap already exists) so a client going away doesn't cancel it. Or, on abandon, keep the generated projectId in the failed entry so the next Set up can try retryProjectClone or look the clone up instead of colliding with its folder.

[brentkelly]

Fixed in b4a015f. I took the second option, keeping the id, because it fits how Set up already retries in place. The worker project id is now chosen before the RPC and stays on the setup through any failure: a timeout, an interrupted call, or a refused start. The next Set up for the same folder reads the worker's clone list first. If the clone is still running or already done, Set up follows it again. If the worker has it as failed, Set up retries it in place. If the worker has forgotten it, Set up starts it afresh under the same id. worker-offline failures can now reuse the id too. folder-exists is still excluded, because a retry discards the destination. New test: a Set up abandoned after the worker accepted follows that clone next time. It fails on 63dd84c, where a second clone start went into the same folder, and passes now. I updated a retry the worker no longer tracks… to expect the same id on the fresh start.

[brentkelly]

Note

Nitpick (2/2): The board-label half of the renew gate is chosen by the client

isBoardSessionLabel checks client.label, and the client sets that label itself as client_label on /oauth/token (auth/http.ts:390). Any bearer client that names itself T3o board (…) can renew its session forever and get around the 30-day TTL. This can't widen access: scopes are copied, and cookie and DPoP sessions are still refused by the method and thumbprint checks. The label check reads like an authorization gate, though, when it is only a convention.

Suggested fix: Say in the comment that the label is advisory. Or tighten the gate with something the client can't pick freely, such as also requiring the bot device type the board exchanges with. The cleaner option is a server-side marker recorded when the board's session is issued.

[brentkelly]

Fixed in b4a015f by correcting the comment. It now says the label is advisory: any plain bearer client can name itself that way on /oauth/token and extend its TTL, but only with the subject and scopes it already holds. I didn't add a check on device type because deviceType is also client-presented (auth/utils.ts:194), and a real server-side marker would need a new column on auth sessions, which is an upstream-owned table. That's more than this gap justifies.

[brentkelly]

Repo change, made alongside this PR: every GitHub Actions workflow in the fork is now disabled.

This PR's auto-merge was held for about 8 hours. Its Mobile Fingerprint Check job asks for a blacksmith-8vcpu-ubuntu-2404 runner, and only upstream has those, so the job stayed queued and never ran. The board counts a pending check as a reason to wait. It retried until it ran out of attempts at 14:15, then held the merge with "Its checks have not finished." #182, #183 and #184 had the same stuck check and were merged by hand.

Workflows from the earliest upstream sync were already disabled. Ones that came in with later syncs were still enabled. These are now disabled in Settings → Actions, so no workflow files changed and upstream syncs won't conflict:

• Forward to Cursor hygiene
• Desktop macOS Preview
• Desktop macOS Preview Publish
• Issue Labels
• Mobile Android APK
• Mobile Fingerprint Check
• Publish AUR package
• Release desktop build
• Web Preview
• Windows Tests

Two queued runs were cancelled: this PR's fingerprint check (39602449216) and a Mobile Android APK run on t3o (39564744884).

Dependency Graph is still listed as active. It's GitHub's built-in dependency submission, and the API can't disable it. It runs on GitHub's own runners and posts no PR checks, so it can't hold a merge. Turn it off under Settings → Code security if you want it gone.

To turn a workflow back on: gh workflow enable <name>.