Skip to content

feat: make assistant writes durable — execution ledger, transactional idempotency and an outbox - #16

Merged
alexmartinezm merged 14 commits into
mainfrom
durable-agent-actions
Aug 11, 2026
Merged

feat: make assistant writes durable — execution ledger, transactional idempotency and an outbox#16
alexmartinezm merged 14 commits into
mainfrom
durable-agent-actions

Conversation

@alexmartinezm

@alexmartinezm alexmartinezm commented Aug 9, 2026

Copy link
Copy Markdown
Owner

What changes

The write gate was already a strong authorization boundary. What it had no answer for was everything
after a decision and before the effect is known. This splits one status into three linked facts —
decision, execution, external effect — and closes the four windows that made the old
design read as safer than it was. ADR 009 records the
reasoning and what it deliberately does not claim.

Six slices, one commit each:

  1. Execution ledger. ActionExecution is the durable record of an attempt: a stable identity
    that doubles as its idempotency key, an attempt count, the settled result or a bounded sanitized
    error, and a status including the one a response cannot give you — Unknown, for a request that
    crossed a non-transactional boundary and lost the answer. Unknown is reachable, is not terminal,
    and is never rendered as Failed. AuditDecision narrows to authorization alone.
  2. Atomic resolution. Approve/reject/expire are now a conditional UPDATE whose affected-row
    count is the decision, with the claim, the audit row and the execution insert in one
    transaction. PendingAction.TryApprove is gone: an in-memory status check was the concurrency bug,
    not the guard against it. Claiming an attempt gets the same treatment one level down.
  3. Transactional idempotency. The business change and the receipt that replays it now commit
    together — the filter owns the transaction and the handler's own SaveChanges enlists in it.
    Expiry became a deletion rather than a filter on reads, so a key can no longer be simultaneously
    "too old to replay" and "already taken".
  4. Approval preconditions. Invoice.Revision is an EF concurrency token; the proposal captures
    it, the approval sends it as If-Match, and a mismatch fails closed. ArgsJson moved from
    jsonb to jsonjsonb normalises what it stores, so the bytes coming back were not the bytes
    hashed, which quietly broke the command fingerprint.
  5. External delivery. send_invoice writes an InvoiceDelivery and an OutboxMessage inside the
    local transaction; a worker leases rows with FOR UPDATE SKIP LOCKED and calls the provider
    outside the lock. An accepted-then-lost response reconciles to Succeeded with exactly one
    provider receipt.
  6. Product proof. GET /api/action-executions/{id}, the outcome payload reporting decision and
    execution separately, and an approval card that shows execution state instead of jumping to
    "Done". Every closing line is still the server's — no extra model call narrates an outcome.

Two contract changes worth not missing in review

  • Idempotency-Key is now required on every invoice write. Missing is 400; the same key with
    different content is 422 rather than a plausible-looking replay of the wrong result.
  • POST /api/invoices/{number}/send now answers 202, with a delivery record. An invoice
    reading Sent no longer doubles as a claim that the customer received it.

Also: the internal milestone shorthand (F1F5) is gone from code, docs and commit messages — it
only resolved inside a tracker. Two sentences in the ui-design skill described the approval card
and usage page as not existing yet; they do, so they are in the present tense now.

How to verify

dotnet format --verify-no-changes
dotnet build
dotnet test                            # 137 passing, real PostgreSQL

dotnet test --filter FullyQualifiedName~DurableActionTests            # 20 concurrent approvals
dotnet test --filter FullyQualifiedName~TransactionalIdempotencyTests # crash boundaries and replay
dotnet test --filter FullyQualifiedName~ApprovalPreconditionTests     # ETag / If-Match
dotnet test --filter FullyQualifiedName~OutboxDeliveryTests           # outbox and reconciler

npm run lint --prefix src/Web && npm run format:check --prefix src/Web
npm run build --prefix src/Web
jq empty policies.json

The durability suites prove their races with real concurrent requests and by stopping the process at
named fault checkpoints, never with sleeps. The headline one is twenty simultaneous approvals
producing one resolution, one execution, one attempt and one paid invoice.

Checks

  • Tests and quality gates green locally
  • Security invariants hold: no write reaches the DB without a policy allow or a human
    approval recorded in AuditEvent
  • Docs updated — a new ADR if a decision changed, .agent/commands.md if commands changed
  • If this touches prompts/system.md, policies.json or the assistant slice: the evals ran on
    a branch of this repository, and the run is linked above

On the evals box: I could not run a real-model eval where I built this (no provider key), so I
opened this PR with the box unticked rather than claim otherwise. CI has since run them for real —
run #40, gpt-4.1-mini-2025-04-14,
41/41 passing in 2m39s, authorization gate run=true rather than the fork/no-key skip — so the
box is now ticked on that evidence. prompts/system.md and policies.json are untouched and all 36
cases are kept; the harness gained execution/delivery assertions so the injection cases can now
assert that no execution row was created, which is a stronger fact than the absence of an audit
auto.

All four checks are green: repo-checks, backend (137 tests), frontend, evals.

PendingActionStatus.Approved was being read as "the work happened". It is
not: it is a decision, recorded before anything was tried. The two facts
diverge exactly when it matters — a crash, a timeout, an API refusal after a
human said yes — and with one status there was nowhere to write the
difference down.

ActionExecution is that second row. It carries a stable identity that doubles
as the idempotency key for every attempt it makes, an attempt count, the
settled result or a bounded sanitized error, and a status that includes the
one a response cannot tell you: Unknown, for a request that crossed a
non-transactional boundary and lost the answer. Unknown is reachable, is not
terminal, and must never be rendered as Failed — "nothing happened" is the
dangerous direction to be wrong in. A unique filtered index on
PendingActionId is the database's guarantee that a confirmed action has at
most one execution.

PendingAction gains the fingerprint of what was authorized: CommandHash over
the tool name, the exact stored argument bytes and the captured resource
revision, plus the revision itself and a resolution reason. The hash reads
the bytes rather than a canonicalized form because the server's serializer is
the only producer of them; the note in CommandHash says what to do the day
that stops being true.

AuditDecision keeps its four values and narrows its meaning to authorization
alone. Whether the command then worked lives on the execution.

The migration drops the compatibility default on CommandHash so a future
insert cannot quietly produce an action with no fingerprint, and closes any
proposal left open by the previous build as deployment_upgrade. They live
five minutes; replaying a command with nothing to bind it to is not a
migration step.

ADR 009 records the three-facts split, the constraint it puts on handlers,
and the language limit: no document here claims exactly-once delivery
without naming the provider capability that makes it so.
…ming

TryApprove loaded the row, checked Status in memory and saved. Two requests
could both read Pending before either save became visible, so "single use"
was a property of how fast the two happened to run rather than a constraint.
The fix is the one the database was always going to have to make: approve,
reject and expire are now a conditional UPDATE whose affected-row count is
the decision, issued by ActionResolver, which is the only place a
PendingAction changes state. The claim, the authorization audit and the
execution insert are one transaction; half of that committed is worse than
none of it.

Claiming an *attempt* is the same problem one level down, so it gets the same
answer. Twenty tabs all arrive at the same execution; ActionExecutor takes
the right to make a request with a conditional update, and the nineteen that
lose are handed the state instead. Without that, two of them would put the
same idempotency key on the wire simultaneously, which races the receipt
rather than using it. ActionExecution has no Start method for this reason —
a cross-request claim is not an entity transition, and leaving one there
would be a door around the constraint.

Approving twice is now a replay rather than a 409. A client that lost the
first response is not doing anything wrong, and the second call returns the
same execution id, the same message and no second write. The transcript gets
its closing line once, and only once the outcome is actually known: an
assistant message saying "Done" beside an unconfirmed execution is exactly
the claim this work exists to stop making.

An auto-allowed write gets an execution too, minted with its audit row
*before* the request goes out. The old code used a throwaway key per attempt
and audited afterwards, so a process that stopped mid-call left nothing
saying a write had been tried and no key a retry could reuse.

The outcome payload reports decision and execution separately, and answers
202 while the answer is still owed. SelfApiClient now surfaces the status
code alongside the payload, because classification needs the number: 2xx
settles, a 4xx proves the effect did not happen, and a 5xx or a dropped
connection proves neither and becomes Unknown.

WriteGateTests loses "cannot be approved twice" to DurableActionTests, which
proves the same guarantee properly — twenty real concurrent approvals, one
resolution, one execution, one attempt, one paid invoice — and the
approved-then-refused case now expects Confirmed plus a failed execution.
The old filter recorded its receipt in a second SaveChanges, after the
handler had already committed. A crash between the two left an effect that
nothing could replay, and the next retry did the work again — which for
mark_invoice_paid means settling an invoice twice. The filter now opens the
transaction itself and runs the handler inside it; the handler's own
SaveChangesAsync calls enlist because it is the same scoped DbContext, so
there is no window between the two facts to crash in. Both crash boundaries
are tested: before the commit, everything rolls back and the key is free;
after it, the retry replays and no second invoice exists.

The key is now required rather than opt-in, and bound to a fingerprint of
method, path, body and If-Match. Same key and same request replays; same key
and different content is 422 rather than a plausible-looking reply to a
question nobody asked; a concurrent twin waits on the row and then replays,
or gets 409 request_in_progress if the wait exceeds a bounded lock timeout.
Credentials are not part of the fingerprint and are never stored.

Computing that fingerprint needs the request body, and an endpoint filter
runs after model binding has consumed it — so buffering is enabled upstream
for requests that carry the header, and the filter refuses loudly if it
cannot rewind. That failure mode is not hypothetical: with the read
short-circuiting on an unseekable stream, every POST hashed identically and
the mismatch check silently became a check on the URL alone. The test caught
it, which is the argument for having written the test that way.

Expiry stops being a filter on reads and becomes a deletion. The old rule
ignored rows older than 24 hours while the unique index reserved them for
ever, so a key could be simultaneously too old to replay and already taken.
Receipts now carry an explicit ExpiresAt, a background service purges them in
bounded batches, and the migration backfills legacy rows with the window the
old code implied.

Because a committed receipt is now proof the effect happened, it is
authoritative enough to settle an execution whose answer was lost:
ActionExecutor checks for one before it considers sending anything again.
That is what makes Unknown a temporary state rather than a permanent shrug.

Also adds the named fault checkpoints from the spec as a seam that is a no-op
in production and is not reachable from configuration, a chat tool or an
endpoint. Races are proved by stopping the process at a named boundary, not
by sleeping and hoping.
Frozen arguments were never quite enough. "Mark 2026-0041 as paid" agreed to
five minutes ago is not the same instruction if somebody cancelled the
invoice in between — the words match, the situation does not, and the old
approval would have executed against state the approver never saw.

Invoice gains an application-managed Revision, bumped by every transition and
mapped as an EF concurrency token, so two writers racing on one invoice raise
rather than quietly last-writer-wins. It surfaces as an ETag on every invoice
response; write endpoints honour If-Match and answer 412 resource_changed on
a mismatch. The server does not refresh and retry — failing closed is the
whole point.

The proposal captures the target's revision, the command hash covers it, and
the approval sends it. A stale approval settles the execution as Failed with
resource_changed, and gets its own sentence rather than the generic refusal:
"the API said no" would send the user looking for a permission problem when
what actually happened is that the thing they were shown moved on.
create_draft_invoice has no existing target, so it carries no precondition.

Three columns move from jsonb to json, and that is not cosmetic. jsonb
normalises what it stores — key order, whitespace, duplicate keys — so a
pending action's frozen arguments came back re-ordered and no longer matched
the hash taken over them at proposal time. The precondition test is what
surfaced it. The same argument applies to a replayed idempotency response and
a stored execution result: "the first answer" ought to be the first answer,
not an equivalent one. Nothing queries inside any of those columns, so jsonb
was buying nothing in exchange.
send_invoice is the only place this system touches something it cannot roll
back, which makes it the right place to demonstrate what that costs.

The send endpoint no longer pretends. It moves the invoice out of draft,
writes an InvoiceDelivery and an outbox row in the same transaction as the
status change and the idempotency receipt, and answers 202 — the ledger
change is done, the email is not. Nothing calls the provider from inside that
transaction; doing so would put somebody else's network inside our lock. The
invoice detail reports delivery separately, so Sent never has to double as a
claim that the customer received anything, and an invoice can honestly read
Sent with a failed delivery.

The worker leases rows with FOR UPDATE SKIP LOCKED and releases the row lock
before calling the provider — a lease in the row, not a database lock across
a network. The three outcomes are deliberately asymmetric, and the asymmetry
is the feature. Accepted settles. A deterministic rejection settles as failed
and completes the outbox row, because retrying would be refused identically.
An ambiguous answer settles as nothing: the row is parked and the reconciler
asks the provider what it actually has. Retrying an ambiguous send is how one
invoice gets emailed twice.

The demo provider holds its receipts in memory rather than in our PostgreSQL,
and that is the honest shape — putting them in our database would quietly
make the boundary transactional and the demonstration meaningless. It honours
the stable key, which is the capability that makes reconciliation produce one
message instead of two, and the tests assert both halves: how many times the
provider was asked, and how many messages that produced.

An execution whose write handed off to a delivery stays Executing until the
delivery settles. The classification keys on 202 rather than on the presence
of a delivery field, because every invoice response mentions its delivery and
reading mark-paid's copy as a hand-off would leave executions waiting on
somebody else's email. Whoever settles the execution now owes the
conversation its closing line, so the worker writes it — minutes after the
person clicked approve, and only once the outcome is actually known.

The named checkpoints make all of this testable: the provider accepts and
loses the answer, a worker dies holding a lease, two workers race one queue.
Every one of those is a deterministic exception at a named boundary rather
than a sleep and a hope.
Approving stopped being the end of the story when a write could hand off to a
delivery, so the card stops behaving as though it were. It goes to "Approved ·
executing", or "Outcome unknown · reconciling", and only then to the server's
closing sentence — polled from a new GET /api/action-executions/{id} for a
bounded window, whose visibility follows the proposal's rather than inventing
a second rule about who may look at one decision.

The in-flight dot is neutral and reuses the tool chip's animate-pulse rather
than adding a fourth motion effect, and the two unfinished states are told
apart by their heading, because anything distinguished by animation alone
disappears under prefers-reduced-motion.

The eval suite gets executions_created. It is a stronger fact than
writes_executed for the injection cases: an execution row exists the moment a
write is attempted, before the API has had a chance to refuse it, so
"executions_created: 0" asserts the assistant never tried. "Refused by the
endpoint" and "never tried" look identical in the invoice table and are not
the same result. All six injection cases now assert it; no case was added or
removed, so the documented counts still hold.

Telemetry gains the four spans and the counters worth alerting on: executions
started and settled by tool and decision, approval-to-effect timing measured
from the decision rather than the request, idempotency replays and fingerprint
mismatches, and the three queue gauges — depth, oldest waiting row, unconfirmed
deliveries — published by the worker rather than queried on scrape, because a
metrics endpoint should not put load on the database it measures.

Documentation: ADR 009 was written up front; architecture.md now carries the
three-facts vocabulary, the idempotency contract table and a guarantees table
that states the limits as plainly as the guarantees. The README gets the
five-minute failure demo, which is the part worth showing — the gate working
is easy, and what this feature adds is what happens when something breaks
halfway.

Verified against a running instance, not only the suite: missing key 400,
replay 201, mismatched body 422, stale If-Match 412, send 202 with a queued
delivery, and the hosted outbox worker taking it to delivered with one
provider receipt and one attempt.
The tracker's shorthand for this piece of work is not something a reader of
this repository can resolve. The comments and docs now name what they are
talking about — durable actions, the transactional write pipeline, ADR 009 —
which is what was meant in each case anyway.

The roadmap entry loses its number rather than keeping one that points at
nothing outside the tracker.
The roadmap read F1 through F4 and then an unlabelled fifth entry, which was
the visible half of a shorthand that only means something inside a tracker.
Every remaining label is now the thing it referred to: the write gate, the
evals job, cost accounting. Same claims, resolvable by a reader who has never
seen the plan.

Two sentences in the design skill were also stale rather than just labelled —
they described the approval card and the usage page as things that did not
exist yet. They do, so they are written in the present tense now.
The feature shipped its guarantees for the run where nothing goes wrong. On
the recovery paths several of them were not there, and a review found nine
places where that shows.

The largest was that Executing was reachable and unrecoverable. An attempt
was claimed with no lease, and the claim would only accept Pending or
Unknown, so a process that died between claiming and sending left a row
nothing could ever move again — no request, no worker, no person. Attempts
now carry a lease; when it lapses the execution is reclaimable under the same
idempotency key, which is what makes a resume a replay rather than a second
write. A background pass settles what receipts and deliveries can settle and
releases the rest. It never re-executes: it holds no bearer token and must
not acquire one, so an execution with no evidence stays unsettled and a
person decides.

Receipt replay read any 2xx as success while the live path knew that 202 with
a delivery id means the outbox owns the outcome. A send whose response was
lost therefore came back from its receipt as Succeeded — "Done", with the
email still queued and no way to correct it, because a settled execution is
terminal. There is one classifier now, used by both.

The outbox deferred every ambiguous send straight back to Pending, so the
passage of time turned "we do not know" into a resend. That was only safe
because the demo provider happens to deduplicate. Ambiguous rows are parked
in their own status, out of the dispatcher's reach, and what may happen next
is read from the provider's declared capabilities: ask when it can answer,
resend only when it deduplicates, otherwise wait for a person. The demo
provider's capabilities are settable and its behaviour follows them, so the
branch that waits is now testable rather than theoretical.

The rest are smaller and each one is a wrong answer rather than a missing
one:

- A delivery was linked to an execution by id alone. An execution id is
  visible to whoever proposed the action even when somebody else approved it,
  so one user could hand another user's execution an unrelated delivery — and
  with it a closing line on somebody else's conversation. The match now
  requires the caller to own the execution, for the right tool, in a state
  that is waiting for this request. The stronger version carries the identity
  in the internal call rather than inferring it from a header; that is a
  change to how SelfApiClient presents itself and is noted where the match
  is, not made here.
- The request fingerprint was truncated at 256 KiB after copying the whole
  body. Two bodies agreeing up to the cap hashed identically, and the second
  was replayed the first one's answer. The read is bounded during the copy
  and an oversized body is refused with 413.
- Any failed execution was narrated "Nothing changed", including a delivery
  the provider refused — where the invoice is Sent and the customer owes
  money. That case has its own sentence, and the execution carries the
  server's words to the client instead of the card rebuilding them from the
  status.
- Rejecting an action wrote no audit row at all: the one decision a person
  makes explicitly was the one the ledger did not record. It writes one, in
  the same transaction as the resolution, and so does a policy refusal.
- The command fingerprint was stored and copied onto the execution, and
  nothing ever recomputed it. It is verified before the compare-and-set, so
  an edited proposal is refused rather than executed under an approval of a
  different sentence.
- The lock timeout the filter sets covers the handler's statements too, and a
  timeout raised there escaped as a 500 — telling a caller its request failed
  when it had not been attempted. It answers 409 like the claim does.

Eleven tests, each starting from a state the system can actually reach: a
crash between claim and send, a lost send response, a provider that can
neither deduplicate nor be asked, a lookup that answers "never received", a
forged idempotency key, an edited proposal, a handler that cannot get its
lock. The question every one of them asks is how many times the money moved
and how many emails the customer got.

@alexmartinezm alexmartinezm left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sign-off on e10c574. GitHub will not take a formal approval here — it is the author's own account — so this is the record instead. And it is a sign-off, not an independent review: the same hands wrote the branch.

What it rests on:

  • All nine findings from the external audit are addressed. Eight were reproduced against the checkout. The ninth (F9) was misdescribed — the filter already caught SQLSTATE 55P03 at both of its own wait sites — but it pointed at a real narrower gap: the lock_timeout covers the handler's statements too, and one raised there escaped the generic catch as a 500. That is what got fixed, and it now answers 409 request_in_progress like the claim does.
  • One thing was deliberately not done. The audit's deeper remedy for the delivery↔execution link — carry the identity in the internal call rather than infer it from a header — is a redesign of how SelfApiClient presents itself. The tightened match it gives as the minimum is in (Id + UserId + ToolName + non-settled state), and the limitation is written down where the match lives rather than left for the next reader to rediscover.
  • 148 tests against real PostgreSQL, eleven of them new. Each starts from a state the system can actually reach: a crash between claiming an attempt and sending it, a send whose response was lost after the local commit, a provider that can neither deduplicate nor be asked, a lookup that answers "never received", another user's execution id used as an idempotency key, a proposal edited after it was shown, a handler that cannot get its lock. The question each one asks is how many times the money moved and how many emails the customer got.
  • repo-checks, backend and frontend are green on this commit. evals is still running; if it goes red I will fix it and push rather than leave this standing on an unfinished check.

The first pass gave every stuck state somewhere to go — a lease to reclaim, a
delay to wait out, a background pass to settle it. A second, independent
review found that "somewhere to go" was not yet "safe to get there": several
of those paths could be walked by two callers at once, and nothing stopped
the second from landing on top of the first.

The idempotency filter's own 409 request_in_progress — a twin still holding
the key — was classified the same as a deterministic refusal. A live retry
that lost this race settled its execution as Failed, for an effect that had
not actually failed; the invoice stayed correct, but the ledger lied about
why. It now reads the same way a lost transport answer does: Unknown, worth
asking about again, not a verdict. The reclaim guard this implies has one
carve-out: a reconciler pass that already checked for a receipt and a
delivery and found neither is itself the "somebody looked" decision the
guard exists to wait for, so it releases the attempt immediately rather than
making the next caller sit through a delay nothing further will resolve.

A reconcile pass that could not settle an Unknown execution left it exactly
as "due" as it started, so the same unresolvable row came back first on
every single pass, ahead of anything a batch limit was supposed to make room
for. It now defers its own next look, the same discipline the outbox side
already had.

The delivery reconciler could ask the provider before the delay it had just
parked a row for had elapsed — the dispatcher and the reconciler run back to
back in one worker tick, and only the missing check stood between "parked
for five seconds" and "asked immediately anyway." A receipt the provider has
not finished writing would have read as "never received," and the recovery
path for that answer is a resend.

The larger one: nothing stopped two settlers from racing the same row.
A worker whose lease had lapsed but was still finishing a slow provider call,
a live retry reconciling from a receipt while the outbox settled the same
execution's delivery, two writes to one invoice close enough together that
the second's revision check failed at save time instead of at the door —
all three reached a plain SaveChanges with nothing to tell the second writer
it had already lost. Postgres's own row version is now an EF concurrency
token on the three rows this feature settles outside a compare-and-set,
and the invoice race gets the same 412 resource_changed an If-Match mismatch
already answers with, rather than an unhandled exception. Foreign keys tie
outbox to delivery, delivery to invoice and execution, and back — an
invariant the code already kept, now one the database enforces rather than
assumes.

Three smaller breaks, each a wrong answer rather than a missing one: a
malformed If-Match was read as no header at all, silently downgrading a
conditional write into an unconditional one; a replayed response dropped
Location and ETag, so a caller that created something and lost the reply
could no longer be told where it landed; and the execution DTO carried a raw
error-detail field its own doc comment said it did not.

The invoice detail panel also said nothing about delivery — an invoice could
read Sent while its email sat queued, failed or unconfirmed, and the UI had
no way to show that. It now does, styled by shape as well as colour: danger
and the aging-overdue red are close enough as hexes that the two badges
needed more than a colour to tell apart.

Nine new tests, each reproducing one of the above from a state the system
can actually reach rather than asserting on the fix's own mechanism.
A third-round audit of the durable-actions work found that the xmin tokens
added last round opened a narrower window of their own: settling a delivery
and settling what it means for the execution waiting on it were one save, so
losing a concurrency race on the execution rolled back the delivery and
outbox row too, even though the provider had already accepted the send. The
outbox would then retry a message a provider without deduplication had
already taken.

- OutboxDispatcher and DeliveryReconciler now commit the delivery/outbox
  outcome in its own save, before attempting the execution's own projection
  of it in a second, independent one. A conflict on the second is caught and
  logged; it never undoes the first.
- ExecutionReconciler gained a fallback sweep for an execution left waiting
  on a delivery with no lease of its own — the case a lost projection save
  used to leave permanently unowned — and stopped misreading a lost race as
  "no evidence": it now inspects what the row actually became before
  deciding to demote a healthy hand-off or re-decide a terminal one.
- A batch worker's DbContext no longer carries a failed save's dirty entries
  into the next row's save, in either dispatcher or reconciler.
- A 412 discovered at save time is now a durable receipt, not just a
  response: a retry under the same losing key replays the refusal instead of
  re-running the handler against whatever the resource has since become.
- The approval card's watch window used to go quiet forever on an
  unconfirmed execution; "Check again" now replays the same authorized
  approve action under the same execution.
- A rejected delivery's sentence no longer interpolates the provider's own
  error text, which had no closed vocabulary and was being recorded as an
  assistant-authored line in the conversation.

Nine new adversarial tests, mostly built on the same trick the existing
concurrent-settler test uses: preload a row in one scope before a second
writer changes it elsewhere, so the first scope's save is deterministically
stale rather than racing on timing.
Approving send_invoice is not the same as confirming delivery, and until
now that state had no screenshot: the card that shows an execution still
unresolved past its watch window, with the "Check again" resume action.

Captured from a real conversation with the model, approved for real, with
the outbox worker held back just long enough to catch the card mid-flight.
The N3 fix was in place, but its tests reached the corrected branch by a
different path than the race the audit sketched — through the fallback
sweep rather than through a reconcile pass losing to a live retry. These
two drive the exact sequences, deterministically.

A new fault checkpoint, AfterReconcilerSelectedBeforeSettle, fires between
a pass selecting a row and settling it — a no-op in production, and the
seam a test uses to move the row out from under the pass's snapshot. The
scripted fault injector gains a one-shot action that runs at a checkpoint
on a pool thread, so the interleaved writer commits before the reaching
pass proceeds.

- A reconcile pass that selected an execution while Unknown, and had it
  settled to a valid AwaitDelivery by a live retry before it could write,
  leaves it Executing rather than degrading it to attempt_abandoned.
- A closing line queued by a settle that then loses its save does not
  survive on the shared context to be inserted by the next row's save.

Both were verified to fail against the pre-fix branch and detach: the
first flips Executing → Unknown, the second inserts the orphaned line into
the wrong conversation.
@alexmartinezm
alexmartinezm merged commit 0b0418d into main Aug 11, 2026
4 checks passed
@alexmartinezm
alexmartinezm deleted the durable-agent-actions branch August 11, 2026 12:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant