Repository navigation
feat(bin): add courier spool pickup for iMessage conversations - #17
Merged
Merged
Conversation
The courier now publishes each owner message and native-poll vote as an immutable record in its own /srv/courier/inbound spool instead of running fm-inbox.sh across accounts inside its sandbox, which the VM rehearsal showed cannot work. bin/fm-courier-pickup.py is Firstmate's side of that wire: it reads the spool read-only as Firstmate, refuses malformed or duplicate records, files each message into the conversation transport exactly as the direct iMessage bridge does (same turn and request ids, transcript, attachment notices, bound poll votes), keeps its cursor in Firstmate's own state, and answers only through Firstmate-owned outbox requests: replies, numbered-question polls and ordered, never-backwards stage requests. It is off unless FM_NOTIFY_COURIER=1, so the direct bridge stays the only path until activation. Measured pickup, rename to queued wake: 0.60-0.74 s. With no cross-account writer left, retire the staged shared interface (fm_shared_interface.py and its ACL/template copying in the conversation transport and the wake library, including the permission-denied liveness and rejected-template lock plumbing that existed only for that peer) and its tests.
Match the courier's ids-only ledger and its final stage ordering wording.
… and vote reaction silence
…dline The courier wire (firstmate-voice PR 47) requires the filed or failed stage within PICKUP_DEADLINE=120 s of reading a message. A message that is not filed is now staged failed as soon as it is given up instead of after the courier's receipt for its notice, stage publication no longer waits behind an unsendable text, transport calls are bounded at 10 s, and replies are not read while a capture waits to retry, so the worst case is about 75 s (measured 75.6 s with a hung transport).
…based poll watches
…uestion associations
… question eligibility
This was referenced Oct 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
The captain chose option (a) on 2026-10-08 ("yes go with a"): the courier writes the captain's inbound iMessages only into its own inbound spool, and Firstmate picks them up from there as itself - replacing the cross-account write path that the VM rehearsal showed cannot work in the sandbox (findings C2-C4: C2, inside the courier's private PID namespace the host Firstmate session-lock owner PID is invisible, so the transport fails with "conversation owner is gone"; C3, Firstmate's state/.lock and state/voice-conversation/policy.json are written 0600 by the owner but the courier transport needed to read both; C4, Firstmate's fm_shared_interface.py fchown/setxattr ACL template copying fails with EINVAL inside the courier's one-UID user namespace, so any cross-UID metadata copy fails there). Security goal: no cross-account writes in either direction; the courier never sees or touches Firstmate, Firstmate never touches the courier's credential. Latency goal: the pickup adds under a second on top of the Linq poll. Owner-only inbound, automatic owner replies, polls, attachments and reactions must keep working exactly as today. Smallest durable change, the way Kun builds Firstmate.
What Changed
Risk Assessment
🚨 High: Recipient replacement can transfer private replies without an established disclosure policy, and delivery delays can silently lose poll answers or break reply ordering.
Testing
Three targeted CLI suites and the syscall-audited manual exchange passed after correcting fixture setup; transcripts and product records were retained, disposable homes were removed, and the accepted real-Linq/two-account limitation remains untested.
Evidence: Targeted CLI transcript
Source: Targeted CLI transcript
Evidence: Captured input, outgoing reply and reaction records
Source: Captured input, outgoing reply and reaction records
Evidence: Manual CLI exchange and guard refusals
Source: Manual CLI exchange and guard refusals
Evidence: Observed courier file operations
Source: Observed courier file operations
Evidence: Disposable user-namespace capability check
Source: Disposable user-namespace capability check
unshare: write failed /proc/self/uid_map: Operation not permittedPipeline
Updates from git push no-mistakes
... (13 earlier update rounds omitted to keep the PR body within GitHub's 65536-char limit; full history is in the run log.)
🔧 Fix applied.
13 issues (10 errors, 3 warnings) still open:
bin/fm-courier-pickup.py:562- Queued replies can disclose the previous captain's conversation to a replacement recipient. Claim a private reply for captain A, leave it queued behind another text, then change policy.toml to captain B: send() reads B's number and publishes A's reply to B. The courier authorizes that request as a current-captain send. Related sites: bin/fm-courier-pickup.py:220-224 reuses the conversation binding without recipient identity; :283-291 reads the current recipient; :497-502 queues text without its authorized recipient; :549-552 also accepts receipts by ID without matching content digest. The source does not establish that recipient replacement authorizes transferring an existing conversation. Decide that policy; preventing transfer requires durable recipient binding, so the remedy needs authorization.bin/fm-courier-pickup.py:539- The vote watch starts when the question is claimed, before it is delivered. If the courier is unavailable for over a day, replies() deletes the watch while the question remains queued; after recovery the courier sends a fresh poll, but its votes are silently discarded. Related sites: bin/fm-courier-pickup.py:514-515 expires the watch; :537-539 registers it at claim time; :566 publishes the question later; :569-584 settles delivery without starting the watch; :377-379 ignores votes after removal. Keep undelivered question watches and start their expiry when the poll's sent receipt is confirmed, matching the direct bridge's delivery-based watch.bin/fm-courier-pickup.py:574- The 300-second consumed-without-receipt fallback can release later reply portions while the first remains deliverable. The courier consumes reply A, records an unknown provider result, and continues retrying it; pickup then drops A after 300 seconds and publishes B. B can arrive first, followed by A after recovery, and A's eventual receipt is never applied. Related sites: bin/fm-courier-pickup.py:98 defines the timeout; :550-552 ignores nonterminal receipts; :571-575 advances the queue; :583-586 skips the abandoned reply's stage. The courier's daemon.py ACTIVE/advance paths retain and retry unknown jobs. No intent requirement needs this blind advancement component; remove it and retain ordering until a terminal receipt settles the request.bin/fm-courier-pickup.py:366- The latest change deliberately changes failure-reaction ordering despite the required criterion that 'reactions must keep working exactly as today.' The hunk calls move(identity, 'failed') immediately after queuing NOT_TEXT. If an earlier text is awaiting its receipt, the failure notice remains queued while signal() publishes the failed tapback. The direct bridge advances this mark only after sending the notice, and the referenced courier wire likewise specifies failed after notification. Related sites: bin/fm-courier-pickup.py:421-423 applies the same change after capture exhaustion; :571-573 holds queued notices; :624 independently publishes stages; tests/fm-courier-pickup-cases.py:321 and :338 require early failure stages. Obtain an explicit amendment permitting early failure reactions, or preserve notice-before-reaction ordering.bin/fm-courier-pickup.py:574- Round 1's R9 fix leaves a permanent receipt-loss sibling: publish reply A, stop pickup before it observes A's sent receipt, and restart after the courier prunes receipts one day later. A remains published in pickup state, so send() never republishes it and blocks every later reply indefinitely. The courier explicitly supports same-ID resubmission to regenerate pruned receipts (receipt contract). Related sites: bin/fm-courier-pickup.py:501 persists publication state; :564 limits publication to the first attempt; :572-574 waits indefinitely; :575 requires a terminal receipt to advance. Preserve strict ordering and recover the receipt using the existing request ID. The remedy adds a receipt-recovery retry path, which needs authorization.bin/fm-courier-pickup.py:537- The criterion 'automatic owner replies, polls, attachments and reactions must keep working exactly as today' conflicts with combining question text and its optional poll through self.owe(..., offered) at :537-538. If question text succeeds but Linq refuses native polls with 402/403, the courier records unknown and retries the poll indefinitely; pickup blocks every subsequent answer. The direct bridge drops the refused poll and continues (existing behavior); the courier propagates poll errors into unknown (courier send). Round 1's R9 fix makes this blockage permanent. Related sites: bin/fm-courier-pickup.py:567-568 emits the combined request; :554 excludes unknown receipts; :573-574 blocks later texts; :585-586 also withholds the question reaction. Restore independent settlement of confirmed question text and best-effort poll failure through the courier contract, preserving strict reply ordering.bin/fm-courier-pickup.py:587- The R11/R12 fix round treats every unknown poll-bearing receipt as confirmed question-text delivery. But the courier emits unknown when the text send itself fails: notify.py:347 propagates that failure through daemon.py:152-154. If question A fails before delivery, pickup removes A, records completed playback, and releases B while the courier continues retrying A; B can arrive first. Related sites: bin/fm-courier-pickup.py:560-563 selects unknown; :590 removes the request; :593 starts its watch; :596-597 records completion; :600-601 advances its reaction. The approved rule assumes confirmed text and an uncertain poll, which this receipt cannot distinguish. The remedy needs authorization to extend the courier receipt with confirmed-text evidence or amend the approved unknown-settlement rule.bin/fm-courier-pickup.py:480- The earlier R5/R6 flag fix introduces an association before a question is published. Let ordinary reply X wait on an unknown receipt, then claim question A behind it. A immediately awaits an answer. When unrelated text C arrives, answered() links A to C even though the captain has never received A's question. C's eventual final answer marks A done without an answer to A. Related sites: bin/fm-courier-pickup.py:435 links every captured text; :539-543 flags and queues the unpublished question; :458 preserves the erroneous link; :469-474 propagates C's terminal stage; :600-601 cannot correct A when its question settles. At the shared answered boundary, exclude questions still queued and unpublished when associating ordinary text answers, while preserving the awaiting flag, established followers, and receipt-independent handling of published questions.bin/fm-courier-pickup.py:481- The R13/R14 fix round leaves unsent-question association siblings. Claim question A, then let outbox publication fail: send() persists published before publish() succeeds, so answered() considers A eligible and links an unrelated text C to it. Alternatively, a terminal denied-limit receipt removes A from the queue while retaining awaiting_answer, producing the same association. C's final answer then marks A done although the captain never received its question. Related sites: bin/fm-courier-pickup.py:545 sets the claim-time flag; :588-590 records publication before it succeeds; :594 and :607-608 discard refused questions without retiring answer eligibility; :437 associates captured texts; :471-476 propagates their completion. Record successful publication only after publish returns, and retire answer eligibility on terminal non-delivery before discarding the question, preserving established follows links and monotonic stages.bin/fm-courier-pickup.py:363- A temporary I/O failure permanently drops valid inbound messages or votes. For example, listing finds sequence 41, but reading it raises EIO; this handler persists cursor=41 without filing it. After recovery or restart, the cursor filter excludes that record forever. Related sites: bin/fm-courier-pickup.py:336 opens the spool; :160-167 opens, inspects and reads records; :365-366 advances and saves the cursor; :351 excludes skipped sequences. Distinguish terminal record refusals from operational OSErrors, and propagate operational failures to the existing tick handler at :630 so the cursor remains unchanged and the next tick retries.bin/fm-courier-pickup.py:205- The addedlen(record['transcript']) > 16000refusal changes the required existing inbound behavior. A valid 0640 courier record containing 16,001 ASCII characters fits the wire's 128-KiB limit, but pickup rejects it and permanently advances its cursor. The direct bridge attempts capture and sends FAILURE when the transport refuses; pickup never sends that notice or its failed acknowledgement. Related sites: bin/fm-courier-pickup.py:369-373 passes the refused record; :404-440 contains the bypassed failure handling. This contradicts the criterion that owner-only inbound must “keep working exactly as today.” Remove the pickup-only character cap, retaining the byte limit and existing capture failure handling. Sources: inbound contract, direct bridge.bin/fm-courier-pickup.py:121- Simplification: the added terminal-result aliasesfailedandrefusedexceed the courier contract; its terminal failures aredenied,denied-limit, andclosed. No intent requirement needs this extra acceptance path. Remove those two aliases. Related sites: bin/fm-courier-pickup.py:571 recognizes them as terminal; :599-603 discards their queued text and retires question eligibility; :616 removes their poll watch. Source: courier terminal results.bin/fm-courier-pickup.py:487- The R14/R15 fix rounds (bd1cb6b/b008343) left a multi-question sibling. Deliver Q1, then queue Q2 on the same request behind an unknown receipt. A text answer to Q1 is captured, but Q2 puts the original message in this unpublished set, preventing its follower link; the answer's final reply never marks the original done. Alternatively, refusing Q2 clears Q1's answer eligibility, so a valid Q1 vote loses the same link. The transport permits successive nonfinal questions with distinct bindings. Related sites: bin/fm-courier-pickup.py:490-493 filters text answers; :550-558 shares the awaiting flag across questions; :601-602 clears it on refusal; :393-400 still accepts Q1's vote; :443 resolves captured answers; :476-483 propagates completion only through established links. Preserve eligibility for an earlier published question when excluding or retiring a later one, using existing mark, queue and watch evidence.🔧 Fix applied.
1 error still open:
bin/fm-courier-pickup.py:562- Queued replies can disclose the previous captain's conversation to a replacement recipient. Claim a private reply for captain A, leave it queued behind another text, then change policy.toml to captain B: send() reads B's number and publishes A's reply to B. The courier authorizes that request as a current-captain send. Related sites: bin/fm-courier-pickup.py:220-224 reuses the conversation binding without recipient identity; :283-291 reads the current recipient; :497-502 queues text without its authorized recipient; :549-552 also accepts receipts by ID without matching content digest. The source does not establish that recipient replacement authorizes transferring an existing conversation. Decide that policy; preventing transfer requires durable recipient binding, so the remedy needs authorization.✅ **Test** - passed
✅ No issues found.
TMPDIR="$PWD/.test-phase-tmp" PYTHONDONTWRITEBYTECODE=1 bin/fm-test-run.sh --jobs 1 tests/fm-courier-pickup.test.sh tests/fm-inbox-conversation.test.sh tests/fm-wake-queue.test.shRan~/.no-mistakes/evidence/01M4FHFHD0EJJR9Z226W7BADVC/manual-pickup-check.pythrough a disposable Bash session-owner fixture; corrected fixture shell lifetime and umask, then reran successfully.Traced realbin/fm-courier-pickup.py onceprocesses usingstrace -f -qq -yy -e trace=%file,%creds; captured public CLI input, outbox requests, stages and unchanged courier-file metadata.unshare --user --map-root-user -- true— UID-map permission denied.Checked completed transcripts for failures and skips, removed disposable test scratch, and verifiedgit status --shortwas clean.✅ **Document** - passed
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.