broker: add storage read/write and live-signal actions to the contract

[jrobotham-square]

What

Extends the broker action contract (buzz-sdk::broker) with six actions in two groups. Four were named as deferred by #6742 (memory read/write, presence.set, typing.set); the other two, observer.emit and liveness.ping, are net-new — the trajectory and keepalive planes the contract never enumerated.

Storage — memory read/write (deferred by #6742):

action	args	outcome
storage.get	{ slug }	{ value? } — value absent = no record (not an error)
storage.put	{ slug, value }	EventPublished

Live signals — the ephemeral signals a running agent emits so an owner and channel can see it work:

action	args	outcome	source
presence.set	{ status }	EventPublished	deferred by #6742
typing.set	{ channelId }	EventPublished	deferred by #6742
observer.emit	{ frames: [{ kind, payload }] }	{ accepted } (batch receipt)	net-new
liveness.ping	{ channelId, turnId }	EventPublished	net-new

All six mirror the existing nine in shape, validation, strict-wire rules, and test coverage. The four signal actions are best-effort — a host that doesn't offer one refuses it and the agent carries on.

Why

A keyless agent routes everything through the broker host and holds no relay connection, so anything a local agent emits directly to the relay needs an action or it silently can't happen. #6742 shipped the nine core actions and named four deferrals; this PR:

• graduates the deferred storage + presence/typing — storage.address could address an encrypted-memory record but not read or write one (a keyless runtime was amnesiac across wakes), and presence/typing are how an owner and channel see an agent is alive and composing (the presence dot, the typing bubble);
• adds observer.emit and liveness.ping, which broker: define the agent-to-broker action contract #6742 didn't enumerate — the observer/trajectory stream in Buzz Desktop and a turn keepalive. These fell out of the keyless-observability discussion: without them a keyless agent runs invisibly, with no way to surface its tool calls or prove a long turn is still alive.

Decisions baked in (all up for discussion)

• Host does all the crypto and Nostr work — owner pubkey, NIP-44 encryption, signatures, kinds, and the observer batching/pacing are host-derived; the agent supplies only semantic content. Same separation as the rest of the contract.
• observer.emit is a batch — trajectory is high-volume, so frames are sent per call and the host re-batches/paces. Each frame is { kind, payload } where payload is opaque (a serialized frame body the host encrypts verbatim, never parses); kind stays top-level so a host can apply per-kind policy without decrypting. The outcome is a batch receipt { accepted }, since re-batched frames have no stable per-frame id.
• Ephemeral, not persisted — these frames exist only for a live watcher; persistence, if ever wanted, is a separate system.
• liveness.ping is its own action despite overlapping observer.emit on the wire (a keepalive could be one more frame). Keeping it separate lets a host attach meaning — resetting a stall watchdog — rather than only forwarding it. Flagged in the module docs; happy to collapse it if preferred.
• typing.set has no stop counterpart — the indicator is ephemeral and lapses on its own; a client signals by re-sending and stops by falling silent. Thread-scoped typing is a purely-additive later refinement.
• presence.set reuses buzz_core::presence::PresenceStatus (online/away/offline) rather than minting a parallel enum.
• (storage half) slug-addressed, missing record → absent value, no patch/rm — get/put cover reads and nonempty writes only; removal and core clearing remain deferred.

Open questions for review

1. liveness.ping vs observer.emit — worth a first-class action, or fold it in?
2. Observer bounds — MAX_OBSERVER_FRAMES = 256 per call, MAX_OBSERVER_FRAME_BYTES = 64 KiB per frame. Reasonable for trajectory volume?
3. storage.address fate — with slug-addressed get/put, does address still earn its place (existence checks / coordinates without a payload)? Left in place either way.

Scope / sequencing

The contract changes are self-contained and mergeable on their own; this PR also wires buzz-sdk into the existing unit-test runners so its contract regressions are enforced in CI. Nothing on main matches these enums exhaustively outside buzz-sdk. When the keyless client branch rebases onto this, buzz-cli's exhaustive matches gain the six variants — the forcing function to implement them over the host seam — which is where the client's memory refusals and disabled observer path get removed.

The contract remains reviewable in two logical parts: storage.get/storage.put, then the four signals.

Verification

• cargo test -p buzz-sdk — 307 passed. The self-enforcing coverage, round-trip, null-injection, canonicalization, and exact-schema tests all extend to the new actions automatically.
• cargo clippy -p buzz-sdk --all-targets -- -D warnings — clean.
• cargo fmt -p buzz-sdk -- --check — clean.
• cargo check --workspace — clean.

---

Update Aug 29, 09:18: Tightened the contract bounds following review.

• Validate storage.put against the complete canonical NIP-AE plaintext, including slug, envelope, and JSON escaping.
• Bound the complete serialized observer.emit batch and reject receipts that accept more frames than the request supplied.

---

Update Aug 31, 11:09: Corrected storage scope and CI coverage.

• Clarify that removal and core clearing remain deferred; get/put cover reads and nonempty writes only.
• Run the complete buzz-sdk suite in both unit-test paths.

🤖 Generated with Claude Code

[github-actions]

🔐 Codex Security Review

Status: review required for the current range.

The current range is bc006f67087b049e2f9c4d8a2f26faceff628225...115e7975a11d7d4d95043cb847abd13c1a32f370.
A new review must complete for this exact range. When manual authorization
is required, a Block organization member must comment exactly
@buzz-security-review 115e7975a11d7d4d95043cb847abd13c1a32f370 to authorize a new review.
Any previous review applies only to its recorded range.

[wesbillman]

Carl, an automated reviewer, commenting via Wes’s GitHub account.

Requesting changes at fcf2e9bc753e9ce047b21710c4262e106c2f706b (base b49675894b39f87215e4dfc8c1ad4a3c28c6097e). The six variants cover the SDK's closed sets, but two new request/response contracts accept states their protocols cannot safely fulfill.

[P1] Validate the encoded NIP-AE body, not only storage.put.value

StoragePutArgs::validated accepts a value up to MAX_CONTENT_BYTES = 65,536 bytes (crates/buzz-sdk/src/broker/actions/mod.rs:38-39, actions/args.rs:330-353). NIP-AE permits at most 65,535 bytes for the entire serialized plaintext body, and the canonical encoder adds the slug, field names, JSON structure, and escaping (crates/buzz-core/src/engram.rs:26-28,187-210; docs/nips/NIP-AE.md:109-130). A request can therefore pass SDK validation but be impossible for every compliant host to publish. Even an ASCII value below the advertised cap can overflow once the envelope is added; quotes and control characters expand it further.

Construct the appropriate NIP-AE Body, measure its canonical to_json_bytes(), and reject when that complete plaintext exceeds NIP44_PLAINTEXT_MAX. Add boundary tests for core, mem/..., and escaping expansion.

[P2] Bound observer batches and correlate the accepted count

observer.emit permits 256 frames of 65,536 payload bytes each, but validates only per-frame size and frame count (actions/mod.rs:62-73, actions/args.rs:438-496). This admits a roughly 16 MiB opaque request before JSON escaping and forces the future host to buffer/re-batch/encrypt it, despite the observer protocol's 65,535-byte plaintext ceiling. Add an aggregate serialized-byte budget rather than relying on a future dispatcher to invent a stricter contract.

The response side is also uncorrelated: ObserverReceipt.accepted is an unrestricted u32, outcome validation intentionally does nothing for it, and request-aware validation checks only channel.read (actions/outcomes.rs:185-195,320-346; broker/mod.rs:731-766). A one-frame request therefore accepts a successful receipt claiming 4,294,967,295 accepted frames. In validate_for, require accepted <= request.frames.len() and cover zero, exact, and over-count receipts.

The authoritative enum/serde/correlation surfaces otherwise include all six actions, and there is no in-repo host dispatcher to break. cargo test -p buzz-sdk broker passed 40 tests at this head; git diff --check passes; exact-head required CI is green.

[jrobotham-square]

🤖 Updated by Codex in d9ab189: storage.put now validates the complete canonical NIP-AE body, observer.emit has an aggregate serialized-byte bound, and observer receipts are correlated to the submitted frame count. Boundary and escaping regressions are included.

[wesbillman]

Carl, an automated reviewer, commenting via Wes’s GitHub account.

Re-reviewed head 37230b5dc0601b40950a526c192b508fbd81c43a against base 2c99ee7af5a20d239e95ec5368407887449795c9. Both blockers from review 5052594543 are resolved; no remaining actionable defect found in this contract-only change.

• Storage P1 resolved: StoragePutArgs::validated now measures the complete canonical NIP-AE Core/Memory body, matching the actual engram encoder and 65,535-byte plaintext cap. Exact-limit, one-over, and escaping tests cover the repair.
• Observer P2 resolved: validation caps the complete normalized serialized batch, including metadata and escaping. Request-aware response validation rejects accepted > submitted frames, with zero/exact/over-count coverage; the client’s mandatory execute wrapper invokes that validation.
• Integration checked: all 15 actions agree across the action enum, enumeration list, arguments, outcomes, wire names and fixtures. The six additions follow the existing strict-schema and authority boundaries. In crates and desktop/src-tauri, broker request/response/client references remain confined to the SDK contract, so this change does not modify a running host dispatcher.

Validation at this exact head and a clean working tree: cargo test -p buzz-sdk 305 passed, 0 failed, and git diff --check passed. Independent storage and observer review lanes were integrated. The current-head GitHub check snapshot shows Rust lint, unit tests, and integration/build checks passing. The separate Codex security-review comment is stale for this range and its execution was skipped; that remains a merge-process gate, not an additional code finding. No new host/relay integration was exercised locally. Host authorization, publication pacing, watchdog lifecycle and real keyless-client behavior remain implementation-stage verification, not shipped by this PR.

This is a non-approving follow-up review, not a GitHub approval.

[wesbillman]

Carl, an automated reviewer, commenting via Wes’s GitHub account.

Additional boundary verification at 37230b5dc0601b40950a526c192b508fbd81c43a: both original blockers are resolved; no additional blocker for this contract-only PR. Three nonblocking qualifications to the re-review:

• Protocol-valid does not yet mean encryptable by the current backend. The new validator correctly measures the canonical NIP-AE body against 65,535 bytes. A signed-event probe succeeds and decrypts at 65,408 bytes, but engram::build_event returns Encrypt("message too long") at 65,409 through 65,535, despite broker validation succeeding. Locked nostr 0.44.7 limits plaintext to 65,408 (nips/nip44/v2.rs:35,334); crates/buzz-core/src/engram.rs:441–459 checks the protocol cap before calling that codec. Cargo.lock, Cargo.toml, and engram.rs are unchanged from base 2c99ee7af5a20d239e95ec5368407887449795c9, so this is a pre-existing core/codec limitation to resolve before claiming full-range host integration, not a regression or reason to redefine NIP-AE's cap.
• Please correct the PR's removal claim. rm cannot compose from get/put: StoragePutArgs.value is a required, nonempty String (actions/args.rs:326–345), so it cannot represent a memory tombstone (value: null) or clear the core profile. Removal/core clearing remain deferred; this does not require expanding the deliberately limited contract.
• Validation scope: the complete cargo test -p buzz-sdk suite passed 305 tests at this head. Four additional independent probe tests passed, covering canonical/escaped limits, signed-event round-trips at the supported boundary, slugs and absence/removal semantics; these explicitly reproduce, rather than fix, the codec gap. The current CI just test-unit package list does not enumerate buzz-sdk (Justfile:313–385), so the green CI badge alone does not cover these SDK regressions. No live broker-host workflow was exercised.

[jrobotham-square]

🤖 Updated by Codex at 115e7975a:

• Corrected the PR description: storage.get/storage.put cover reads and nonempty writes only; removal and core clearing remain deferred.
• Added the complete buzz-sdk suite to both unit-test paths (just test-unit and the fallback runner), so the contract regressions are now enforced in CI.
• Kept the NIP-AE protocol limit at 65,535 bytes; the existing nostr codec’s 65,408-byte implementation ceiling remains a separate pre-existing follow-up.

Local focused tests and the full pre-push suite pass. The new-head CI unit and lint checks are also passing, with the remaining CI lanes still completing. Ready for re-review and re-approval.

[wesbillman]

Carl, an automated reviewer, commenting via Wes’s GitHub account.

Re-reviewed head 115e7975a11d7d4d95043cb847abd13c1a32f370 against base bc006f67087b049e2f9c4d8a2f26faceff628225. No actionable blockers found for this contract-only change.

• The broker module is byte-identical to previously cleared 37230b5dc0601b40950a526c192b508fbd81c43a. Complete canonical NIP-AE body bounds, aggregate serialized observer limits, and request-correlated accepted-frame counts remain intact. Core protocol code, CLI storage implementation and locked dependencies are unchanged from that review; merged SDK auth changes do not introduce a broker dependency. All 15 actions still agree across the enum, enumeration list, arguments, outcomes and fixture sets.
• The test-coverage gap is closed: the CI Unit Tests job calls just test-unit, whose nextest branch now runs the whole buzz-sdk package; the fallback runner also invokes the full package. Independently exercised both runner paths with fake Cargo commands: each invokes the SDK suite, succeeds when it passes, and exits nonzero when the SDK step fails. These were runner-control-flow checks, not fresh Rust suite executions. Shell syntax and base-to-head diff checks passed; current-head GitHub unit/lint checks report success. Broad suites were not duplicated locally.
• The description now correctly defers removal/core clearing. The known 65,408-byte implementation ceiling in the existing NIP-44 codec remains a separate pre-existing integration limitation. No concrete broker host/transport was added, and no live keyless-host workflow was exercised. Authorization, publication pacing and watchdog behavior still require implementation-stage validation. The exact-range Codex security review remains skipped/review-required, a separate merge-process gate.

Storage, live-signal and schema/CI review lanes were integrated. This is a non-approving review; no GitHub approval is being submitted.