feat(bot-mode): Group Chats survive the authority gateway dying — replicated log and fenced takeover (#97681)

[teknium1]

Summary

A hosted Group Chat now survives its authority gateway dying: every participant gateway can keep a durable replica of the room's ordered log and continue the room at epoch + 1 with a lineage-proving takeover, while the returning stale host fences itself out. Implements the durability requirement from #97681 on top of the authority foundation merged in #99007.

Changes

• gateway/hosted_room_replicas.py (new, 570 LOC): replica store in root state.db — ingest_page() persists authority-stamped groups.log pages idempotently (gap- and epoch-regression-safe), promote_replica() continues the room locally at epoch + 1 with an authority.claimed claim event carrying the full lineage, demote_room() records authority.lost and fences a returning stale authority so split-brain writes are impossible
• tui_gateway/methods_groups.py: groups.replicate / groups.replica_state / groups.promote / groups.demote RPC methods; capabilities now advertise log_replication + authority_takeover
• Promotion requires confirm: true: storage makes takeover atomic and provable; the caller (explicit user action today, a lease/quorum driver later) decides when it is safe — the exact boundary blessed on Bot Group Chats should keep working after Desktop closes #97681

Validation

Check	Result
New replica + RPC suites	20 passed (incl. full failover round-trip)
Whole hosted-rooms area	69 passed
E2E: two real gateway stores, real install identities — A hosts 6 events, B replicates in 2 incremental pages, A dies, B promotes with complete history (claim at seq 7), work continues at epoch 2, A returns → demoted, stale write rejected	passed
Ruff	clean
Stale-base gate	0 behind origin/main

Infographic

[github-actions]

૮ >ﻌ< ა ci review

_{ran on 6af548a — feat(bot-mode): Group Chats survive the authority gateway dy}

⚠️ Warnings

OSV vulnerability scan · View job

6 known vulnerabilities found in pinned dependencies.

• CVE-2026-67213 in website/package-lock.json
• CVE-2026-71554 in uv.lock
• CVE-2026-70608 in package-lock.json
• CVE-2026-56876 in package-lock.json
• CVE-2026-70606 in package-lock.json
• CVE-2026-71554 in uv.lock

How to fix:

Review the findings in the Security tab. Update the affected dependencies if a patched version is available.

---

debug info

CI timings

CI timings · View report · View job

Wall time 4m14s vs 4m15s (-0.4%). 5 job(s) slower, 3 faster, 4 unchanged.

• OS-specific tests / Windows-only tests: -25.0s
• OS-specific tests / macOS-only tests: +4.0s
• Python tests / e2e: +3.0s
• Python lints / Windows footguns (blocking): -3.0s
• OSV scan / Scan lockfiles / osv-scan: +3.0s

[dokterdok]

I reproduced four current-head blockers on exact 6af548a1f73f26bca54dd2e23f0d278c37f3ba6c using two real SQLite stores and the production functions.

P1 data loss: a partial replica can be promoted

promote_replica() reads last_seq but not latest_seq and never requires the replica to be caught up (gateway/hosted_room_replicas.py:355-383). I ingested the first 2 events of an authority log whose page reported latest_seq=5, then promoted. The new authority contained only those 2 events plus the claim:

PARTIAL_PROMOTION source_latest=5 stored_before=2 promoted_latest=3 visible_events=3

Promotion must require last_seq == latest_seq, with a monotonic non-regressing latest_seq and no unresolved gap.

P1 integrity: contradictory overlap is silently accepted

Ingest drops every event with seq <= last_seq without comparing it to the stored row (:210-214). Replaying sequence 1 with a different payload returned ingested: 0, caught_up: true. Idempotency must compare the exact canonical event ID/kind/actor/epoch/payload/created-at for every overlap and reject any mismatch.

P1 authority forgery: page stamp is not tied to the event lineage

The caller supplies page.authority; ingest only rejects a lower epoch (:122-156, :205-208). It does not require event epochs to match the stamped authority or an authority.claimed chain. Changing an epoch-1 page stamp to an unrelated gateway at epoch 99 was accepted, then promoted as that gateway's successor at epoch 100:

FORGED_LINEAGE_PROMOTION previous_gateway_id=install:ffff... previous_epoch=99 authority_epoch=100

Because groups.replicate accepts the page verbatim from client params, transport trust does not supply this missing proof. Ingest/promotion need to validate the complete authority-claim chain and bind room identity/membership to authoritative state, not caller prose.

P1 divergent canonical history on demotion

Promotion appends system:authority-claimed:2 at last_seq + 1 (:383-449). The old host's demote_room() independently appends system:authority-lost:2 at its own same next sequence (:528-563). The reproduced logs therefore disagree at sequence 4:

DIVERGENT_SEQ seq=4 old_host_event=system:authority-lost:2 new_host_event=system:authority-claimed:2

The returning host must first ingest the actual promotion claim and converge on that canonical event. A local demotion receipt, if needed, belongs outside the room log; it cannot consume the claim's sequence.

P1 split brain: two survivors can both become epoch 2

confirm: true is not bound to an expected authority/cursor or a unique winner. Two independent replicas of A both promoted successfully: B became (B, epoch 2) and C became (C, epoch 2), and both accepted writes. An explicit recovery operation still needs one globally exclusive decision/fence; a boolean acknowledgement is not that fence.

P1 tombstone resurrection

The replica schema carries no terminal/disband state. Ingesting the complete replay of a real disband_room() and then promoting created a new active room (disbanded_at=NULL) with the old room.disbanded event followed by an authority claim. Promotion must preserve the permanent retired-ID contract and reject a replicated tombstone.

These are storage-contract blockers independent of the later automatic driver. The PR body is also currently stronger than the implementation: no code continuously sends pages or invokes promotion/demotion, so this head provides replica/takeover primitives, not yet a Group Chat that automatically survives authority-host death.

Two claim corrections also need attention: the two new test files collect 16 tests, not the stated 20, and the replica store's 256 MiB per-room allowance bypasses #99007's 16 MiB root-database safety budget during direct SQL promotion.

Recommended gate: add hostile regressions for all six P1 shapes, then run the two-store round trip again with partial pages, duplicate conflicting pages, A -> B -> C claim validation, concurrent B/C takeover, disbanded rooms, crash boundaries, capacity accounting, and old-host convergence onto the exact promoted log.

[dokterdok]

Corrective draft is now #99107. It preserves passive replicas but disables unfenced takeover, adds database-enforced room-ID reservation for mixed-version processes, quarantines any #99047 promotion/demotion lineage already written, preserves disband tombstones, validates overlap/coverage, and shares the root-DB byte budget. The current exact safety suite is 84/84; final independent review and CI are in progress.