Skip to content

fabric DB: replace git under the fabric event log - #11723

Merged
briansrls merged 9 commits into
mainfrom
session/gentle-seal-606
Sep 20, 2026
Merged

briansrls merged 9 commits into
mainfrom
session/gentle-seal-606

Conversation

@briansrls

@briansrls briansrls commented Sep 19, 2026 •

Copy link
Copy Markdown
Contributor

Replaces git as the storage of the fabric event log with our own fabric DB (operator decision 2026-09-19). This is a replacement migration (DESIGN §3): the git plumbing is deleted in one motion, with no dual path and no fallback to git. Scope is steps 1–3 of docs/plans/fabric-db.md.

What changed

1. Interface: std.fabric_db. Immutable content-addressed objects plus compare-and-set named heads, built from std.content_hash keys and std.durable_compare_and_set rather than new vocabulary.

  • Links the store can see. An object carries its links as data the store reads, so a partition read is one closure walked where the objects are held. This is the fix for P0-B: git never saw the parent inside the event JSON, so a fetch moved only the tip.
  • Typed refusal arms:
    • FabricStoreUnreachable
    • FabricStoreRefused (carries CasStoreFailure)
    • FabricObjectMissing
    • FabricObjectCorrupt
    • FabricReplyUndecodable
    • FabricHeadNameRefused
  • Lost race. A lost race is the outcome FabricHeadMoved, carrying both heads. Under git, a lost race and an unreachable store both exited 128.
  • Empty object. An object body is NonEmptyStr, so git's "empty blob published as head" defect has no constructor.
  • Stated divergence from std.artifact_store. That store evicts entries least-recently-used; log history must never be evicted.

2. Realization. Interface, realization and policy are three separate facts.

  • File store: gunbc.fabric_db_file_store. Heads are durable_cas_file_store O_EXCL slots, objects are write-once files beside them, and the closure walk is bounded.
  • Endpoint: gunbc.fabric_db_serve. It is gunbc serve on srv1, bound to 127.0.0.1, reachable only through tailscale serve on :10000 at the root mount. srv1's :443 is the dashboard and :8443 is ntfy.
  • Caller side: gunbc.fabric_db_client. The placed host runs the store in process; every other host makes one bounded curl POST. The new extdeps.http.client PostStdinWithin keeps curl's exit code, which extdeps.tools.curl CurlExitOutcome classifies. Connect failure or timeout becomes FabricStoreUnreachable before any reply is read, and a non-200 reply becomes FabricReplyUndecodable.
  • Wire and policy: gunbc.fabric_db_wire is the one text form all arms cross the hop in. gunbc.fabric_db_placement is the policy row: srv1 and the store root, with the URL derived from the fleet tailnet domain.
  • Deploy:
    • The store root is an ensured host directory (FabricDbStoreDirectory), so no retract can remove the log. The old mirror root was owned and torn down with rm -rf.
    • The endpoint's unit and tailscale route are owned by the srv1-live deployment only (fabric_db_placed_on).

3. Cutover. gunbc.fabric_event_log runs on the fabric DB; the mirror, ls-remote, fetch, cat-file, push-with-lease and placement ensure are deleted.

  • Typed refusals end to end. Refusals carry a typed EventLogRefusal, and seat outcomes gained SeatStoreRefused / SeatStandingStoreRefused.
  • Consumers updated:
    • harness_seat, harness_cli
    • fabric_quota, fabric_event_log_host
    • roadmap_publish_observe, a consumer the survey missed
    • both probes (fabric_event_log_probe gained a _served entry; the collision probes take an endpoint)

Single writer by construction. The head CAS stays O_EXCL on srv1's local disk, so concurrent requests cannot both win; nothing relies on the serve loop being serial. Restart loses nothing: state is the files, and the handler holds nothing in memory.

Drain (operator condition)

Partitions start empty; nothing is imported from git.

  • Seat term. The longest term the fabric grants is gunbc.harness.harness_cli harness_seat_policy.maximum_duration_seconds (request deadline + release allowance).
  • Quota term. The live quota caller (roadmap_publish_observe) takes 120 s.
  • After deploy. For at most one seat term, a pool can over-grant by seats still held under the git log. Deploy with harness turns paused, or accept that window.
  • Old log. /opt/gunbc/fabric-event-log.git on srv1 stays read-only as history, and nothing consults it.

Access (operator condition)

  • Reach. Only tailnet members reach the endpoint: the backend is loopback-bound behind tailscale serve, per extdeps.tailscale.identity.
  • Declared gap. No principal is refused; any tailnet member may append. The next rung is a modeled fabric-writer principal roster, sufficient for the handler to refuse a login outside it.

Evidence

All runs are gunbc run --claim-run at this head, built locally.

New witnesses (all pass):

  • test.claim.fabric.fabric_db_witness: 9 pure claims (decode, truncation, stream, verification, head admission).
  • test.claim.fabric.fabric_db_wire_witness: 10. Each arm survives the hop, and the transport arms are checked: exit 7/28 → unreachable, a non-200 → undecodable.
  • Live on the local-repo wet lane, each enrolled as a triple (schedule, route-gap chunk 17, exclusion row):
    • test.claim.fabric.fabric_db_file_store_wet_witness: 9. These are the discriminating REDs for lost race, missing object, hole, corrupt object, bound and navigating name, plus the served handler's store side.
    • test.claim.fabric.fabric_event_log_wet_witness: 3, including the inhabitance claim: three chained appends read back whole through the real store.

Evidence moved, not dropped (§4b(4)). The git red control for identity_hashed_from_a_shared_mutable_path moved to test.claim.devboot_text_blob_real_execution, since devboot is git's remaining stdin caller. The failure-mode row's evidence was updated to match.

Live checks:

  • fabric_event_log_probe against a fresh store: all 9 steps pass (appends, stale race, chain read, seat grant → full → release → grant).
  • Collision probe against a dead port: store unreachable: could not connect (curl exit 7).
  • Served probe against a stub HTTP server: decodes the reply through the real curl operation.

Re-run touched suites (pass): harness guidance, turn admission, turn completion; roadmap_publish_observe; devboot_subject_identity; desired_roster; live_deploy apply and emit (67); local_repo_wet_terminal; witness_exclusion_reconciliation; serving_availability_bind_wet; http_client_get.

Pre-existing reds, reproduced at base 43643bad4f, not caused here:

  • deployed_tree_remote_witness the_remote_converges_after_its_inputs_and_before_the_belt
  • harness_seat_witness the_tolerant_pool_is_a_separate_pool_whose_ceiling_is_not_the_engines_sequence_limit
  • live_deploy_unit_emission_oracle the_serve_unit_matches_the_deployed_bytes and serve_unit_execstart_carries_admitted_entrypoint

Pending: the live read of the group A partition on srv1, which needs deploy plus srv1 access. The partition starts empty after cutover, so the first live receipt is fabric_event_log_probe_served against the srv1 endpoint, followed by a real seat bind.

Known limits (declared in the plan doc)

  • Object refs use the non-cryptographic structural digest; a collision is refused at put, never read through.
  • The head generation probe and the chain walk are bounded at 4096; compaction is step 5.

🤖 Generated with Claude Code

Brian Searls and others added 4 commits September 19, 2026 15:37
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… deploy and witness rosters (WIP)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…route), ensured store root, served-handler claim, plan doc

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@gunbai-bot
gunbai-bot Bot marked this pull request as ready for review September 19, 2026 17:35
… a verb that never landed (review 68555)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@gunbai-bot

gunbai-bot Bot commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

Addressed review 68555 in e3d9233:

  • Transport bounds are now Second (fabric_db_connect_bound = 5 s, fabric_db_request_bound = 60 s), stringified at the argv edge through extdeps.http.client http_client_max_time_flag. Re-run: curl receives --connect-timeout 5 --max-time 60, and a dead port still refuses as store unreachable (curl exit 7). The operation inputs stay string words like every other http.Client operation, since they are argv; no new scalar authority remains in the product layer.
  • The stale gunbc.fabric_db_verb citation in gunbc.fabric_db_file_store now names gunbc.fabric_db_serve / gunbc.fabric_db_client. grep fabric_db_verb is empty.

Brian Searls and others added 2 commits September 19, 2026 18:56
… over it (floor NewUnresolvedness)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ung drop (review 68598)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@gunbai-bot

gunbai-bot Bot commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

Addressed review 68598 in 0a43a9a. The finding holds: under git an append needed an ssh credential on srv1, and the served endpoint refuses no principal.

  • Not restored in this change. The identity a fleet writer presents through tailscale serve has not been read yet. Tailscale-User-Login is set for user-owned devices and absent for tagged ones, and a roster written before that reading would either refuse every real writer or admit everyone.
  • Filed as a §4b(3) declared drop instead: gunbc.rung_drop.fabric_db_append_principal_unrefused, mechanically preventable -> mitigatable, reason ReplacementStaged.
    • Population: fabric_db_serve_handle, event_log_append, fabric_seat_acquire.
    • Restoration trigger (the capability, not an artifact): a fabric-writer principal roster grounded in an observed reading of each writer's tailscale identity, sufficient for the handler to refuse any identity outside it (absent included), with a discriminating RED over the real handler.
  • Wiring: enrolled at the end of gunbc.rung_drop.roster, and docs/design-rung-drops.md regenerated via tools.docs_projection_gate regen (only that file changed). The handler's ACCESS note now cites the row.
  • Next: that reading needs srv1 access, which I've requested for the deploy. I'll take the roster as the follow-up once the reading exists.

Brian Searls and others added 2 commits September 19, 2026 19:47
…ded rows kept, projection regenerated)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…s status line through extdeps.tools.curl classify_curl_http_code_after_newline (one authority)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@briansrls
briansrls added this pull request to the merge queue Sep 20, 2026
Merged via the queue into main with commit 1a5d4db Sep 20, 2026
4 checks passed
@briansrls
briansrls deleted the session/gentle-seal-606 branch September 20, 2026 00:44
briansrls pushed a commit that referenced this pull request Sep 20, 2026
Resolves: fabric_event_log_append_real_execution_witness_test deleted on main
(#11723) while migrated here -- deletion taken. Main added a new shell.Symlink
consumer (devboot_text_blob_real_execution_witness_test); migrated onto
ln_symbolic_force_command like the others, so the service deletion stands.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
gunbai-bot Bot pushed a commit that referenced this pull request Sep 20, 2026
…ng meets the participation roster

main's #11723 (fabric DB under the event log) replaced layout: EventLogLayout with store: FabricDbBinding
through the harness candidate/placement path, where this branch threads participation_roster; every
such function now carries both. fabric_event_log takes main's side wholesale (the branch's
cat_file_chasing_remote had no referent once the git mirror was replaced). serving_incarnation_observe
takes main's Optional refusal arm with this branch's names: threading re-applied. ci_spec and
fleet_converge_workflow union both sides' modes and steps. floor_route_gap keeps both expectation
populations, this branch's chunk renumbered to 18. Review 68783: the apply script's dead changed=0/1
computation is deleted.

The emitted fleet-converge.yml is main's copy pending regeneration: three local regeneration attempts
were refused (MemoryStallRefusedPageThrash) or killed by session restarts on a saturated host, so the
generated-artifact job is the adjudicator.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant