Skip to content

feat(custody): add independent Rust quorum barrier; keep migration staged - #153

Draft
seonghobae wants to merge 13 commits into
fix/remove_stale_hourly_pr_steward_test_20260910from
fix/bootstrap_secret_mounts_no_dotenv_20260910
Draft

feat(custody): add independent Rust quorum barrier; keep migration staged#153
seonghobae wants to merge 13 commits into
fix/remove_stale_hourly_pr_steward_test_20260910from
fix/bootstrap_secret_mounts_no_dotenv_20260910

Conversation

@seonghobae

@seonghobae seonghobae commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Current scope — 2026-09-10

Keyverse itself is the independent custody/KMS and secret-lifecycle owner. External KMS/HSM is optional. Replacing .env with static credential files, _FILE or Kubernetes Secrets does not satisfy the migration requirement. Central register: ContextualWisdomLab/.github#2063.

Exact source head: 8418d8bde90ad65a3c4b98aeccee830426f34f14.
Exact tree: 47b47ce6f0641116cdf8463ed646424729d33cc9.
Lifecycle: Draft / internal native source added / compilation and runtime acceptance unverified / no deployment.

This ordinary child preserves all earlier #153 transport and architecture-correction deltas. It is not a released KMS, and no consumer may depend on this branch.

Native implementation added

services/key_custody is an internal Rust library with publish = false, unsafe code forbidden and public documentation required:

  • OS-random wrapping root and independent recovery factor; bounded Shamir quorum shares are returned separately from an authenticated encrypted root record.
  • Initialization and reconstruction return sealed. A full, distinct, matching quorum and successful encrypted-root authentication are required before installing any root state.
  • Reseal drops the locally held zeroizing root. No public root export, Clone or serialization API is provided.
  • XChaCha20Poly1305 record protection binds the instance and length-framed tenant/environment/namespace/key/version/purpose fields. Wrong context, ciphertext or nonce does not authenticate.
  • Bounded parsing, input sizes and share counts; entropy failure has no fallback; sensitive buffer Debug output is redacted.
  • No environment, dotenv, credential-file, database or network reader; no external vault/KMS bootstrap dependency.

Context binding is cryptographic association, not workload authentication or authorization. This internal primitive requires a trusted in-process caller; it is deliberately not exposed as an unauthenticated remote API.

Authored tests and actual evidence

There are 18 authored Rust test functions: 15 lifecycle/context integration cases, two private error/entropy cases, and one independent C-generated ciphertext interoperability case. They cover quorum combinations, insufficient/duplicate/foreign/corrupt shares, every root-record byte mutation, sealed operations, reseal/recovery, all context dimensions, ambiguous field concatenation, data mutation/truncation, size bounds and redacted diagnostics.

Native tests executed locally: zero. Cargo/rustc are unavailable in the current container and toolchain downloads failed. Rust compilation, Clippy, rustdoc, formatting and native coverage are not verified.

Test-first scaffold a66a65f71c8c048c9590e70bfbaf42d11d64ef56 was published before the implementation, but run 34424727022 / native job 102707407636 remained queued at the observed reads. Its assertion was not observed failing before implementation. This is not a completed RED→GREEN cycle.

Actual independent local verification: the installed libsodium 1.0.18 C API encrypted/decrypted public unit-test root/data fixtures, rejected all 52 one-bit-per-byte data ciphertext/tag mutations and rejected changed associated data. The Rust interoperability case consumes those exact fixtures but has not executed. This validates the test oracle only, not this new Rust composition. Local new-file whitespace checks passed, and source/test/README Git blob hashes match the published objects.

Detailed chronology, input values and SHA-256 receipts are in docs/doctoring/key_custody_barrier.md. The product gap baseline adds this delta while preserving the historical identity/MCP obligations.

Current CI and dependency-lock gate

Fresh exact-head CI run 34426740038 is queued at the latest read. The native job runs on Draft with contents-read only, exact-head checkout and no credential/deployment/publication step. It is in the existing product CI rather than a duplicate review/security workflow.

The manifest pins dependency versions, but a real committed Cargo.lock is still missing. The temporary candidate-resolution step prints the generated public lock for review. The final git ls-files --error-unmatch Cargo.lock and clean-lock gate must fail until it is captured and committed. Remove candidate generation and rerun locked verification before promotion. Do not invent checksums or call unlocked resolution immutable acceptance.

A passing historical Python test, a skipped Draft job, a queued native job or an inspected source file is not native GREEN evidence.

Preserved transport work — not accepted final deployment

The earlier source deletes .env.example, mounts three /run/keyverse-bootstrap/* files, uses POSTGRES_PASSWORD_FILE, and exports mounted values immediately before Keycloak starts. That hardening is retained, but its producer still has no Keyverse issuance/rotation/revocation lifecycle. It remains a staged compatibility proposal, not the independent standalone architecture or proof of organization migration.

The native library is not yet wired into Compose or a consumer. A real service boundary must authenticate enrollment/workloads, authorize operations, durably audit, persist atomically, handle recovery/rotation and publish immutable contracts before cutover. Existing static transport must then be confined or replaced without losing deployed-data recovery.

Remaining release and integration gates

No real credentials were read, migrated, rotated or deleted. No force push, self-approval, status fabrication, gate weakening, merge, release or deployment occurred.

@coderabbitai

coderabbitai Bot commented Sep 9, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae
seonghobae marked this pull request as ready for review September 9, 2026 23:37
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@coderabbitai review

Please review exact head cf8498dd33b7d64363a8359d76cab4cb47577ba9, focusing on the self-bootstrap trust boundary: Compose secret mounts, upstream PostgreSQL _FILE semantics, the Keycloak file-to-native-environment entrypoint, accidental dotenv fallback, exposure through process/log/argument surfaces, and the distinction between root bootstrap and ordinary Keyverse workload secrets. Hosted exact-head CI/security checks are now running; this is not an approval or merge request.

@seonghobae
seonghobae changed the base branch from main to fix/remove_stale_hourly_pr_steward_test_20260910 September 9, 2026 23:56
@seonghobae
seonghobae marked this pull request as draft September 9, 2026 23:57

Copy link
Copy Markdown
Contributor Author

2026-09-10 owner correction: Keyverse must itself be usable as an independent KMS/cryptographic trust service; an external KMS/HSM is optional, not a production prerequisite. Moving the same static password from .env into POSTGRES_PASSWORD_FILE does not satisfy the migration.

I am correcting this PR's ADR-0017, ADR index and doctoring only, from cf8498dd33b7d64363a8359d76cab4cb47577ba9. The existing transport hardening delta is preserved, but remains unaccepted compatibility work, not a standalone Keyverse release or a completed consumer migration. Current source/default Compose still requires repair before adoption. No deployment, credential rotation/deletion, force push, Ready transition, merge or release is performed. Keep this PR Draft while native custody, independent initialization/unseal, workload authorization and database credential lifecycle remain unimplemented.

…tion acceptance

Owner correction: Keyverse must itself supply independent KMS/cryptographic custody. External KMS/HSM is optional; POSTGRES_PASSWORD_FILE and static mounts are transport changes, not lifecycle migration. Supersede the earlier ADR-0017 choice while preserving existing source delta and explicitly retaining Draft/non-release boundaries.

Document independent initialization/seal/unseal, non-exportable key operations, separate secret retrieval, PostgreSQL credential/session lifecycle and exact evidence gaps. No runtime tests or deployment success claimed; no production source, credential or workflow change.
@seonghobae seonghobae changed the title fix(deploy): remove dotenv from Keyverse self-bootstrap fix(deploy): preserve transport hardening; require independent Keyverse custody Sep 10, 2026

Copy link
Copy Markdown
Contributor Author

Continuing the independent-custody correction on this existing owner branch; no second vault owner or consumer cutover. Source scope: new services/key_custody Rust internal library, executable quorum/seal/context-tamper tests, and a non-secret native verification job in the existing repository CI. Existing #129/#151/#154 source deltas remain untouched. The full open-PR inventory showed no native custody implementation; #128 also proposes ADR-0017, so this increment uses a uniquely named specification/plan rather than allocating another conflicting ADR number.

First deliverable is the internal custody barrier: OS-random root, threshold recovery shares via an existing Shamir implementation, authenticated encrypted root record, sealed-by-default recovery, and context-bound record protection without a root-key export API. It is not a remote KMS, workload authenticator, durable audit service, database credential issuer, or production rollout. External KMS and static credential files are not required by this library.

The local container has no Rust toolchain and cannot resolve download hosts. Rust execution must therefore be obtained from actual hosted CI; no compile/test success will be inferred from source inspection. The initial dependency lock and RED fixture execution are preparation only. Keep Draft; do not merge a lock-bootstrap or deliberately RED revision. No existing credentials, deployments, protections or other writers' refs are modified.

Preserve #153's complete transport/custody-correction history. Add a deliberately unavailable native scaffold and an initialization assertion, with a contents-read-only job in existing CI that can execute on Draft. This is not a production custody implementation and must not merge.

Local Rust toolchain/network is unavailable. Hosted execution must establish the actual RED and generate the initial Cargo lock; remove the temporary candidate-lock preparation after recording the lock. No source-writing workflow, deployment, credential access or verification claim.
@seonghobae seonghobae changed the title fix(deploy): preserve transport hardening; require independent Keyverse custody feat(custody): implement independent Rust custody beneath the staged migration Sep 10, 2026
…records

Preserve the complete #153 lineage and add an internal, unpublished custody component. Initialize an OS-random root with separate recovery shares, authenticate before unseal, reseal by dropping held root material, and protect records with exact instance/tenant/environment/namespace/key/version/purpose binding. There is no public root export, network listener, environment credential source or external KMS dependency.

Authored verification: 17 behavioral test functions plus an independent libsodium ciphertext interoperability case. Native Rust compilation, execution, coverage, Clippy and rustdoc are NOT verified: the test-first scaffold job remained queued and was not observed RED before this implementation. The independently executed local C reference roundtrip, mutation and AAD checks validate only the test oracle, not this Rust composition. Public source blob hashes match the local files and local new-file whitespace validation passes.

A real Cargo.lock is still required. The Draft CI candidate-resolution step must be removed after capturing and committing the real lock; its final committed-lock gate deliberately prevents an untracked lock from producing acceptance. Keep Draft. Workload authentication/authorization, enrollment, durable audit/persistence, anti-rollback, key/credential lifecycle and PostgreSQL acceptance remain release gates. No consumer cutover, deployment, secret access, merge or release.
@seonghobae seonghobae changed the title feat(custody): implement independent Rust custody beneath the staged migration feat(custody): add independent Rust quorum barrier; keep migration staged Sep 10, 2026

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Please review exact head 8418d8bde90ad65a3c4b98aeccee830426f34f14 as unverified internal custody source, not a deployable KMS. Focus on the actual Rust dependency APIs, Shamir share indexing/metadata and recovery failure atomicity, AEAD header/context framing, sealed-state invariants, allocation bounds, secret-copy/zeroization limits, and the independent C ciphertext fixture. Confirm the native job cannot pass while Cargo.lock is untracked.

There is no native compilation/test result yet and no claim of a completed RED→GREEN cycle. Workload authentication, authorization, durable audit, anti-rollback and lifecycle are not implemented; do not treat cryptographic context binding as authorization. Keep Draft and the existing owner/predecessor deltas. This is a review request, not an approval or merge authorization.

@coderabbitai

coderabbitai Bot commented Sep 10, 2026

Copy link
Copy Markdown

Oops, something went wrong! Please try again later. 🐰 💔

@seonghobae seonghobae added the enhancement New feature or request label Sep 12, 2026 — with ChatGPT Codex Connector
@seonghobae seonghobae added the priority: medium Normal-priority or P2 work label Sep 12, 2026 — with ChatGPT Codex Connector
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request priority: medium Normal-priority or P2 work

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant