Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 0 additions & 28 deletions .env.example

This file was deleted.

33 changes: 33 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -80,3 +80,36 @@ jobs:
env:
IDP_DB_PASSWORD: ci-placeholder
IDP_BOOTSTRAP_ADMIN_PASSWORD: ci-placeholder

key-custody-tests:
if: ${{ github.event_name != 'pull_request' || github.event.action != 'closed' }}
runs-on: ubuntu-24.04
defaults:
run:
working-directory: services/key_custody
steps:
- name: Checkout exact native source
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
persist-credentials: false
- name: Install pinned native toolchain
run: rustup toolchain install 1.97.1 --profile minimal --component rustfmt --component clippy
- name: Resolve initial candidate dependency lock
run: |
cargo +1.97.1 generate-lockfile
printf '\nBEGIN_CUSTODY_CANDIDATE_LOCK\n'
cat Cargo.lock
printf '\nEND_CUSTODY_CANDIDATE_LOCK\n'
- name: Execute native custody contract
run: cargo +1.97.1 test --locked --all-targets
- name: Reject native lint warnings
run: cargo +1.97.1 clippy --locked --all-targets -- -D warnings
- name: Validate native API documentation
env:
RUSTDOCFLAGS: -D warnings
run: cargo +1.97.1 doc --locked --no-deps
- name: Require reviewed committed dependency lock
run: |
git ls-files --error-unmatch Cargo.lock
git diff --exit-code -- Cargo.lock
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ Keep a Changelog, and releases use semantic versioning.

### Added

- ADR-0017 and a bounded Keycloak secret entrypoint define the root-only
supervisor/KMS bootstrap exception without creating a general application
secret-file fallback.
- ADR-0008 and the non-fork RP authorization matrix, requiring explicit
Keyverse token validation, tenant/resource ABAC, bounded RBAC, and
cross-tenant acceptance evidence per application.
Expand Down Expand Up @@ -55,6 +58,11 @@ Keep a Changelog, and releases use semantic versioning.

### Changed

- Standalone Compose no longer uses a repository-local `.env` credential
template. PostgreSQL consumes its root bootstrap password through `_FILE`,
while Keycloak reads exactly three supervisor/KMS-mounted bootstrap files at
its final process boundary. Non-secret deployment configuration remains
separate from Keyverse secret custody.
- Federation PUT and apply now report `applied_to_keycloak: true` only after a
fresh live Keycloak identity-provider observation matches the desired
observable representation. Keycloak's fixed mask for the known
Expand Down Expand Up @@ -111,6 +119,9 @@ Keep a Changelog, and releases use semantic versioning.

### Fixed

- Removed the standalone `.env.example` credential path so root bootstrap
credentials cannot silently become a reusable dotenv authority for CWL
consumers.
- Prevented relying-party inventory from silently accepting a KV key/body
identity mismatch, rejected unsafe live or `Location`-derived client UUIDs,
and aligned exact client discovery with Keycloak's documented
Expand Down
72 changes: 48 additions & 24 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@ authenticator is removed from the login flow). cwl-idp is the hub; employer and
corporate identity systems are external deployment data and compatibility
targets, never the hub.

Keyverse is also the canonical CWL secret-lifecycle owner. Identity,
authorization, ordinary configuration, and secret custody remain separate
bounded contexts even though they live under one product authority.

## Common commands

Make targets work with Docker or Podman (`COMPOSE="podman compose" make up`):
Expand All @@ -25,17 +29,18 @@ make up # bring up Keycloak + Postgres + admin service
make down # tear down while retaining volumes
make logs # follow logs
make ready # poll readiness (deploy/scripts/healthz.sh)
make install # install the admin service development environment
make test # run account-unification unit tests
make lint # run Ruff + interrogate docstring coverage
make validate-realm # validate deploy/keycloak/realm-cwl.json
make seed-bootstrap # create a local SQLite KV bootstrap store
make install # install the admin service development environment
make test # run account-unification unit tests
make lint # run Ruff + interrogate docstring coverage
make validate-realm # validate deploy/keycloak/realm-cwl.json
make seed-bootstrap # create a local SQLite configuration bootstrap store
```

Compose bring-up needs `.env` (from `.env.example`) and
`deploy/bootstrap/bootstrap.yaml` (from `bootstrap.example.yaml`). Keycloak
console: `http://localhost:8080`; admin service:
`http://localhost:8099/healthz`.
Compose bring-up does **not** use `.env`. A trusted supervisor/KMS adapter first
materializes the three root-bootstrap files documented in `README.md` under
`/run/keyverse-bootstrap`; `deploy/bootstrap/bootstrap.yaml` remains a
non-secret locator/configuration descriptor. Keycloak console:
`http://localhost:8080`; admin service: `http://localhost:8099/healthz`.

Per-service commands matching CI, from `services/account_unification/`:

Expand All @@ -58,6 +63,9 @@ export CWL_IDP_BOOTSTRAP=/path/to/bootstrap.yaml
uvicorn app.main:app --port 8099
```

`CWL_IDP_BOOTSTRAP` is a non-secret locator, not credential transport. Do not add
secret-valued environment variables or dotenv discovery to the account service.

## CI gates (`.github/workflows/ci.yml`)

1. **account-unification-tests** — locked dependencies, Ruff, 100% interrogate
Expand All @@ -70,8 +78,9 @@ uvicorn app.main:app --port 8099
registration and reset-password remain off; no external IdP or user-storage
federation may be committed; public RP access-token lifetime is bounded; real
client secrets are forbidden.
3. **compose-config-validates** — validates `docker-compose.yml` with placeholder
bootstrap passwords.
3. **compose-config-validates** — validates `docker-compose.yml` with the
supervisor/KMS bootstrap secret mounts. CI must not fabricate production
credential values merely to make Compose parsing succeed.

CodeQL, Semgrep, Security Scan, current-head review, and unresolved-thread gates
remain authoritative. `.clusterfuzzlite/` is a discovery marker; the fuzz
Expand All @@ -86,10 +95,15 @@ Three runtime containers run on two networks (`docker-compose.yml`; the Helm
chart has the same shape):

- **idp_database** — Postgres 17, Keycloak's system of record. Internal network
only.
only. In standalone Compose its password is read with PostgreSQL's `_FILE`
mechanism from the root-bootstrap secret mount.
- **idp_engine** — Keycloak 26, `start --import-realm`; imports the portable,
passwordless-first `cwl` realm. Health is exposed on management port 9000.
TLS terminates at the WAF edge, so HTTP is enabled internally.
TLS terminates at the WAF edge, so HTTP is enabled internally. Because
Keycloak consumes bootstrap credentials through native environment options,
`deploy/keycloak/secret-entrypoint.sh` reads only the three mounted bootstrap
files and immediately `exec`s Keycloak. This exception must not spread to
ordinary CWL product credentials.
- **account_unification_service** — FastAPI admin service (Python ≥3.11) on port
8099. It talks to Keycloak only through the Admin REST API using a confidential
service-account client. It provides account inspect/link/merge, inbound SCIM,
Expand All @@ -106,27 +120,37 @@ is required by the normal suite.

### Deployment layout

- `deploy/keycloak/` — portable realm config-as-code and
`kcadm-bootstrap.sh`. The realm contains no employer-specific federation.
- `deploy/keycloak/` — portable realm config-as-code, `kcadm-bootstrap.sh`, and
the narrow Compose bootstrap adapter. The realm contains no employer-specific
federation.
- `deploy/templates/` — explicit private deployment contracts. SAML/OIDC use
Keyverse desired-state endpoints. `oidc-rp-naruon.json` is the reviewed public
Naruon runtime RP profile with one audience mapper and bounded routing claims.
LDAP is preflighted through Keyverse and then applied through private Keycloak
Admin REST in this release. All `{{placeholders}}` are resolved from KV before
use.
- `deploy/bootstrap/` — the bootstrap pointer locating the KV/DB config store.
Admin REST in this release. Private placeholders are resolved by the approved
deployment secret authority before use; do not introduce dotenv as an
intermediate store.
- `deploy/bootstrap/` — non-secret bootstrap pointer locating the typed
configuration store.
- `helm/cwl-idp/` — the same three components; Keycloak and Postgres may be
disabled in favor of externally managed services. Secrets come from
pre-created Kubernetes secrets populated from KV.
disabled in favor of externally managed services. Bootstrap secrets come from
pre-created Kubernetes Secret objects populated by a deployment secret
controller/KMS integration, never repository-local dotenv files.
- The repository is **standalone AND submodule-embeddable**: a parent compose can
`include:` `docker-compose.yml`, or depend on `helm/cwl-idp`.

## Key conventions

- **Config and secrets come from the KV/DB store, never runtime `os.getenv`.**
Environment variables are bootstrap transport only. The admin service reads
`CWL_IDP_BOOTSTRAP`, which points at the bootstrap file and then the typed KV
configuration.
- **Keyverse owns CWL application secret lifecycle; typed configuration is not a
secret store.** Ordinary consumers use only an immutable released Keyverse
workload-resolution contract and never query Keyverse persistence directly.
- **No dotenv credential authority.** Do not add `.env`, home-directory dotenv
discovery, `env_file`, or secret-valued environment fallback. Non-secret
deployment settings and locators may remain explicit typed configuration.
- **Root bootstrap is a narrow exception.** Keyverse cannot obtain the secrets
required to start its own database/Keycloak engine from its locked API. The
supervisor/KMS mount is allowed only for those bootstrap inputs and must
converge toward managed workload identity plus external KMS/HSM custody.
- **SAML/OIDC federation is desired state.** Validate registrations through
`POST /federation/identity-providers:validate`, persist with `PUT`, and
converge through the federation service. Preflight must not write, call
Expand Down
74 changes: 54 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,16 +34,16 @@ account-unification admin service, the product:

> Employer ADFS and corporate directories are **external compatibility
> targets**, not peer hubs. Customer-specific federation stays in the
> deployment controller and KV store.
> deployment controller and Keyverse-owned secret/configuration boundaries.

OAuth 2.0 ([RFC 6749](https://www.rfc-editor.org/rfc/rfc6749)) is the official
authorization-framework record. [OAuth 2.1](https://datatracker.ietf.org/doc/draft-ietf-oauth-v2-1/)
is an IETF Internet-Draft (`draft-ietf-oauth-v2-1-15`, work in progress) and
is not cited here as a final RFC.

RP client registrations and confidential values live in the **IdP DB / KV**,
never in an RP's environment. Authorized identity data stays usable under
purpose-bound access control, encryption, and audit.
RP client registrations and confidential values live in the **IdP / Keyverse
secret boundary**, never in an RP's dotenv file. Authorized identity data stays
usable under purpose-bound access control, encryption, and audit.

## Architecture

Expand All @@ -65,17 +65,39 @@ Trust boundaries: [`ARCHITECTURE.md`](ARCHITECTURE.md). Network diagram:

## Run this repository alone

No sibling repository checkout is required. Docker or Podman with the compose
plugin is enough:
No sibling repository checkout is required. Docker or Podman with the Compose
plugin is enough, but Keyverse deliberately does not bootstrap itself from a
repository-local `.env` file. A trusted supervisor, host credential agent, or
KMS/HSM adapter must first materialize the three **root-bootstrap-only** values
below as private files outside the repository:

```text
/run/keyverse-bootstrap/idp_database_password
/run/keyverse-bootstrap/idp_bootstrap_admin_username
/run/keyverse-bootstrap/idp_bootstrap_admin_password
```

These files exist only to break Keyverse's own bootstrap cycle. They are not the
credential distribution mechanism for other CWL products. Do not commit them,
copy them into a bootstrap YAML, or expose them in shell arguments, logs,
artifacts, screenshots, or model context.

Then create the non-secret account-service bootstrap descriptor and start the
stack:

```bash
cp .env.example .env # populate values from your KV (bootstrap transport)
cp deploy/bootstrap/bootstrap.example.yaml deploy/bootstrap/bootstrap.yaml

docker compose up -d # or: podman compose up -d
./deploy/scripts/healthz.sh # waits for Keycloak realm + admin service to be READY
```

PostgreSQL receives its password through the image's `_FILE` contract. Keycloak
requires the database and one-time bootstrap-admin values in its native process
environment, so `deploy/keycloak/secret-entrypoint.sh` reads the mounted files at
the final container boundary and immediately `exec`s Keycloak. There is no
dotenv discovery or fallback.

- Keycloak console: `http://localhost:8080`
- Admin service health: `http://localhost:8099/healthz`

Expand All @@ -84,7 +106,12 @@ The stack imports the **passwordless-first** realm at first start
WebAuthn passwordless authenticator and **no password authenticator**, plus
`registrationAllowed:false` / `resetPasswordAllowed:false`.

Production-shaped clusters use [`helm/cwl-idp/`](helm/cwl-idp/).
Production-shaped clusters use [`helm/cwl-idp/`](helm/cwl-idp/). Their bootstrap
secret objects must likewise be populated by a deployment secret controller or
KMS integration, not hand-maintained dotenv files. The longer-term Key Vault
roadmap moves the root encryption key to managed workload identity plus external
KMS/HSM custody; ordinary application credentials resolve through the released
Keyverse workload API.

### Optional parent include

Expand Down Expand Up @@ -124,11 +151,11 @@ returned in ordinary Keyverse responses. See
### Register external federation

The portable realm contains no employer ADFS, LDAP/AD source, or other
customer-specific federation. Render deployment values from KV and preflight
every private payload before apply.
customer-specific federation. Resolve private values through the deployment
secret boundary and preflight every private payload before apply.

LDAP preflight redacts `bindDn` and `bindCredential` and must never be used
as the apply payload; apply the original private file only. The first
as the apply payload; apply the original private payload only. The first
directory profile is LDAPS-only, read-only, Kerberos-disabled, and
`trustEmail=false`.

Expand All @@ -144,10 +171,17 @@ Design: [`docs/merge-unification-flow.md`](docs/merge-unification-flow.md).

## Configuration and secrets

Config and secrets are read from the **KV / DB store**, not from runtime
`os.getenv`. Environment variables are **bootstrap transport** only
(`CWL_IDP_BOOTSTRAP` → `deploy/bootstrap/bootstrap.yaml`). Database objects
use two-word-or-longer snake_case names (`idp_config_entries`,
Non-secret configuration and bootstrap locators remain typed configuration.
Application credentials are not configuration: Keyverse is the canonical CWL
secret-lifecycle owner, and consumers must adopt only an immutable released
workload-resolution contract. The account service's `CWL_IDP_BOOTSTRAP` value is
a non-secret locator to `deploy/bootstrap/bootstrap.yaml`, not a secret value.

Keyverse's own root bootstrap cannot depend on the locked Keyverse API. Compose
therefore uses the three protected files described above as a narrow
self-bootstrap exception. It does **not** restore `.env`, a plaintext config DB,
or consumer-side secret files as fallback authorities. Database objects use
two-word-or-longer snake_case names (`idp_config_entries`,
`account_merge_audit`, `user_operation_lock_state`).

## Engine and licensing
Expand All @@ -160,8 +194,8 @@ use two-word-or-longer snake_case names (`idp_config_entries`,

| Path | What |
| --- | --- |
| [`docs/adr/`](docs/adr/README.md) | Accepted architecture decisions (0001–0008 on this branch) |
| [`docs/REFERENCES.md`](docs/REFERENCES.md) | APA 7th bibliography for ADR 0001–0007 |
| [`docs/adr/`](docs/adr/README.md) | Architecture decisions and proposed changes |
| [`docs/REFERENCES.md`](docs/REFERENCES.md) | APA 7th bibliography |
| [`docs/doctoring/`](docs/doctoring/) | Feature-specific standards interpretation |
| [`docs/product-technical-gap-baseline.md`](docs/product-technical-gap-baseline.md) | Current buyer-visible product and technical gap register |
| [`docs/papers/`](docs/papers/README.md) | Offline copies of selected primary sources |
Expand All @@ -174,10 +208,10 @@ use two-word-or-longer snake_case names (`idp_config_entries`,

| Path | What |
| --- | --- |
| `docker-compose.yml` | Standalone bring-up: Keycloak + Postgres + admin service (pinned by digest) |
| `deploy/keycloak/` | Portable Keycloak realm config-as-code and service-account bootstrap |
| `docker-compose.yml` | Standalone bring-up with protected root-bootstrap mounts |
| `deploy/keycloak/` | Portable realm config and bounded Keycloak bootstrap adapter |
| `deploy/templates/` | Private deployment templates for preflight and desired state |
| `deploy/bootstrap/` | Bootstrap pointer to the KV/DB config store |
| `deploy/bootstrap/` | Non-secret bootstrap pointer to the account-service config store |
| `deploy/scripts/healthz.sh` | Cross-component readiness probe |
| `scripts/validate_realm.py` | Realm config-as-code validator |
| `services/account_unification/` | FastAPI admin service (link, merge, SCIM, federation, RP desired state) |
Expand Down
Loading
Loading