diff --git a/AGENTS.md b/AGENTS.md index 4021d92d..0f9528b0 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,87 +1,118 @@ # LifeOS agent contract -This file is the canonical repository-wide operating contract for coding agents. `ARCHITECTURE.md` defines durable system boundaries, while feature specifications, implementation plans, and runbooks provide scoped detail. +This file is the canonical repository-wide operating contract for coding agents. `ARCHITECTURE.md` defines durable system boundaries; `docs/PRD.md` and `docs/TRD.md` define canonical product/technical requirements; ADRs, data/API/UML/privacy/threat/test/operability/release/standards/traceability docs and scoped feature specifications/runbooks provide detail without weakening those boundaries. -## Pull-request loop +## Work-conserving pull-request loop For every open pull request: -1. inspect the exact current head; -2. read every human, CodeRabbit, AppGuardrail, code-scanning, and security finding; -3. diagnose the root cause of failed or required checks; -4. make a complete correction with tests and documentation; -5. rerun or wait for checks on the corrected exact head while continuing independent work; -6. resolve only review threads whose underlying issue is addressed; -7. merge only when all required evidence passes and no actionable review finding remains; -8. continue with the next buyer-visible development slice. +1. refetch the exact current contributor head and exact live base tip; +2. read every human, CodeRabbit, AppGuardrail, code-scanning, security and other configured finding; +3. diagnose the root cause of every failed, missing, stale or required gate; +4. verify the narrowest remedy is operationally real before relying on it; +5. make a complete test-first causal correction with required documentation/cleanup; +6. rerun/inspect required checks on the corrected exact head; +7. resolve only review threads whose underlying issue is actually addressed; +8. merge only when repository policy accepts the exact unchanged head; +9. immediately continue with the next safe PR/review/cleanup/product/documentation/release-readiness action while run budget remains. -Never use an administrative bypass or claim completion from stale checks. Routine progress narration is not repository evidence. +A queued check, unavailable provider, Draft state, failed first tool path, reviewer delay or writer conflict blocks only the affected action. Do not spend an otherwise productive run narrating or repeatedly proving an unchanged blocker. + +Never use administrative bypass, fabricate approval/evidence, or claim completion from stale/predecessor/synthetic-merge-only checks. + +## Writer lease + +Before a branch-affecting write, refetch exact head/base/ref/blob state. If another source writer moves the same target, discard stale assumptions and freeze writes to that target for the run while continuing non-conflicting review, RCA, testing, documentation, dependency, product-gap or operability work. A writer conflict is a concurrency condition, not repository-wide completion. ## Code-owner review gates — disabled on hold -As of 2026-08-04, code-owner review requirements (`require_code_owner_reviews` in branch protection and `require_code_owner_review` in rulesets) are disabled across the ContextualWisdomLab organization because there is one maintainer and that gate cannot be satisfied. Do not re-enable CODEOWNERS-based merge gates until the organization has multiple maintainers. Independent automated review, security checks, and exact-head verification remain required where configured. +As of 2026-08-04, code-owner review requirements (`require_code_owner_reviews` in branch protection and `require_code_owner_review` in rulesets) are disabled across the ContextualWisdomLab organization because there is one maintainer and that gate cannot be satisfied. Do not treat this historical repository setting as permission to fabricate an approval or bypass whatever independent review/security checks are actually configured now. Refetch live policy before merge. ## Modular MSA rules -- Every bounded service must run independently and remain composable in the LifeOS monorepo deployment. +- Every bounded service runs independently and remains composable in the LifeOS monorepo deployment. - Services communicate through versioned HTTP, event, saga, plugin, or MCP contracts. - A service must not read or mutate another service's database tables. -- Each service owns migrations, runtime configuration, observability, tests, and shutdown behavior. -- Internal identifiers are opaque UUIDv4 strings. Numeric provider identifiers never become internal primary keys. -- Database objects use names containing at least two words, preferably `snake_case`, unless an external protocol mandates another form. -- Rename stale internal product or caller names when they no longer match the public software identity. - -## Quality and documentation - -- Production declarations require explanatory docstrings sufficient for a new contributor to understand the contract without reconstructing the implementation. -- Packages that enforce coverage gates must retain 100% statement, branch, function, and line coverage. -- Tests prove realistic domain accuracy and failure behavior, not only mocked call counts. -- Standards, papers, and research claims are recorded in `docs/research/` or the approved feature specification with APA 7 references and clear final/draft/preprint status. -- Update `ARCHITECTURE.md`, `CLAUDE.md`, `CHANGELOG.md`, capability evidence, design specifications, implementation plans, and operating runbooks when their boundary changes. -- A release version and tag are created only when the repository proves release readiness; otherwise changes remain under `CHANGELOG.md` → `Unreleased`. +- Each service owns migrations, runtime configuration, persistence adapters, observability, tests, and shutdown behavior. +- Internal identifiers are opaque UUIDv4 strings. Numeric/provider-native identifiers never become internal primary keys. +- Database objects use descriptive multiword `snake_case` names unless an external protocol mandates another spelling. +- Browser-local drafts/caches are not durable product truth until an authorized owning service confirms persistence. +- Rename stale internal product/caller names when they no longer match public software identity, preserving compatibility/migration evidence. + +## Canonical documentation and status + +The repository must be understandable without reconstructing chat history, old PR bodies, or assistant output. + +Canonical documentation graph: + +1. `docs/PRD.md` — product outcomes, users, requirements and status. +2. `docs/TRD.md` — shared technical/runtime/security/release requirements. +3. `ARCHITECTURE.md` — durable bounded contexts and authority. +4. `docs/adr/README.md` — material decisions/supersession history. +5. `docs/DATA_MODEL.md` — logical service-owned ERD/data model. +6. `docs/UML.md` — component/sequence/state/deployment/failure views. +7. `docs/API_CONTRACTS.md` — API/event/provider ownership and evolution. +8. `SECURITY.md` and `docs/THREAT_MODEL.md` — reporting policy versus architecture threats. +9. `docs/PRIVACY_DATA_LIFECYCLE.md` — sensitive-data lifecycle, retention, export and erasure boundaries. +10. `docs/TEST_STRATEGY.md` — deterministic/live quality evidence. +11. `docs/OPERABILITY.md` — deployment/diagnostics/backup/recovery ownership. +12. `docs/RELEASE_AND_MIGRATION.md` — versioning/migration/compatibility/rollback contract. +13. `docs/STANDARDS_TRACEABILITY.md` — standards/research source class and product-evidence mapping. +14. `docs/TRACEABILITY.md` — requirement/decision/capability → source/test/runbook/gap evidence. +15. `docs/DOCUMENTATION_ASSESSMENT.md` — completeness and historical reconciliation. +16. `docs/operations/`, `docs/research/`, `docs/legal/`, `docs/superpowers/specs/`, `docs/superpowers/plans/` — scoped supporting evidence. +17. `CHANGELOG.md` — buyer-visible unreleased/released changes. + +Use exact statuses: `Implemented on protected main`, `Implemented on active PR`, `Partial`, `Accepted architecture`, `Planned`, `Research only`, `Superseded`, `Out of scope`. + +Never describe target architecture as shipped behavior. Protected-main source/migrations/tests outrank stale prose. The 2026-08-02 combined LifeOS design is historical input where canonical docs/ADRs supersede it. + +When a documentation audit finds a real implementation gap, continue into the smallest executable test/code/migration/API/UX task when safe; documentation completion is not a terminal product result. + +## Quality and testing + +- Production declarations have explanatory docstrings sufficient for a new contributor to understand the contract without reconstructing implementation. +- Packages enforcing exact coverage retain 100% statement, branch, function, and line coverage with meaningful assertions. +- Tests prove realistic domain accuracy, tenant isolation, concurrency/replay, failure/recovery and customer journeys rather than mocked call counts only. +- Use real PostgreSQL integration evidence for material persistence semantics. +- Deterministic merge gates remain separate from bounded live-provider conformance. +- Standards/papers/research claims are recorded in `docs/research/` or approved feature specs with APA 7 references and publication status and indexed from `docs/STANDARDS_TRACEABILITY.md` when repository-wide decisions depend on them. +- Update relevant canonical docs, scoped specs/plans/runbooks and `CHANGELOG.md` when a behavior/authority boundary changes. +- A release version/tag is created only when exact integrated release readiness is proven; otherwise changes remain under `CHANGELOG.md` → `Unreleased`. ## AI and model-provider rules - AI proposals are inert, explainable suggestions and cannot silently mutate user-owned data. -- `COPILOT_GITHUB_TOKEN` is prohibited. -- Model-assisted tests and scheduled agents use `NVIDIA_NIM_API_KEY` through the approved OpenCode or contextual-orchestrator boundary. -- Do not alter or reuse the key scheme of existing review agents. -- Provider credentials, browser cookies, bearer material, raw prompts, raw responses, hidden reasoning, and stack traces do not enter retained artifacts. -- Live-provider availability is not a deterministic pull-request merge requirement; missing or unavailable providers produce explicit sanitized evidence. +- Model-assisted tests/development use `NVIDIA_NIM_API_KEY` through the approved OpenCode or contextual-orchestrator boundary where model access is required. +- Do not alter/reuse the credential scheme of existing independent review agents merely to make a development agent work. +- Provider credentials, browser cookies, bearer material, raw prompts/responses, hidden reasoning and stack traces do not enter retained public artifacts. +- Live-provider availability is not a deterministic PR merge requirement unless a separately reviewed gate explicitly requires it; missing/unavailable providers produce sanitized evidence. +- The bounded hourly OpenCode commercial-development workflow is protected-main product-development automation after PR #122; it remains subject to deterministic policy and normal review/security/exact-head merge gates and does not gain product-data authority. ### Test-time compute allocation -A strong single-model route is the mandatory baseline. Deeper orchestration is justified only by measured quality or heterogeneous capability coverage. Explicitly model and ablate: - -- reasoning effort; -- workflow stages; -- planner, worker, verifier, and synthesizer roles; -- task decomposition; -- recursive depth; -- access lists and communication topology; -- homogeneous versus heterogeneous model pools. - -Fugu release evidence (final product release and technical report; Fugu Team, 2026), Conductor (peer-reviewed ICLR 2026 conference paper; Nielsen et al., 2026), TRINITY (peer-reviewed ICLR 2026 conference paper; Xu et al., 2026), and strong-single-agent evidence (arXiv preprint and ICLR 2026 submission; Xu et al., 2026) guide the design, but repository tests and retained measurements determine the deployed policy. Latency is recorded but is not the sole or primary decision criterion. Complete APA 7 references and publication-status links are maintained in [`docs/superpowers/specs/2026-08-06-ai-nim-live-conformance-design.md`](docs/superpowers/specs/2026-08-06-ai-nim-live-conformance-design.md#references). +A strong single-model route is the mandatory baseline. Deeper orchestration is justified only by measured quality/control benefit or heterogeneous capability coverage. Explicitly model/ablate reasoning effort, workflow stages, planner/worker/verifier/synthesizer roles, task decomposition, recursive depth, access lists/topology and model-pool composition. Latency/token use are capacity evidence rather than the sole decision criterion. ## Mathematical and psychometric modules -Any future mathematical or psychometric computation layer must: +Any future production mathematical/psychometric computation layer must: - implement numerical kernels in Rust; -- support deterministic CPU multithreading with low context switching and a GPU execution boundary; -- test true-parameter recovery, bias, interval coverage, convergence, and RMSE on realistic simulations; -- model multilevel and multiple-membership structure to avoid atomistic fallacy; -- model temporal change, repeated measurement, drift, or state evolution where the estimand changes over time; -- document assumptions, estimands, numerical precision, fallback behavior, and reproducibility controls with APA 7 references. +- support deterministic low-context-switch CPU multithreading and a parity-verified GPU boundary where justified; +- test true-parameter recovery, bias, interval coverage, convergence and RMSE; +- model multilevel/multiple-membership structures where required to avoid invalid aggregation; +- model temporal change/repeated measurement/drift/state evolution where the estimand changes over time; +- document estimands, assumptions, precision, seed/convergence/fallback behavior with APA 7 references. ## Security and privacy -- Treat every external response, stored JSON value, environment value, model output, and connector result as untrusted until bounded and validated. +- Treat external responses, stored JSON, environment values, model output, connector results, calendar/plugin payloads and database rows as untrusted until bounded and validated. - Keep SQL structure static and parameterize dynamic values. -- Fail closed on malformed ownership, identifiers, signatures, digests, timestamps, pagination, or provider configuration. -- Public problems, metrics, logs, and artifacts are credential-free and bounded. -- Temporary write-capable repair workflows must be removed before merge; persistent workflows receive the least permissions needed. +- Fail closed on malformed ownership, identifiers, signatures, digests, timestamps, pagination, provider configuration and privileged grants. +- Public problems, metrics, logs and artifacts are credential-free/bounded and avoid unnecessary personal content. +- Sensitive access is purpose/resource/actor scoped with auditable evidence where privileged access exists; indiscriminate masking is not a substitute for authorization. +- Temporary repair scaffolding is removed after durable evidence is secured unless it is intentionally accepted product automation. ## Waiting and escalation -Waiting for checks, reviews, or a long-running OpenCode agent is not a blocker. Continue non-conflicting analysis, documentation, testing, or the next planned slice. Escalate only when a product decision or permission cannot be derived from repository policy, evidence, standards, or available tools. +Waiting for checks, reviews or a long-running model/agent is not by itself a blocker. Continue non-conflicting analysis, tests, documentation, cleanup, buyer-gap or operational work. Escalate only when a product/governance/permission/secret decision cannot be resolved from live repository policy, evidence, standards and realistically available tools and no other safe work remains for the finite run. diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index e4c9a054..b98ce2a2 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1,10 +1,16 @@ # LifeOS architecture decisions -This document is the architectural source of truth for repository-wide boundaries. Feature-level specifications and runbooks may add detail, but they must not weaken these decisions. +This document is the architectural source of truth for repository-wide boundaries. `docs/PRD.md` defines product outcomes, `docs/TRD.md` defines shared technical requirements, and feature-level specifications/runbooks add scoped detail without weakening these decisions. + +## 0. Architecture status and history + +LifeOS is a multi-user, server-backed, self-hostable modular MSA product. Historical proposals for a login-free browser-only local-first product, a single-application primary architecture, and UUIDv7 internal identifiers are retained as rationale but are **superseded** by current protected-main decisions. See `docs/DOCUMENTATION_ASSESSMENT.md` and `docs/adr/`. + +Use these status meanings throughout canonical docs: `Implemented on protected main`, `Implemented on active PR`, `Partial`, `Accepted architecture`, `Planned`, `Research only`, `Superseded`, and `Out of scope`. ## 1. Product and deployment boundary -LifeOS is a modular, self-hostable personal operating system. Every bounded service must work independently and remain composable inside the monorepo deployment. Services communicate through versioned HTTP/event contracts and never read another service's database tables directly. +Every bounded service must work independently and remain composable inside the monorepo deployment. Services communicate through versioned HTTP/event/saga/plugin/MCP contracts and never read or mutate another service's database tables directly. ```mermaid flowchart LR @@ -17,33 +23,57 @@ flowchart LR G --> A[AI proposal service] G --> C[Calendar integration service] G --> X[Plugin integration service] + G --> V[Privacy access service] + P -. domain events .-> N[(NATS JetStream)] H -. domain events .-> N - R -. domain events .-> N - subgraph Data ownership - IDB[(Identity PostgreSQL schema)] - PDB[(Planning PostgreSQL schema)] - HDB[(Habit PostgreSQL schema)] - ADB[(AI audit PostgreSQL schema)] - NDB[(Notification PostgreSQL schema)] - end - I --> IDB - P --> PDB - H --> HDB - A --> ADB + R -. projection/events .-> N + N -. reminder/event inputs .-> D[Notification service] + + I --> IDB[(Identity-owned PostgreSQL)] + P --> PDB[(Planning-owned PostgreSQL)] + H --> HDB[(Habit-owned PostgreSQL)] + R --> RDB[(Review-owned PostgreSQL / projections)] + A --> ADB[(AI audit PostgreSQL)] + D --> NDB[(Notification PostgreSQL)] + V --> VDB[(Privacy access PostgreSQL)] + + C --> CP[Google Calendar / CalDAV] + I --> IP[Google / GitHub identity] + A --> MP[Local rule model or approved model boundary] ``` +The diagram is a logical topology. Cross-service arrows never imply cross-service database authority. + ### Required invariants -- Internal object identifiers are opaque UUIDv4 strings. Numeric provider identifiers are never reused as internal primary keys. +- Internal object identifiers are opaque UUIDv4 strings. Numeric/provider-native identifiers never become internal primary keys. - Database object names contain at least two words and use `snake_case` unless an external standard requires another form. -- Each service owns migrations, runtime configuration, persistence adapters, tests, and shutdown behavior. -- Cross-service writes require an explicit API, event, saga, or plugin contract; shared-table coupling is prohibited. +- Each service owns migrations, runtime configuration, persistence adapters, tests, observability, and shutdown behavior. +- Cross-service writes require an explicit API, event, saga, plugin, or MCP contract; shared-table coupling is prohibited. +- Browser-local state is draft/cache/offline state unless an authorized owning service confirms durable persistence. - Public errors, metrics, logs, artifacts, and review evidence exclude credentials and unbounded tenant data. +- External provider/model/plugin responses are untrusted until bounded and validated. + +## 2. Data authority and consistency + +PostgreSQL is the durable source of truth for persisted domain records. A development/self-hosted deployment may place multiple service-owned schemas/databases on one cluster, but logical ownership and credentials remain separated. -## 2. AI proposal safety boundary +Shared `workspace_id`/`actor_id` values are correlation and authorization inputs, not permission to query another service's tables. Cross-service derived views are projections and cannot silently become mutation authority. -AI output is an inert proposal, not an execution command. The AI service can generate, persist, retrieve, and record explicit decisions about proposals, but it has no planning mutation repository or generic command bus. +Concurrency-sensitive flows use explicit revision/digest/ETag/idempotency/fencing evidence appropriate to the domain. Current examples include habit completion replay protection, notification worker claims/outcomes, calendar preconditions, AI proposal revision/digest decisions, and privacy grant consumption. + +See `docs/DATA_MODEL.md` for the logical ERD and `docs/API_CONTRACTS.md` for owned protocol/version boundaries. + +## 3. Identity, tenant, and private service context + +Google and GitHub are the required login providers in the current product boundary. Provider identity is mapped to LifeOS-owned UUIDv4 identities and workspace authority. Downstream services derive ownership from authenticated/signed context instead of trusting browser-selected workspace/actor headers. + +Where a private service context is signed, key identifier, method, path, workspace, actor, and issuance/lifetime are integrity protected. Key rotation supports an explicitly bounded active/previous overlap where implemented. Browser credentials and provider keys do not become generic downstream inputs. + +## 4. AI proposal safety boundary + +AI output is an inert proposal, not an execution command. The AI service can generate, validate, persist, retrieve, evaluate and record explicit decisions about proposals, but it has no planning mutation repository or generic command bus. ```mermaid sequenceDiagram @@ -57,18 +87,38 @@ sequenceDiagram Web->>Identity: Validate session Identity-->>Web: Workspace UUIDv4 + actor UUIDv4 Web->>AI: Signed method/path/tenant/actor context - AI->>AI: Validate bounded request and model output + AI->>AI: Validate bounded request and untrusted model output AI->>Audit: Persist immutable proposal evidence - Audit-->>AI: Recorded digest evidence - AI-->>Web: Inert proposal requiring confirmation + Audit-->>AI: Recorded revision/digest evidence + AI-->>Web: Inert proposal requiring user decision Web-->>Browser: Credential-free response ``` -The signed private context uses one active HMAC key and at most one previous verification-only key. Key identifiers, method, path, workspace, actor, and issuance time are integrity protected. Browser credentials and provider keys never reach the AI service. +Explicit accept/reject history is append-only/replay-safe evidence. Any future capability that applies an accepted proposal to planning data requires its own narrowly authorized domain command contract; proposal acceptance alone is not mutation authority. + +## 5. Purpose-bound privacy access -## 3. Test-time compute and live conformance +Sensitive personal data cannot be governed by masking alone. The privacy bounded context authorizes access by actor, workspace/resource, purpose, lifetime and exact operation, records append-only access decisions/events, and may issue bounded/single-use signed grants. -The deterministic proposal evaluator is authoritative for proposal validity, operation conformance, grounding, benign utility, forbidden-text leakage, and prompt-injection resistance. Live provider execution is governance evidence and is not a pull-request availability gate. +This keeps sensitive information useful for authorized product operations while minimizing standing privilege and retained audit payloads. Expired, reused, wrong-purpose, wrong-resource, or wrong-actor grants fail closed. + +See `docs/adr/0005-purpose-bound-sensitive-data-access.md`, `docs/THREAT_MODEL.md`, and `docs/PRIVACY_DATA_LIFECYCLE.md`. + +## 6. Notification and calendar boundaries + +The notification service owns reminder occurrence/claim/outcome/in-app delivery persistence and timezone/fatigue/retry rules. Worker claims are bounded and recoverable; outcomes are immutable evidence where the persistence contract requires it. + +The calendar integration service owns provider adaptation. CalDAV/Google writes use deterministic identifiers/preconditions where supported and return classified, credential-free conflict/unavailable evidence. Provider state is untrusted. Hosted per-user Google Calendar credential storage/refresh/revocation/provider selection remains **Partial** and is tracked by issue #129; an operator-supplied runtime token is not equivalent to a multi-user credential product. + +## 7. Plugin and external integration boundary + +The plugin SDK/integration service exposes versioned manifest/event contract discovery, validation, and tenant-scoped preparation. It does not grant plugins direct database access. + +Generic installation, durable plugin secret storage, outbound webhook delivery, inbound arbitrary commands, and external network authority remain separately reviewed capabilities with least-privilege, SSRF, provenance, retry and audit requirements. + +## 8. Test-time compute and live conformance + +The deterministic proposal evaluator is authoritative for proposal validity, operation conformance, grounding, benign utility, forbidden-text leakage, and prompt-injection resistance. Live provider execution is governance evidence and is not a pull-request availability gate unless a separately reviewed gate says otherwise. ```mermaid flowchart TB @@ -90,38 +140,59 @@ flowchart TB - A strong single-model route is always measured first. - Reasoning effort, workflow stage, decomposition, recursion depth, role, and access topology are explicit test cells rather than hidden defaults. - Deeper orchestration is justified by measured fixture-level quality or heterogeneous capability coverage, not by agent count. -- Latency and token use are recorded for capacity review but are not the optimization objective. +- Latency/token use are recorded for capacity review but are not the sole optimization objective. - Unsupported capabilities remain explicit unavailable cells; tests never fabricate an ablation result. +- Live model evidence excludes provider credentials, prompts, raw responses, traces and hidden reasoning from retained artifacts. + +## 9. Mathematical and psychometric modules + +LifeOS currently contains no production psychometric computation service. Any future mathematical or psychometric module must before production treatment: + +- implement numerical kernels in Rust; +- support deterministic CPU multithreading with low context switching and a parity-verified GPU boundary where justified; +- test true-parameter recovery, bias, interval coverage, convergence, and RMSE on realistic simulations; +- model multilevel and multiple-membership structures where the estimand requires them; +- model temporal change/repeated measurement/drift/state evolution where the estimand changes over time; +- document numerical precision, seed control, convergence diagnostics and fallback behavior; +- cite substantive statistical assumptions/estimands in APA 7 style. + +## 10. Deployment and recovery boundary + +Docker Compose composes local/self-hosted development behavior. `infra/kubernetes` is a provider-neutral hardened production reference; it does not provision a cluster, database, NATS, ingress/TLS/DNS, registry pipeline, KMS/secret manager, or operator monitoring backend. -The hourly live workflow pins `ContextualWisdomLab/contextual-orchestrator` to an exact reviewed commit, installs hash-locked dependencies, seeds only `NVIDIA_NIM_API_KEY` through the encrypted credential bootstrap, executes the pinned checkout on loopback, and retains no prompts, responses, hidden reasoning, credentials, or raw traces. +Logical PostgreSQL backup/restore verifies archive checksum and requires a deliberately empty target. It is not point-in-time recovery. Deployment rollback claims cover only the workload state explicitly captured/verified by the deployment workflow; completed migrations and external infrastructure are not silently represented as reversible. -## 4. Mathematical and psychometric modules +See `docs/OPERABILITY.md` and `docs/RELEASE_AND_MIGRATION.md`. -LifeOS currently contains no psychometric computation service. Any future mathematical or psychometric module must follow these additional decisions before it can be treated as production-capable: +## 11. Automation and merge safety -- the numerical kernel is implemented in Rust; -- CPU parallelism minimizes context switching and GPU acceleration is available behind a deterministic capability boundary; -- true-parameter recovery, bias, coverage, and RMSE are tested on realistic simulations; -- multilevel and multiple-membership structures are modeled to avoid atomistic inference; -- temporal change, repeated measurement, drift, and state evolution are explicit model dimensions; -- numerical reproducibility, precision, seed control, convergence diagnostics, and fallback behavior are documented; -- statistical assumptions and estimands are cited in APA 7 style. +Pull requests follow one loop: inspect every review/check on the exact current head, perform evidence-backed RCA, make a test-first causal fix, rerun exact-head gates, resolve only addressed threads, and merge only after real repository protections are satisfied. Administrative bypasses and fabricated approval/evidence are prohibited. -## 5. Automation and merge safety +Automation is work-conserving: a blocked PR/check/provider/tool path blocks only that action. A repository writer lease prevents competing branch writes, while remaining safe read-only/non-conflicting work continues. One successful commit, documentation update, PR creation, merge or check dispatch is an intermediate result while executable work remains. -Pull requests follow one loop: inspect every review and check, fix root causes, rerun the exact head, resolve addressed threads, and merge only after all required evidence passes. Administrative bypasses are prohibited. +The bounded hourly OpenCode commercial-development workflow and deterministic policy package are **Implemented on protected main** from merged PR #122 (`876850018a17323900844e79845ba395b7bf6a9a`). The model does not receive generic GitHub/product-data authority; deterministic policy, exact-head/base/diff checks, and normal review/security/merge gates remain authoritative. Existing independent review-agent credentials are not repurposed. -Scheduled model-assisted automation uses `NVIDIA_NIM_API_KEY`; `COPILOT_GITHUB_TOKEN` is prohibited. Existing dedicated review-agent credentials are not repurposed. Deterministic audit and merge eligibility remain independently enforceable even when a model provider is unavailable. +The pinned OpenCode configuration disables project-local overrides, explicitly reloads reviewed repository instructions, enables only NVIDIA, registers and whitelists one model label independently of the bundled catalog, pins primary and small-model work to it, and checks that effective catalog offline before its credential bridge starts; the bridge exposes no provider-wide discovery route. Model-generated source verification runs without Docker authority. A later trusted operation parses the accepted candidate's explicitly selected Compose file, while credential-free pull-request CI starts digest-pinned images, proves PostgreSQL query execution and NATS JetStream availability, binds published ports to loopback, and tears down unconditionally. This hardening is **Implemented on protected main** from merged PR #133 (`38dde3fa44f6ad24d848c378237e55cbee79b3f0`). -## 6. Documentation hierarchy +## 12. Canonical documentation hierarchy -1. `AGENTS.md` — repository-wide agent and merge rules. -2. `ARCHITECTURE.md` — durable architectural decisions and diagrams. -3. `CLAUDE.md` — Claude-compatible operational handoff that defers to `AGENTS.md`. -4. `docs/superpowers/specs/` — approved feature designs. -5. `docs/superpowers/plans/` — implementation sequences. -6. `docs/operations/` — operator runbooks and SLOs. -7. `docs/research/` — standards and research rationale with APA 7 references. -8. `CHANGELOG.md` — user-visible unreleased and released changes. +1. `docs/PRD.md` — product outcomes, users, requirements, scope, status. +2. `docs/TRD.md` — shared technical/runtime/security/release requirements. +3. `ARCHITECTURE.md` — durable bounded contexts, authority and architecture invariants. +4. `docs/adr/README.md` — material decisions and supersession history. +5. `docs/DATA_MODEL.md` — logical service-owned data model/ERD. +6. `docs/UML.md` — component/sequence/state/deployment/failure views. +7. `docs/API_CONTRACTS.md` — API/event/provider ownership and evolution registry. +8. `SECURITY.md` — vulnerability reporting/security policy. +9. `docs/THREAT_MODEL.md` — trust boundaries, threats, mitigations and residual risk. +10. `docs/PRIVACY_DATA_LIFECYCLE.md` — sensitive-data lifecycle, retention/export/erasure authority and gaps. +11. `docs/TEST_STRATEGY.md` — deterministic/live quality evidence and release testing. +12. `docs/OPERABILITY.md` — deployment/diagnostics/backup/recovery/operator ownership. +13. `docs/RELEASE_AND_MIGRATION.md` — versioning, migrations, compatibility and rollback contract. +14. `docs/STANDARDS_TRACEABILITY.md` — standards/research source class and product-evidence mapping. +15. `docs/TRACEABILITY.md` — requirement/decision/capability to code/test/runbook/gap evidence. +16. `docs/DOCUMENTATION_ASSESSMENT.md` — documentation completeness and historical reconciliation. +17. `docs/operations/`, `docs/research/`, `docs/legal/`, `docs/superpowers/specs/`, `docs/superpowers/plans/` — scoped supporting evidence. +18. `CHANGELOG.md` — buyer-visible unreleased/released changes. -A behavior or boundary change is incomplete until the relevant level is updated and executable tests prove the claim. +The original `docs/superpowers/specs/2026-08-02-life-os-design.md` is retained as historical design input, not a parallel current PRD/TRD. A behavior or boundary change is incomplete until relevant canonical docs and executable tests match the implementation. diff --git a/CHANGELOG.md b/CHANGELOG.md index a6982da2..c76b6791 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,7 @@ All notable changes to LifeOS are documented in this file. ### Added +- A canonical product documentation graph covering the PRD, TRD, architecture decisions, logical ERD/data model, UML interaction/deployment views, threat model, test strategy, operability boundary, requirements traceability, and a machine-checked documentation contract. - An hourly and manually dispatchable NVIDIA NIM live-conformance harness that pins contextual-orchestrator to an exact reviewed commit, compares strong single-route reasoning with bounded conducted workflows, and retains only validated credential-free quality, safety, orchestration, usage, and ablation evidence. - A versioned, immutable AI proposal quality evaluator that separates production validity, semantic operation conformance, evidence grounding, benign utility, forbidden-text leakage, and prompt-injection resistance across realistic English, Korean, temporal, empty-context, completed-item, and adversarial fixtures. - An explicit `contextual-orchestrator` proposal-model mode with bounded OpenAI-compatible transport, strict structured output, model provenance, and an independent local rule-based default. @@ -23,6 +24,9 @@ All notable changes to LifeOS are documented in this file. ### Fixed +- The OpenCode development loop now prevents project settings from overriding its pinned offline NVIDIA model, records catalog failures accurately, parses the accepted candidate's exact Compose file outside the model account, and requires digest-pinned PostgreSQL queries plus NATS JetStream probes in pull-request CI. +- Canonical documentation now marks early browser-only local-first, single-application, and UUIDv7 design proposals as superseded where they conflict with the current multi-user modular MSA and UUIDv4 protected-main contracts. +- Root architecture and agent handoff documents now include current notification/privacy bounded contexts and distinguish protected-main, active-PR, partial, planned, research-only, superseded, and out-of-scope behavior. - Live contextual-orchestrator responses now classify successful empty bodies as evaluation failures, emit exactly one terminal observation, canonicalize retained timestamps safely, and preserve null metric denominators instead of fabricating deltas. - Stale AI proposal revision conflicts now belong to the technology-independent audit domain while the PostgreSQL adapter preserves its compatibility export. - Planning search now normalizes browser query text and prevents stale or unmounted requests from replacing the latest visible result state. @@ -32,6 +36,7 @@ All notable changes to LifeOS are documented in this file. ### Security +- The commercial-development model account no longer performs Docker commands, never receives Docker-socket authority, and cannot trigger provider-wide model discovery through the credential bridge. - The scheduled live-model harness uses only `NVIDIA_NIM_API_KEY`, seeds it through the encrypted contextual-orchestrator credential registry, installs hash-locked dependencies from an exact commit, confines LifeOS traffic to loopback, allowlists NVIDIA NIM egress, and excludes provider credentials, prompts, responses, traces, and hidden reasoning from retained artifacts. - Proposal quality reports now discard nested model failures and response bodies, normalize labeled sentinel checks, expose no provider credential or mutation dependency, and measure prompt-injection resistance together with benign utility instead of rewarding blanket refusal. - External proposal generation now accepts only one credential-free HTTPS orchestrator origin, stops responses at 65536 bytes, enforces a bounded abort timeout, supplies no tools, treats planning context as untrusted data, and exposes only sanitized failures. diff --git a/CLAUDE.md b/CLAUDE.md index 06439b77..123cd418 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,47 +1,63 @@ # Claude operating contract for LifeOS -`AGENTS.md` is the canonical repository-wide instruction file. This document maps that contract into a concise execution order for Claude-compatible agents and must not override `AGENTS.md`, `ARCHITECTURE.md`, branch protection, or security policy. +`AGENTS.md` is the canonical repository-wide agent instruction file. The canonical product/technical evidence graph is `docs/PRD.md`, `docs/TRD.md`, `ARCHITECTURE.md`, `docs/adr/README.md`, `docs/DATA_MODEL.md`, `docs/UML.md`, `docs/API_CONTRACTS.md`, `SECURITY.md`, `docs/THREAT_MODEL.md`, `docs/PRIVACY_DATA_LIFECYCLE.md`, `docs/TEST_STRATEGY.md`, `docs/OPERABILITY.md`, `docs/RELEASE_AND_MIGRATION.md`, `docs/STANDARDS_TRACEABILITY.md`, `docs/TRACEABILITY.md`, and `docs/DOCUMENTATION_ASSESSMENT.md`. This document maps them into a concise execution order and must not override live branch protection, security policy, or protected-main behavior. ## Execution order -1. Inspect every open pull request before starting unrelated implementation. -2. Read all human, CodeRabbit, AppGuardrail, code-scanning, and security feedback. -3. Determine the root cause of every failing or pending-required check. -4. Make the smallest complete correction, including tests and documentation. -5. Re-run the exact pull-request head and resolve only threads whose finding is actually addressed. -6. Merge only when required checks pass, no actionable findings remain, and the repository's merge policy accepts the exact head. -7. Continue with the highest-impact buyer-visible gap after the pull-request queue is empty. +1. Refetch every open PR, exact current head and exact live base tip before relying on historical state. +2. Read all human, CodeRabbit, AppGuardrail, code-scanning, security and configured automated feedback. +3. Determine the first causal boundary of every failed/missing/stale/required check and verify a remedy is operationally real. +4. Make the smallest complete test-first correction, including documentation/cleanup/migration evidence required by the root cause. +5. Rerun/inspect exact-head verification and resolve only threads whose underlying finding is actually fixed. +6. Merge only when repository protections accept the unchanged exact head and no actionable finding remains. +7. Immediately continue with another safe PR/review/cleanup/documentation/product/operability/release-readiness task while finite run budget remains. +8. When PR work is exhausted, implement the highest-impact bounded buyer-visible product gap rather than ending at gap identification. -Routine progress narration is not a substitute for repository evidence. Record decisions in code, tests, ADRs, specifications, plans, runbooks, issues, and pull-request descriptions. +A successful commit, PR creation, merge, documentation pack, review request, check dispatch, queued workflow, or RCA is an intermediate result while safe work remains. Routine status narration is not repository evidence. -## Non-negotiable boundaries +## Writer/concurrency discipline -- Never use `COPILOT_GITHUB_TOKEN`. -- Scheduled model-assisted work uses `NVIDIA_NIM_API_KEY` through the approved OpenCode or contextual-orchestrator boundary. -- Do not alter or repurpose the credential scheme of existing review agents. -- Never forward browser cookies, provider credentials, hidden reasoning, raw prompts, raw model responses, or stack traces into retained artifacts. -- Internal identifiers are UUIDv4 strings; numeric external identifiers are mapped through an explicit provider-identity boundary. -- Database objects use multiword `snake_case` names unless an external protocol mandates a different spelling. +Before branch-affecting writes, refetch exact target head/base/ref/blob. If another source writer moves the same target, discard stale assumptions, freeze only that target for the run, and continue non-conflicting work. Never turn one branch conflict or unavailable tool path into a repository-wide stop. + +## Architecture boundaries + +- LifeOS is multi-user, server-backed, self-hostable and domain-oriented; early browser-only local-first/single-app primary designs are historical/superseded. +- Internal identifiers are opaque UUIDv4 strings; old UUIDv7 design language is superseded. +- Database objects use descriptive multiword `snake_case` unless an external protocol requires otherwise. - Services do not read or mutate another service's database tables. -- AI proposals remain inert until a separately authorized user-confirmed execution capability exists. -- Mathematical and psychometric numerical kernels require Rust, deterministic CPU/GPU execution boundaries, realistic parameter-recovery tests, multilevel or multiple-membership structure, and temporal modeling where applicable. +- Browser-local drafts/caches do not become durable truth until an authorized owning service confirms persistence. +- AI proposals remain inert until a separately authorized domain execution capability exists. +- Sensitive access uses purpose/resource/actor/lifetime controls and auditable evidence; blanket masking is not an authorization model. +- The bounded hourly OpenCode commercial-development loop is protected-main automation after PR #122, but deterministic policy and normal review/security/exact-head gates remain authoritative and the automation has no product-data authority. + +## Documentation discipline + +Use exact statuses: `Implemented on protected main`, `Implemented on active PR`, `Partial`, `Accepted architecture`, `Planned`, `Research only`, `Superseded`, `Out of scope`. + +Do not present active-PR/roadmap behavior as shipped. When source/tests disagree with prose, correct the prose or behavior according to the approved product decision; do not use documentation to override protected-main runtime truth. + +The original `docs/superpowers/specs/2026-08-02-life-os-design.md` is historical input. Canonical docs/ADRs explicitly reconcile its local-first/single-app/UUIDv7/post-MVP assumptions with current architecture. + +When documentation exposes a missing product journey, stale runtime contract, migration/recovery gap, security/privacy flaw, accessibility issue or release blocker, continue into executable work when safe rather than stopping at the audit. ## LLM orchestration decisions -Use a strong single-model route as the mandatory baseline. Allocate additional test-time compute only through explicit profiles that identify reasoning effort, workflow stages, role assignment, decomposition, recursive depth, and access topology. Use measured proposal validity, grounding, utility, and prompt-injection resistance to justify deeper orchestration. Do not optimize this decision for latency alone. +Use a strong single-model route as the mandatory baseline. Allocate additional test-time compute only through explicit profiles identifying reasoning effort, workflow stages, roles, decomposition, recursive depth and access topology, justified by measured validity/grounding/utility/safety evidence rather than latency alone. -Live model tests may use `NVIDIA_NIM_API_KEY`. Deterministic pull-request checks must remain meaningful when that secret or the provider is unavailable. Provider failures produce sanitized unavailable evidence, never fabricated scores. +Model-assisted tests/development use `NVIDIA_NIM_API_KEY` through the approved OpenCode/contextual-orchestrator boundary where required. Do not casually alter independent review-agent credential schemes. Deterministic checks remain meaningful when the live provider is absent or unavailable; provider failures produce sanitized unavailable evidence, never fabricated scores. ## Verification standard - Production declarations have explanatory docstrings. -- Changed production code maintains 100% statement, branch, function, and line coverage where the package enforces those gates. -- Tests model realistic domain outcomes, not only mocked implementation calls. -- Standards and research claims are documented with APA 7 references and publication status is distinguished from drafts or preprints. +- Packages with exact gates maintain 100% statement, branch, function and line coverage using meaningful tests. +- Persistence behavior uses realistic PostgreSQL tests for tenant, transaction, replay, concurrency and recovery semantics. +- Core web journeys include accessibility/localization/mobile/PWA evidence where relevant. +- Standards/research claims use appropriate primary/current sources with APA 7 traceability and publication-status distinctions. +- API/event, privacy/data lifecycle, migration/rollback and operability docs remain synchronized with owning implementations. - `CHANGELOG.md` records buyer-visible behavior. -- `ARCHITECTURE.md` and relevant feature ADR/specification files record boundary changes. -- Release tags and versions are created only after the repository proves release readiness; unreleased work stays under `Unreleased`. +- Canonical evidence documents remain code-current and `docs/TRACEABILITY.md` distinguishes protected-main evidence from active-PR/planned gaps. +- Release versions/tags are created only after exact integrated release readiness; unreleased work stays under `Unreleased`. ## Safe escalation -Escalate only for a decision or permission that cannot be resolved from repository policy, tests, standards, or available credentials. Waiting for checks or reviews is not itself an escalation condition; continue independent analysis, documentation, or the next non-conflicting planned task while preserving merge safety. +Escalate only for a concrete external decision, permission, secret, governance action or safety boundary that cannot be derived/resolved from current repository evidence and realistically available tools after alternatives are tested. Waiting for checks/reviews/providers is not itself an escalation condition; continue independent work. diff --git a/README.md b/README.md index 59412fab..188cf27d 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,32 @@ LifeOS connects everyday action to longer-term direction. It is designed as a mu ## Status -LifeOS is in active foundation development. The current `main` branch contains the monorepo, gateway, bounded services, shared contracts, responsive web shell, PostgreSQL persistence, NATS JetStream configuration, security gates, and commercial-readiness evidence loop. Interfaces and migrations may still change before the first stable release. +LifeOS is in active foundation development. The current `main` branch contains the monorepo, gateway, bounded services, responsive web shell, PostgreSQL persistence, NATS JetStream configuration, security gates, and commercial-readiness evidence loop. Interfaces and migrations may still change before the first stable release. + +Canonical documents distinguish protected-main behavior from active-PR, partial, planned, research-only, superseded, and out-of-scope work. Do not use the original 2026-08-02 combined design as a parallel current PRD/TRD; it is retained as historical design input. + +## Canonical documentation + +Start here when evaluating or changing the whole product: + +- [Product requirements](docs/PRD.md) +- [Technical requirements](docs/TRD.md) +- [Architecture](ARCHITECTURE.md) +- [Architecture decisions](docs/adr/README.md) +- [Logical data model / ERD](docs/DATA_MODEL.md) +- [UML and interaction views](docs/UML.md) +- [API and event contract registry](docs/API_CONTRACTS.md) +- [Security policy](SECURITY.md) +- [Threat model](docs/THREAT_MODEL.md) +- [Privacy and data lifecycle](docs/PRIVACY_DATA_LIFECYCLE.md) +- [Test strategy](docs/TEST_STRATEGY.md) +- [Operability and deployment boundary](docs/OPERABILITY.md) +- [Release, migration, and rollback contract](docs/RELEASE_AND_MIGRATION.md) +- [Standards and research traceability](docs/STANDARDS_TRACEABILITY.md) +- [Requirements and evidence traceability](docs/TRACEABILITY.md) +- [Documentation completeness assessment](docs/DOCUMENTATION_ASSESSMENT.md) + +Scoped implementation plans, feature designs, runbooks, standards/research, and legal material remain under `docs/superpowers/`, `docs/operations/`, `docs/research/`, and `docs/legal/` as supporting evidence. ## Architecture @@ -14,13 +39,15 @@ LifeOS is in active foundation development. The current `main` branch contains t Web / PWA | API Gateway / BFF - |---------------------------------------------| -Identity Planning Habit Review - | | | | -PostgreSQL schemas / databases + NATS JetStream events + |-------------------------------------------------------------| +Identity Planning Habit Review AI Calendar Privacy Plugin + | + Notification / event consumers + +Service-owned PostgreSQL persistence + versioned NATS JetStream events ``` -The MVP deliberately keeps goals, projects, milestones, and tasks in one Planning bounded context. Services own their persistence boundaries; direct cross-service table access is prohibited. +The product deliberately keeps strongly coupled goals, projects, milestones, and tasks in one Planning bounded context. Services own their persistence boundaries; direct cross-service table access is prohibited. Internal identifiers are opaque UUIDv4 strings under the current protected-main contract. ## Repository layout @@ -32,11 +59,16 @@ apps/ planning-service/ habit-service/ review-service/ + ai-service/ + notification-service/ + privacy-service/ integration-calendar-service/ integration-service/ packages/ contracts/ plugin-sdk/ + observability/ + commercial-readiness/ infra/ docs/ ``` @@ -57,7 +89,7 @@ docker compose up -d pnpm dev ``` -Default endpoints: +Default endpoints include: - Web: `http://localhost:3000` - Gateway health: `http://localhost:4000/v1/health` @@ -71,56 +103,80 @@ Default endpoints: - Plugin contract discovery: `http://localhost:4107/v1/plugin-contract` - NATS monitoring: `http://localhost:8222` -Metrics endpoints contain operational data. Production ingress must restrict them to the monitoring network. +Exact endpoints for each bounded service are owned by current source/contracts. Metrics endpoints contain operational data; production ingress must restrict them to the monitoring network. ## Authentication -Google and GitHub OAuth are the required login providers. Provider credentials are supplied through environment variables and must never be committed. Deployment operators are responsible for provider registration, redirect URI policy, secret rotation, and production access controls. +Google and GitHub OAuth are the required login providers. Provider credentials are supplied through environment variables/protected deployment secrets and must never be committed. Deployment operators are responsible for provider registration, redirect URI policy, secret rotation, and production access controls. + +LifeOS maps provider identities to opaque internal UUIDv4 identities. Browser-selected workspace/actor identifiers are not trusted as authorization merely because they are syntactically valid. + +## Planning, Today, habits, and reviews + +Planning-service owns durable goals, projects, milestones, tasks, search, and planning mutation rules. The web Today experience can maintain explicitly labeled browser-local draft state, but local state is not durable truth until an authorized service confirms persistence. + +Habit-service owns recurring habit definitions and completion history. Review-service owns review projections/snapshots and does not directly rewrite planning/habit source tables. Complete multi-device optimistic synchronization of the whole durable Today aggregate remains a product gap until protected-main evidence proves the end-to-end conflict/recovery contract. ## Calendar synchronization The calendar integration service supports explicit `caldav` and `google` provider modes. Set `CALENDAR_PROVIDER` and the matching variables in `.env.example` before starting the service. -CalDAV writes use deterministic resource names, `If-None-Match: *` for creation, and strong `If-Match` ETags for updates. Google Calendar writes use a deterministic API event identifier to prevent duplicate creation and the same strong-ETag precondition for updates. Neither adapter exposes delete, move, or copy operations through the LifeOS provider contract. +CalDAV writes use deterministic resource names, `If-None-Match: *` for creation, and strong `If-Match` ETags for updates. Google Calendar writes use a deterministic API event identifier to prevent duplicate creation and the same strong-ETag precondition for updates. Neither adapter exposes delete, move, or copy operations through the current LifeOS provider contract. + +`GOOGLE_CALENDAR_ACCESS_TOKEN` is an operator-supplied runtime secret for this bounded adapter slice. Per-user OAuth credential storage, token refresh, revocation, calendar discovery, and encrypted persistence remain deferred and are tracked by issue #129 before a multi-user hosted deployment enables unattended Google Calendar synchronization. + +## Notifications + +The notification service provides durable PostgreSQL reminder occurrences, expiring worker claims, immutable outcomes, timezone-aware quiet hours/fatigue controls, and idempotent in-app delivery according to its current contracts. See [notification persistence](docs/operations/notification-persistence.md) for the scoped runbook. + +## AI proposals + +AI output is an inert proposal, not an execution command. The AI service persists proposal evidence before return and records explicit replay-safe accept/reject decision history. It has no generic planning mutation repository/command bus. + +The web boundary derives workspace/actor authority from the active session and signs the exact upstream context; browser credentials are not forwarded to the AI service. Deterministic proposal-quality/security gates are separate from bounded live-provider conformance. + +## Privacy access + +The privacy service provides purpose-bound authorization and durable decision/grant/event evidence for sensitive-data access. It uses explicit actor/resource/purpose/lifetime boundaries rather than treating indiscriminate masking as a complete authorization model. Public logs/errors/artifacts remain content-minimized and credential-free. -`GOOGLE_CALENDAR_ACCESS_TOKEN` is an operator-supplied runtime secret for this bounded adapter slice. Per-user OAuth credential storage, token refresh, revocation, calendar discovery, and encrypted persistence remain deferred and must be implemented before a multi-user hosted deployment enables Google Calendar synchronization. +Complete tenant export/deletion productization remains partial and is tracked by issue #55; see [privacy and data lifecycle](docs/PRIVACY_DATA_LIFECYCLE.md). ## Plugin contract -The `@life-os/plugin-sdk` package defines strict versioned manifests, tenant-scoped CloudEvents 1.0 structured JSON envelopes, deterministic canonical serialization, and HMAC-SHA256 delivery-proof helpers. The integration service exposes contract discovery, manifest validation, and event preparation only. +The `@life-os/plugin-sdk` package defines strict versioned manifests, tenant-scoped CloudEvents-style envelopes, deterministic canonical serialization, and delivery-proof helpers. The integration service exposes contract discovery, manifest validation, and event preparation only. -This slice deliberately has no plugin installation, secret persistence, outbound webhook delivery, inbound commands, or direct database access. Those require separately reviewed least-privilege authorization, durable audit, and SSRF-safe delivery boundaries. +The current slice deliberately has no generic plugin installation, durable plugin secret persistence, outbound webhook delivery, inbound arbitrary commands, or direct database access. Those require separately reviewed least-privilege authorization, durable audit, and SSRF-safe delivery boundaries. ## Backup and recovery -`infra/backup/backup.sh` creates a private PostgreSQL custom-format archive, checksum, and non-secret metadata set. `infra/backup/restore.sh` verifies the selected archive and restores only into a deliberately empty non-system database. The Linux CI contract performs a real dump and restore with pinned PostgreSQL client tools and verifies exact tenant records, non-empty-target refusal, and checksum-corruption refusal. +`infra/backup/backup.sh` creates a private PostgreSQL custom-format archive, checksum, and non-secret metadata set. `infra/backup/restore.sh` verifies the selected archive and restores only into a deliberately empty non-system database. CI contracts perform real dump/restore verification, including tenant records, non-empty-target refusal, and checksum-corruption refusal. -This logical-dump tier is not point-in-time recovery and does not schedule, encrypt, replicate, or retain backups automatically. Deployment owners must follow the [backup and restore runbook](docs/operations/backup-and-restore.md), establish independent encrypted storage, rehearse recovery, and add WAL archiving when the required recovery point is shorter than the dump interval. +This logical-dump tier is not point-in-time recovery and does not schedule, encrypt, replicate, or retain backups automatically. Deployment owners must follow the [backup and restore runbook](docs/operations/backup-and-restore.md), establish independent encrypted storage, rehearse recovery, and add WAL/PITR when required by their recovery objective. ## Production reference deployment -`infra/kubernetes` contains a provider-neutral Kustomize reference for the current web and gateway edge workloads. It encodes a Restricted Pod Security namespace, non-root and read-only containers, probes, resource bounds, rolling updates, disruption budgets, topology spread, ClusterIP services, disabled service-account token automount, and default-deny network policy. The committed image digests and public origin are deliberately non-deployable sentinels. +`infra/kubernetes` contains a provider-neutral Kustomize reference for current workloads. It encodes restricted runtime defaults such as non-root/read-only containers, probes, resource bounds, rolling updates, disruption budgets, topology spread, service-account restrictions, and default-deny network policy where the current manifests define them. -The manual deployment workflow accepts only digest-pinned images and an exact HTTPS web origin, uses one shared renderer, optionally applies forward-only migrations, runs through the protected GitHub `production` environment, and performs server-side dry-run and diff. Before applying, it captures whether each Deployment exists and its current revision. A failed apply or rollout must either verify rollback to that captured revision or verify deletion of a first-time Deployment; a separate failure is reported when workload-state recovery itself fails. Namespace policy, completed migrations, external infrastructure, and other non-Deployment resources are not automatically reversed. The reference does not provision a cluster, database, NATS, ingress, TLS, DNS, image pipeline, or secret manager. Operators must follow the [production deployment runbook](docs/operations/production-deployment.md) and preserve those explicit ownership boundaries. +The protected deployment path accepts only bounded reviewed inputs and verifies its claimed workload rollback/recovery behavior. The reference does not provision a cluster, database, NATS, ingress, TLS, DNS, image pipeline, or secret manager. Operators must follow the [production deployment runbook](docs/operations/production-deployment.md) and preserve those explicit ownership boundaries. ## Privacy and deployment responsibility This is a public repository. It contains synthetic examples only. Personal goals, health information, relationship data, credentials, access tokens, private prompts, customer data, and production exports must not be committed. -The upstream project does not operate every LifeOS deployment. A self-hosting organization controls its deployment data and must establish its own privacy notice, retention policy, security controls, subprocessors, and legal basis. See the [upstream privacy notice](docs/legal/privacy.md) and [upstream project terms](docs/legal/terms.md) for the upstream project boundary. +The upstream project does not operate every LifeOS deployment. A self-hosting organization controls its deployment data and must establish its own privacy notice, retention policy, security controls, subprocessors, legal basis, backups, monitoring, and incident response. See the [upstream privacy notice](docs/legal/privacy.md) and [upstream project terms](docs/legal/terms.md). -## Documentation +## Additional documentation -- Product and architecture design: `docs/superpowers/specs/2026-08-02-life-os-design.md` +- Historical initial combined design: `docs/superpowers/specs/2026-08-02-life-os-design.md` (**historical/superseded where canonical docs differ**) - Foundation implementation plan: `docs/superpowers/plans/2026-08-02-life-os-foundation.md` - Gateway service-level objectives: `docs/operations/service-level-objectives.md` - Planning-service service-level objectives: `docs/operations/planning-service-level-objectives.md` -- [Plugin contract surface plan](docs/superpowers/plans/2026-08-04-plugin-contract-surface.md) -- [PostgreSQL backup and restore runbook](docs/operations/backup-and-restore.md) -- [Production Kubernetes deployment runbook](docs/operations/production-deployment.md) -- [Upstream privacy notice](docs/legal/privacy.md) -- [Upstream project terms](docs/legal/terms.md) -- [Vulnerability reporting](SECURITY.md) +- Plugin contract surface plan: `docs/superpowers/plans/2026-08-04-plugin-contract-surface.md` +- Backup/restore runbook: `docs/operations/backup-and-restore.md` +- Production deployment runbook: `docs/operations/production-deployment.md` +- Upstream privacy notice: `docs/legal/privacy.md` +- Upstream project terms: `docs/legal/terms.md` +- Vulnerability reporting: `SECURITY.md` ## Contributing diff --git a/docs/API_CONTRACTS.md b/docs/API_CONTRACTS.md new file mode 100644 index 00000000..de840e5c --- /dev/null +++ b/docs/API_CONTRACTS.md @@ -0,0 +1,139 @@ +# LifeOS API and Event Contract Registry + +**Baseline:** protected `main` at `876850018a17323900844e79845ba395b7bf6a9a` + +## 1. Purpose + +This document is the canonical registry of **contract ownership and versioning**, not a hand-maintained duplicate of every route schema. Exact request/response shapes remain authoritative in the owning service source, exported package types, tests, and generated schema where present. + +A route or event mentioned here is not permission for one service to access another service's database. + +## 2. Shared contract primitives + +`packages/contracts/src/index.ts` currently defines the shared TypeScript primitives used across bounded contexts: + +- `WorkspaceRole` +- `RequestContext` +- `ProblemDetails` +- versioned `DomainEvent` +- `planning.task.completed.v1` / `TaskCompletedEvent` + +The shared event envelope carries an opaque event ID, explicit type/version, occurrence time, actor/workspace/correlation identity, optional causation identity, and bounded typed payload. + +### Contract rules + +- Internal IDs use UUIDv4 under the repository-wide invariant. +- Workspace/actor identifiers carried in a request/event do not become authority unless the receiving boundary verifies the authenticated/signed context required by that contract. +- Public problems are credential-free and bounded. +- Breaking semantic changes require a new route/event/schema version; changing TypeScript shape without versioning does not constitute a safe protocol migration. +- Unknown event/operation versions fail closed rather than silently falling back to a semantically different contract. + +## 3. HTTP bounded-context registry + +| Boundary | Owner | Current responsibility | Status | +| --- | --- | --- | --- | +| Browser session / OAuth start/callback/introspection | Identity service | Google/GitHub external identity, revocable browser session, workspace/actor authority | Implemented on protected main | +| Planning goals/projects/tasks/search | Planning service | durable planning source of truth and search | Implemented on protected main | +| Habit definition/completion | Habit service | recurring habits and immutable/replay-safe completion lifecycle | Implemented on protected main | +| Daily/weekly review | Review service | review snapshots/projections/observations | Implemented on protected main | +| Today composition / browser APIs | Gateway/Web BFF | authenticated composition; no durable cross-domain database authority | Implemented on protected main | +| Durable Today workspace synchronization | Planning service + Web BFF | explicit local-to-durable migration, strong revision preconditions, multi-device conflict reconciliation | Implemented on active PR | +| Calendar synchronization | Calendar integration service | CalDAV/Google provider adaptation with duplicate/conflict protection | Implemented on protected main | +| Calendar connection/credential lifecycle | Calendar integration service target boundary | encrypted per-user credential lifecycle/provider selection | Planned | +| Reminder scheduling/delivery | Notification service | durable occurrences, claims, outcomes, inbox delivery | Implemented on protected main | +| AI proposal/evidence/decision | AI proposal service | inert proposal generation/persistence/decision audit | Implemented on protected main | +| Purpose-bound sensitive access | Privacy access service | access decisions/grants/events | Implemented on protected main | +| Plugin contract discovery/validation/event preparation | Integration service + plugin SDK | versioned plugin boundary without direct DB access | Implemented on protected main | +| Generic plugin install/secrets/outbound delivery | Future integration boundary | least-privilege install/network/secret/runtime authority | Planned | +| Tenant export/erasure orchestration | Identity-owned data-rights core + future contributors | deterministic export and fail-closed erasure coordination with incomplete end-to-end lifecycle | Partial | + +Tracking evidence is kept outside the status cells so status remains machine-checkable. Durable Today synchronization is implemented on active PR #127 and is not protected-main evidence until that PR integrates. Hosted per-user calendar credential lifecycle remains tracked by issue #129. Generic plugin runtime delivery remains tracked by issue #130. Complete tenant export/erasure productization remains tracked by issue #55. + +## 4. Authentication and private context + +Browser-facing routes authenticate through the identity/web boundary. Downstream services do not trust arbitrary client-supplied `workspace_id`/`actor_id` fields as authorization. + +For signed internal context contracts: + +- bind the exact workspace and actor; +- bind the exact HTTP method and path where required; +- include bounded issuance/lifetime/key identity; +- verify only explicitly active/overlap keys; +- reject retired/unknown/malformed contexts; +- never forward browser cookies, OAuth access tokens, model credentials, or unrelated service secrets as generic context. + +## 5. Mutation contract + +A state-changing HTTP contract defines applicable: + +- ownership/role/purpose requirement; +- request size and string/collection bounds; +- UUID/version/schema validation; +- idempotency/replay key; +- stale-write precondition (`revision`, digest, ETag, or equivalent); +- transactional persistence boundary; +- public conflict/problem classification; +- immutable audit/outcome evidence; +- recovery/retry behavior. + +A caller receiving HTTP success must not infer a side effect that the owning service contract does not explicitly guarantee. + +## 6. Event contract registry + +### `planning.task.completed.v1` + +**Owner:** Planning service. +**Envelope:** shared `DomainEvent`. +**Meaning:** one planning task completion occurred at the supplied instant for the authenticated/authorized workspace/actor context. + +Consumers may project or trigger bounded follow-up behavior. They may not use the event as permission to update planning tables. + +### Additional event families + +Notification/review/integration features may use service-specific events or durable polling according to current source. Add any cross-service event to this registry when it becomes a supported product contract. Do not infer a durable event from an internal TypeScript callback or test fixture. + +## 7. Provider adapter contracts + +### Identity providers + +Provider account identifiers are external metadata. Callback/state/error payloads are untrusted and provider-specific. The identity service converts them to LifeOS authority only after verification. + +### Calendar providers + +The provider port exposes only reviewed operations. Current CalDAV/Google synchronization preserves deterministic resource identity and strong preconditions where supported. DELETE/MOVE/COPY are not implicitly authorized because the provider supports them. + +### Model/contextual-orchestrator provider + +The model boundary receives bounded prompt/context and returns untrusted structured output. It has no implicit product mutation authority. Live-provider failures are classified/sanitized instead of transformed into fabricated proposals or quality metrics. + +### Plugin adapters + +Plugin manifests/events are versioned, bounded and tenant scoped. Generic outbound network access or secret storage is not part of the existing plugin contract. + +## 8. Contract evolution + +A breaking contract change requires: + +1. identify owning bounded context; +2. document old/new schema and compatibility window; +3. add consumer/provider contract tests; +4. add migration/replay/concurrency behavior if stateful; +5. update shared package types only after the protocol decision is explicit; +6. preserve old version during the reviewed compatibility window or fail explicitly; +7. update PRD/TRD/architecture/UML/data/threat/operability/traceability where the change crosses those boundaries; +8. update `CHANGELOG.md` for buyer/operator-visible behavior. + +Do not rely on simultaneous deployment of every service unless the release explicitly proves that coupling. + +## 9. Contract evidence hierarchy + +For an exact current contract, inspect in order: + +1. owning service route/controller/domain/persistence source on protected main; +2. exported shared/plugin SDK types; +3. owning integration/contract tests; +4. migrations and provider adapters; +5. scoped feature specification/runbook; +6. this registry. + +This registry is intentionally concise so it cannot silently drift into a second implementation specification. diff --git a/docs/DATA_MODEL.md b/docs/DATA_MODEL.md new file mode 100644 index 00000000..fd9bbfbf --- /dev/null +++ b/docs/DATA_MODEL.md @@ -0,0 +1,241 @@ +# LifeOS Logical Data Model and ERD + +**Baseline:** protected `main` at `876850018a17323900844e79845ba395b7bf6a9a` + +## 1. Scope and authority + +This is a **logical cross-service model**, not one physical shared database schema. LifeOS services own their persistence independently. Relationships drawn across service boundaries mean application/API/event relationships only; they do **not** authorize cross-service SQL joins, foreign keys, reads, or writes. + +Entity labels use: + +- **Persisted** — protected-main persistence or migration/repository evidence exists. +- **Logical** — product/domain relationship, but the exact physical table may differ or remain planned. +- **Projection** — derived/read-optimized state; not source-of-truth mutation authority. +- **Planned** — accepted or tracked product concept without protected-main persistence evidence. + +Internal IDs use opaque UUIDv4 under the current repository contract. + +## 2. Bounded-context ownership + +| Bounded context | Logical entities | Persistence status | +| --- | --- | --- | +| Identity | user account, external identity, browser session, workspace, workspace membership | Persisted / partially named by service migrations | +| Planning | goal, project, task, durable planning/search/Today-related state | Goal/project/task persisted; complete durable Today aggregate is Partial | +| Planning future extensions | milestone, task dependency | Planned/logical only on this baseline; not represented as protected-main Planning tables | +| Habit | habit definition, recurrence state, completion event | Persisted | +| Review | daily/weekly review snapshot, review observation | Persisted/projection depending on type | +| Calendar integration | provider connection/sync record/resource binding | Partial; provider adapters implemented, hosted per-user credential lifecycle incomplete | +| Notification | reminder occurrence, worker claim, delivery outcome, inbox message | Persisted | +| AI proposal | proposal evidence, proposal decision event, model provenance | Persisted | +| Privacy access | access decision, access grant, access event | Persisted | +| Plugin integration | plugin manifest/contract/event preparation | Logical/runtime contract; installation and durable plugin secret state are not claimed | +| Audit/operations | correlation/provenance/backup/deployment evidence | Distributed across owning services/artifacts rather than a single shared DB | + +Protected-main Planning migrations currently create only `planning.goals`, `planning.projects`, and `planning.tasks` as the core hierarchy. This document therefore does not present milestones or task dependencies as persisted current entities. + +## 3. Protected-main logical ERD + +```mermaid +erDiagram + USER_ACCOUNT ||--o{ EXTERNAL_IDENTITY : links + USER_ACCOUNT ||--o{ BROWSER_SESSION : owns + USER_ACCOUNT ||--o{ WORKSPACE_MEMBERSHIP : receives + WORKSPACE ||--o{ WORKSPACE_MEMBERSHIP : authorizes + + WORKSPACE ||--o{ GOAL_RECORD : contains + WORKSPACE ||--o{ PROJECT_RECORD : contains + GOAL_RECORD ||--o{ PROJECT_RECORD : owns + WORKSPACE ||--o{ TASK_RECORD : contains + PROJECT_RECORD ||--o{ TASK_RECORD : owns + + WORKSPACE ||--o{ HABIT_RECORD : contains + HABIT_RECORD ||--o{ HABIT_COMPLETION_EVENT : records + + WORKSPACE ||--o{ REVIEW_SNAPSHOT : summarizes + REVIEW_SNAPSHOT ||--o{ REVIEW_OBSERVATION : contains + + WORKSPACE ||--o{ REMINDER_OCCURRENCE : schedules + REMINDER_OCCURRENCE ||--o{ REMINDER_CLAIM : claimed_by + REMINDER_OCCURRENCE ||--o{ REMINDER_OUTCOME : produces + REMINDER_OCCURRENCE ||--o| INBOX_MESSAGE : may_deliver + + WORKSPACE ||--o{ CALENDAR_CONNECTION : authorizes + CALENDAR_CONNECTION ||--o{ CALENDAR_RESOURCE_BINDING : synchronizes + + WORKSPACE ||--o{ AI_PROPOSAL_RECORD : receives + AI_PROPOSAL_RECORD ||--o{ AI_PROPOSAL_DECISION : decides + + WORKSPACE ||--o{ PRIVACY_ACCESS_DECISION : governs + PRIVACY_ACCESS_DECISION ||--o{ PRIVACY_ACCESS_GRANT : may_issue + PRIVACY_ACCESS_GRANT ||--o{ PRIVACY_ACCESS_EVENT : consumed_as +``` + +`GOAL_RECORD` → `PROJECT_RECORD` → `TASK_RECORD` reflects the current Planning migration contract: one project references one goal in the same workspace, and one task references one project in the same workspace. + +## 4. Planned Planning extension model + +Milestones and task dependencies appeared in earlier product planning but are not current protected-main Planning tables. If task dependencies are implemented, a dependency record must connect **exactly one predecessor task and exactly one successor task**, while either task may participate in many dependency records: + +```mermaid +erDiagram + TASK_RECORD ||--o{ TASK_DEPENDENCY : predecessor + TASK_RECORD ||--o{ TASK_DEPENDENCY : successor +``` + +A future migration must define cycle/self-dependency policy, workspace-safe composite references, deletion behavior, indexing, concurrency semantics, and realistic PostgreSQL tests before this section can be promoted from `Planned` to `Persisted`. + +## 5. Identity and workspace + +### `user_account` — logical/persisted + +Represents the LifeOS user independent of any provider-specific account identifier. + +Minimum logical attributes: + +- `user_account_id` — UUIDv4 +- display/profile fields owned by identity-service +- lifecycle timestamps/state according to the service implementation + +### `external_identity` — logical/persisted + +Maps Google/GitHub or another explicitly supported identity provider to `user_account`. Provider identifiers are external metadata and never reused as LifeOS primary keys. + +### `browser_session` — persisted + +Revocable authenticated session bound to a LifeOS user/workspace authorization context. Secret/token material is represented by digests or protected server-side values according to the identity-service contract, not exposed to downstream services. + +### `workspace` / `workspace_membership` — logical/persisted + +A workspace is the tenant/data-ownership boundary. A membership binds a user to workspace authority. Personal workspaces are the current primary UX; future team UI does not change the requirement that every domain operation be tenant scoped. + +## 6. Planning model + +### `goal_record` — persisted + +Longer-term objective. Current protected-main Planning persistence stores the goal as the top level of the implemented Goal → Project → Task hierarchy. + +### `project_record` — persisted + +Finite coordinated outcome/action set. The current Planning migration requires exactly one `goal_id` in the same workspace for each persisted project. + +### `task_record` — persisted + +Actionable planning item. The current Planning migration requires exactly one `project_id` in the same workspace for each persisted task. Completion and stale-update semantics are defined by planning-service. + +### `milestone_record` — planned/logical + +Historical product planning included milestones, but protected-main Planning migrations at this baseline do not create a milestone table. Do not treat milestones as durable current behavior until a reviewed migration/domain/API/test slice implements them. + +### `task_dependency` — planned/logical + +Historical product planning included directed task dependencies. Protected-main Planning migrations at this baseline do not create a task-dependency table. The intended logical shape is one predecessor + one successor per dependency record as shown above, but no persistence/API claim is made yet. + +### Durable Today state — partial + +Protected main has durable planning objects and a Today action loop, but complete cross-device optimistic-concurrency handling for the whole Today aggregate remains issue #121. Browser-local drafts are not equivalent to this durable aggregate. + +## 7. Habit and completion model + +### `habit_record` — persisted + +Recurring behavior definition owned by habit-service. + +### `habit_completion_event` — persisted, immutable evidence + +Records an accepted completion with workspace/habit/idempotency identity. Current integration tests exercise duplicate/concurrent completion protection and append-only history behavior. + +A recurrence-definition edit must not rewrite historical completion evidence. + +## 8. Review model + +### `review_snapshot` — persisted/projection + +Daily/weekly review state derived from authorized planning/habit evidence. Review-service owns the projection and may persist it, but the snapshot is not authorized to rewrite source planning/habit tables. + +### `review_observation` — logical/persisted as defined by current review service + +Bounded observation recorded during a review ritual. + +## 9. Notification model + +### `reminder_occurrence` — persisted + +One due reminder instance, including tenant/time semantics. + +### `reminder_claim` — persisted/ephemeral durable coordination + +Expiring/fenced worker authority for one processing attempt. Claims must be recoverable after expiry and must not turn into duplicate delivery authority. + +### `reminder_outcome` — persisted immutable evidence + +Delivery/defer/failure evidence. Current database tests explicitly exercise immutability behavior. + +### `inbox_message` — persisted + +Idempotent in-app delivery artifact where the current notification runtime uses it. + +## 10. Calendar integration model + +### `calendar_connection` — logical/partial + +Represents a user/workspace authorization to a provider. Protected-main provider adapters exist, but issue #129 tracks the missing hosted per-user encrypted credential/refresh/revocation/provider-selection lifecycle. The exact physical connection table is not claimed until that slice lands. + +### `calendar_resource_binding` — logical + +Relates a LifeOS scheduling object to a remote calendar resource/ETag/deterministic provider identifier. Exact physical storage must follow the calendar service implementation and must not be invented from this logical model. + +## 11. AI proposal model + +### `ai_proposal_record` — persisted immutable proposal evidence + +Contains bounded proposal content/evidence/provenance and ownership context. Proposal output is inert. + +### `ai_proposal_decision` — persisted append-only decision event + +Binds explicit accept/reject to exact proposal revision/content digest, actor, workspace, idempotency identity and decision time. A decision event is evidence; it is not a generic command bus. + +## 12. Privacy access model + +### `privacy_access_decision` — persisted append-only evidence + +Records whether a purpose/resource/actor request is authorized/denied under the privacy-service policy. + +### `privacy_access_grant` — persisted bounded grant + +Represents a signed/time-bounded/single-use authorization where required by the service contract. Deletion/arbitrary mutation is prohibited by current persistence controls. + +### `privacy_access_event` — persisted append-only evidence + +Records governed use/consumption of access authority. It contains bounded audit facts, not a copy of every sensitive payload. + +## 13. Cross-service relationship rules + +1. Shared `workspace_id`/`actor_id` values are logical correlation identifiers, not permission to query another service database. +2. Cross-service mutation uses HTTP/event/saga/plugin/MCP contracts. +3. A service receiving a shared identifier still validates the authenticated/signed authority relevant to that request. +4. Derived projections may cache foreign-domain identifiers/evidence but cannot become source-of-truth mutation authority. +5. Provider-native IDs remain at provider adapter boundaries. +6. Audit/provenance records store the minimum bounded evidence needed to reconstruct authority/outcome without retaining secrets or unnecessary personal content. + +## 14. Version and temporal requirements + +Where the domain can lose data under concurrency, records/commands expose an explicit revision, digest, ETag, idempotency key, immutable event ID, or equivalent concurrency identity. + +Time fields distinguish: + +- event/decision/completion occurrence time when relevant; +- persistence/recording time where needed for audit; +- local calendar date/timezone where reminder/habit behavior depends on a user's civil time. + +This document does not prescribe one global bitemporal schema because protected-main transactional services do not currently share such a requirement. + +## 15. Physical-model rule + +Before adding a logical/planned entity to a migration: + +- identify the owning bounded service; +- use a descriptive multiword `snake_case` object name; +- define tenant/index/constraint/idempotency/concurrency semantics; +- add migration and rollback/forward-fix evidence; +- add realistic PostgreSQL integration tests; +- update this document only after physical names and status are verified. diff --git a/docs/DOCUMENTATION_ASSESSMENT.md b/docs/DOCUMENTATION_ASSESSMENT.md new file mode 100644 index 00000000..991c5b98 --- /dev/null +++ b/docs/DOCUMENTATION_ASSESSMENT.md @@ -0,0 +1,195 @@ +# LifeOS Documentation Completeness Assessment + +**Assessment date:** 2026-08-09 +**Current protected-main reference:** `876850018a17323900844e79845ba395b7bf6a9a` +**Documentation baseline:** active PR #126 +**Verdict:** **The canonical documentation families are now substantially sufficient in scope, but documentation correctness remains an exact-head gate. PR #126 must pass its current tests/reviews and merge before this baseline becomes protected-main architecture evidence.** + +## 1. Executive assessment + +LifeOS already had substantial engineering evidence before this work: a root architecture file, repository agent contracts, many feature designs/plans, operator runbooks, research notes, legal documents, capability evidence, a detailed changelog, executable service tests, migrations, and security workflows. The problem was not a lack of prose. The problem was that a future maintainer, operator, acquirer, or autonomous agent had to reconstruct current truth by reconciling: + +- an initial combined product/technical design whose architecture assumptions had drifted; +- protected-main code and migrations; +- many scoped feature specifications and implementation plans; +- `product/capabilities.json` and the commercial-readiness report; +- runbooks, research notes, legal/security documents and changelog entries; +- open issues and active PRs; +- historical conversation decisions. + +Before PR #126, protected main did not contain a canonical whole-product PRD, TRD, ADR index, discoverable logical ERD/data model, UML/interaction registry, architecture threat model, repository-wide test strategy, operability boundary, API/event contract registry, privacy/data lifecycle contract, release/migration contract, standards/research index, or requirement-to-evidence traceability matrix. + +PR #126 establishes that canonical spine and adds deterministic documentation contract tests. It also records historical supersession instead of silently deleting earlier choices. + +A later exact-head audit of PR #126 found an important regression: canonical docs again contained composite status values such as `Accepted architecture / implemented evaluation support`, `Implemented documentation boundary`, and `Implemented on protected main as reference`, while the test had regressed to an ADR-only `Accepted/Proposed/Deprecated` vocabulary. The associated review threads had previously been resolved, proving that resolved review state is not sufficient current-source evidence. The branch is therefore being repaired with an exact canonical status vocabulary, active-PR traceability, real link/ADR-target checks, and source/configuration evidence checks. + +The resulting document **family coverage is substantially sufficient for product/technical/architecture governance**. The exact-current branch still requires machine/review verification and protected merge before the repository may treat this documentation baseline itself as `Implemented on protected main`. + +Documentation sufficiency and product sufficiency are separate gates. + +## 2. Completeness matrix + +| Documentation family | Before PR #126 | PR #126 result | Current assessment | +| --- | --- | --- | --- | +| Product requirements | Initial combined design + capability manifest | `docs/PRD.md` with journey/status/evidence | **Substantially sufficient; exact-head status/active-PR reconciliation required** | +| Technical requirements | Feature specs + root architecture | `docs/TRD.md` | **Substantially sufficient** | +| Architecture | Root `ARCHITECTURE.md` | updated bounded contexts/status/history | **Strong** | +| ADRs | Decisions embedded in specs/plans/chat | indexed ADR set including readiness semantics | **Sufficient baseline; expand with new material decisions** | +| UML | Scattered Mermaid diagrams | `docs/UML.md` | **Sufficient baseline with exact status discipline** | +| ERD / data model | Domain prose + migrations | `docs/DATA_MODEL.md` logical service-owned ERD | **Sufficient logical baseline; physical schemas remain owning-service source** | +| API/event contracts | Source/shared package/feature docs | `docs/API_CONTRACTS.md` ownership/version registry | **Sufficient registry; exact shapes remain source-generated/owned** | +| Security policy | `SECURITY.md` | retained | **Good** | +| Threat model | Feature/security notes | `docs/THREAT_MODEL.md` | **Sufficient upstream architecture baseline** | +| Privacy/data lifecycle | legal docs + privacy/data-rights code | `docs/PRIVACY_DATA_LIFECYCLE.md` | **Sufficient architecture baseline; product lifecycle still Partial in places** | +| Test strategy | strong tests + feature quality plans | `docs/TEST_STRATEGY.md` | **Strong canonical policy** | +| Operability | multiple runbooks/SLO docs | `docs/OPERABILITY.md` | **Strong operator boundary/index after status normalization** | +| Release/migration/rollback | runbooks/CI scattered | `docs/RELEASE_AND_MIGRATION.md` | **Sufficient repository-wide contract** | +| Research/standards | strong scoped evidence | `docs/STANDARDS_TRACEABILITY.md` | **Sufficient index; scoped APA references stay authoritative** | +| Requirements-to-code/test traceability | capability manifest + changelog | `docs/TRACEABILITY.md` | **Strong baseline; now includes active PR #127/#131/#133 and issue #130** | +| Documentation consistency | manual review only | `documentation-contract.test.mjs` | **Executable semantic regression baseline; current exact head still awaiting CI** | + +## 3. Historical design drift reconciled + +### 3.1 Local-first/private proposal → multi-user server-backed product + +Early conversation/design work considered a login-free local-first PWA storing personal state only in browser storage. That option was useful for privacy and fast prototyping, but LifeOS then became a public/multi-user, self-hostable server-backed product with Google/GitHub OAuth, personal workspaces, PostgreSQL durability, and cross-device behavior. + +**Canonical status:** `Superseded` as the primary product architecture. Browser-local state may still support drafts/offline UX, but it is not durable system-of-record state until accepted by an owning service. + +### 3.2 Single-Docker application → domain-oriented modular MSA + +A single-Docker application was considered as a simple deployment option. Current protected main contains independent bounded services, gateway/BFF composition, service-owned persistence, NATS/event boundaries, Docker Compose composition, and a provider-neutral Kubernetes reference. + +**Canonical status:** `Superseded` as the durable architecture. Compose remains a supported deployment/composition profile, not permission to collapse service ownership. + +### 3.3 UUIDv7 proposal → UUIDv4 protected-main invariant + +The original 2026-08-02 design proposed UUIDv7. Current protected-main agent/architecture contracts, migrations and tests use opaque UUIDv4 internal identifiers and forbid provider-native numeric IDs as internal primary keys. + +**Canonical reconciliation:** **UUIDv7 proposal → UUIDv4 protected-main invariant**. UUIDv7 is `Superseded`; UUIDv4 remains current until a future reviewed ADR safely changes it. + +### 3.4 Old “post-MVP” labels → implemented bounded contexts + +The original design treated calendar synchronization, notifications, review, AI assistance and plugin/integration capabilities as post-MVP. Protected main now includes material implementations of those capabilities plus purpose-bound privacy access, backup/restore and a production deployment reference. + +**Canonical status:** old phase labels are historical planning evidence, not current product-status truth. + +### 3.5 Autonomous OpenCode loop: active PR → protected main + +During PR #126, PR #122 completed its exact-head CI/security/CodeRabbit gates and was guarded-squash-merged as `876850018a17323900844e79845ba395b7bf6a9a`. + +**Canonical status:** the bounded hourly OpenCode commercial-development workflow is `Implemented on protected main`; model output still has no product-data/merge/release authority and remains subject to deterministic policy and normal review/security/exact-head gates. PR #133 is a separate `Implemented on active PR` hardening path and must not be promoted early. + +## 4. Exact canonical status vocabulary + +Canonical requirement/diagram/ADR status fields use only: + +- **Implemented on protected main** +- **Implemented on active PR** +- **Partial** +- **Accepted architecture** +- **Planned** +- **Research only** +- **Superseded** +- **Out of scope** + +Scope qualifiers, PR numbers, `reference` wording, implementation notes, and evidence belong in separate prose/evidence columns. Composite status strings are a documentation defect. + +## 5. Canonical documentation graph + +PR #126 establishes this repository-wide hierarchy: + +1. `docs/PRD.md` — product outcomes, users, requirements, journey, status. +2. `docs/TRD.md` — shared technical/runtime/security/readiness/release requirements. +3. `ARCHITECTURE.md` — durable bounded contexts, authority and architecture invariants. +4. `docs/adr/README.md` + ADRs — material decisions and supersession history. +5. `docs/DATA_MODEL.md` — logical service-owned data model and ERD. +6. `docs/UML.md` — component/sequence/state/deployment/failure views. +7. `docs/API_CONTRACTS.md` — HTTP/event/provider contract ownership/version/evolution registry. +8. `SECURITY.md` — vulnerability reporting and upstream security policy. +9. `docs/THREAT_MODEL.md` — assets, trust boundaries, threats, controls and residual risk. +10. `docs/PRIVACY_DATA_LIFECYCLE.md` — sensitive-data collection/access/retention/export/erasure/backup lifecycle. +11. `docs/TEST_STRATEGY.md` — deterministic/live test separation, coverage and release evidence. +12. `docs/OPERABILITY.md` — deployment profiles, diagnostics, dependency degradation, backup/recovery and operator responsibility. +13. `docs/RELEASE_AND_MIGRATION.md` — versions, schema/API migration, rollout, rollback and release gates. +14. `docs/STANDARDS_TRACEABILITY.md` — normative/guidance/research evidence classes and implementation mapping. +15. `docs/TRACEABILITY.md` — requirement/ADR/capability → code/test/runbook/issue/active-PR evidence. +16. `docs/DOCUMENTATION_ASSESSMENT.md` — this completeness/supersession assessment. +17. `docs/operations/`, `docs/research/`, `docs/legal/`, `docs/superpowers/specs/`, `docs/superpowers/plans/` — scoped supporting evidence. +18. `CHANGELOG.md` — buyer-visible unreleased/released changes. + +The historical `docs/superpowers/specs/2026-08-02-life-os-design.md` remains available but is not a parallel current PRD/TRD where canonical docs/ADRs supersede it. + +## 6. Machine-checkable fitness rules + +The documentation contract on PR #126 is intended to protect semantic rather than existence-only invariants: + +- canonical files exist; +- README canonical local links resolve to real repository targets and cannot escape the repository; +- Markdown fences remain balanced; +- canonical PRD/traceability/diagram/ADR statuses use the exact vocabulary above; +- the ADR index points to exact ADR files and the material ADR set includes ADR-0001 through ADR-0008; +- current architecture retains UUIDv4/multi-user-MSA/supersession truth; +- representative protected-main source/configuration paths still exist for planning, OpenCode automation, privacy, backup and Kubernetes claims; +- OpenCode automation still uses the approved NVIDIA credential boundary rather than a Copilot development credential; +- logical ERD does not imply cross-service database authority; +- AI proposal authority remains inert; +- API/privacy/release/standards contracts retain explicit boundaries; +- traceability includes active PR #127, #131 and #133 plus issue #130 instead of presenting them as protected-main work. + +A previously resolved review comment does not waive these exact-current-head checks. + +## 7. Important product gaps and active implementation paths + +### Issue #121 / PR #127 — durable Today multi-device synchronization + +Durable planning and the Today action loop exist on protected main. PR #127 is the active implementation path for the complete bounded versioned Today aggregate, explicit local-draft migration, strong optimistic concurrency, idempotency, PostgreSQL persistence and conflict/recheck browser journey. The canonical requirement is `Implemented on active PR`, not protected main. + +### Issue #55 — complete tenant export and deletion orchestration + +The identity-owned data-rights core already provides deterministic export/erasure coordination semantics, but concrete domain adapters, durable request/receipt/reconciliation, recent-auth gateway enforcement, encrypted export delivery/expiry/download audit, legal-hold/backup-expiry behavior, and operator-visible stuck-request recovery remain incomplete. + +### Issue #129 — hosted per-user calendar credentials + +Conflict-safe CalDAV/Google provider adapters exist. A deployment-wide `GOOGLE_CALENDAR_ACCESS_TOKEN` is not a multi-user product credential model. #129 tracks encrypted per-user credential persistence, OAuth callback/state/PKCE, refresh/revocation, calendar discovery/selection, cross-user isolation and safe migration from the development token. + +### Issue #130 — plugin runtime last mile + +Versioned manifest/event validation exists, but installation grants, encrypted plugin secret lifecycle, SSRF-safe outbound delivery, bounded retries/audit, and revocation are `Planned` rather than implied by the current validation/preparation surface. + +### Issue #128 / PR #131 — readiness false gap exhaustion + +Configured capability maturity can reach 100% while accepted buyer journeys remain open. ADR-0008 separates configured capability-evidence maturity from canonical buyer-gap exhaustion. PR #131 is the active implementation path for a versioned repository-owned buyer-gap registry and `open`/`resolved`/`unknown` reconciliation without turning arbitrary issue prose into executable policy. + +### PR #133 — autonomous development runtime hardening + +The #122 OpenCode loop is already protected-main behavior. PR #133 is active technical hardening for explicit NVIDIA model catalog resolution and real digest-pinned PostgreSQL/NATS Compose verification. It must remain active-PR evidence until exact integration. + +## 8. Fitness rules for future documentation changes + +The canonical graph is fit only when all applicable conditions hold: + +- planned/active-PR behavior is not presented as protected-main behavior; +- exact status fields use only the canonical vocabulary; +- service names, identifiers, authority, external providers and failure states match current source; +- every material PRD requirement has evidence or explicit issue/PR/status; +- every repository-wide material architecture/governance decision has an ADR or explicit architecture invariant; +- ERD cross-service relationships remain logical, not hidden SQL coupling; +- UML/API/event diagrams/contracts match owning-service authority and versions; +- security policy and threat model remain separate but consistent; +- privacy lifecycle does not promise completed deletion/retention/export behavior without exact evidence; +- release/rollback docs do not imply database/external side-effect reversibility that is not implemented; +- standards/research references state source class/publication status and map to executable product evidence; +- deployment docs do not invent infrastructure, credentials, certification or unmeasured SLA/RPO/RTO; +- active PR closure/replacement/rebase triggers status revalidation; +- documentation-discovered gaps enter the executable maintenance/product backlog rather than terminating the run. + +## 9. Final assessment + +**Before this work:** ADR/PRD/TRD/Architecture/UML/ERD and related documentation were not sufficiently consolidated for whole-product governance despite extensive feature-level material. + +**Current PR #126 family coverage:** PRD, TRD, Architecture, ADRs, UML, logical ERD/data model, API/event registry, security/threat model, privacy lifecycle, test strategy, operability, release/migration, standards traceability and requirement/evidence traceability are now represented at a level suitable for a canonical baseline. + +**Current exact-head caveat:** a semantic regression was discovered after earlier review resolution, so file presence or resolved threads alone cannot establish sufficiency. The branch now includes stricter status/link/ADR/source/active-PR regression contracts, but their exact current head still requires CI/security/review verification and protected merge. + +**Product completion remains false:** #127/#121, #55, #129, #130, #131/#128, #133, and integrated stable-release acceptance remain work. The autonomous maintenance loop must continue after documentation work and must not treat this assessment as an endpoint. diff --git a/docs/OPERABILITY.md b/docs/OPERABILITY.md new file mode 100644 index 00000000..88f4555b --- /dev/null +++ b/docs/OPERABILITY.md @@ -0,0 +1,211 @@ +# LifeOS Operability and Deployment Boundary + +**Baseline:** protected `main` at `876850018a17323900844e79845ba395b7bf6a9a` + +## 1. Purpose + +This document is the operator-facing index for what upstream LifeOS supplies and what a deployment owner must supply. Detailed procedures remain in `docs/operations/`. + +LifeOS does not turn a reference Compose/Kubernetes configuration into an operated SLA, managed database, managed cluster, managed identity provider, or managed secret store. + +## 2. Deployment profiles + +### Local development / Compose + +**Status:** Implemented on protected main + +Repository composition supports local development/self-hosting with LifeOS services and declared local dependencies. Operators still supply valid environment secrets/provider registrations where a feature needs them. + +Use this profile for: + +- contributor development; +- disposable integration testing; +- self-hosted evaluation; +- recovery/backup rehearsal where the runbook permits it. + +Do not infer production HA, PITR, ingress/TLS or secret-manager guarantees from local Compose. + +### Portable self-hosted runtime + +**Status:** Partial + +The repository provides a modular self-hostable composition and explicit service ownership, while production-grade external PostgreSQL/NATS, secret management, network/ingress, provider credentials and operated availability remain deployment-owner responsibilities. This status describes upstream operational completeness, not a collapse of the accepted modular architecture. + +### Kubernetes production reference + +**Status:** Implemented on protected main + +Current repository artifacts encode a provider-neutral hardened reference with non-root/read-only containers, probes, resource bounds, rolling update/disruption/topology/network policies and protected deployment workflow behavior. `Reference` is a scope qualifier, not a separate status value. + +They deliberately do **not** provision: + +- a Kubernetes cluster/control plane; +- PostgreSQL; +- NATS JetStream; +- ingress controller/WAF; +- TLS certificates or DNS; +- image registry/build pipeline; +- KMS/secret manager; +- cloud IAM/private networking; +- operator monitoring backend. + +## 3. Operator-owned configuration + +Deployment owners are responsible for, at minimum: + +- Google/GitHub OAuth application registration and redirect policy; +- database/NATS credentials and network access; +- signing/encryption/provider keys; +- per-environment allowed origins; +- external calendar/model provider tenant/accounts; +- production ingress/TLS/DNS; +- encrypted backup storage and retention; +- monitoring/alert routing; +- incident response/on-call processes; +- legal basis, privacy notice, retention/deletion policy and subprocessors; +- security patch deployment and dependency upgrade cadence. + +Upstream repository defaults/examples are not production secrets. + +## 4. Health, readiness and metrics + +Services expose health/readiness/metrics according to their implemented contracts. Health endpoints answer process/component questions only to the extent documented by the owning service; they must not claim a downstream provider is usable if the probe does not test that dependency. + +Prometheus-compatible metrics are an **operator surface**. Production ingress must restrict metrics appropriately because labels/counts/timing can expose operational or tenant-sensitive metadata even without raw personal content. + +Public diagnostics must remain: + +- bounded; +- credential-free; +- free of raw stack traces and dependency bodies; +- stable enough to classify retry/conflict/unavailable behavior. + +## 5. Observability + +Use existing service/runbook contracts for exact metric names and SLO calculations. Repository-wide expectations are: + +- correlation/request identifiers where applicable; +- structured logs; +- bounded metric label cardinality; +- request/error/latency/queue/database saturation evidence appropriate to the service; +- no raw personal action text, provider tokens, prompts/responses or browser cookies in telemetry; +- exact provider/model/deployment provenance in retained evidence when needed without retaining secret values. + +Canonical architecture documentation does not invent global numeric SLA/SLO targets. Numeric objectives belong to measured operator/service-specific SLO documents. + +## 6. External dependency degradation + +| Dependency | Expected degradation | +| --- | --- | +| Google/GitHub identity provider | New login/link flows may fail; unrelated domain data must not fabricate authentication success | +| PostgreSQL for owning service | Durable operations for that service fail closed/classified; no in-memory fallback that pretends persistence succeeded | +| NATS | Event-driven projections/notifications may lag/fail according to owning contract; source-of-truth DB writes do not become unauthorized cross-service writes | +| Calendar provider | Synchronization returns retry/conflict/unavailable evidence; local LifeOS truth remains intact | +| Model/contextual-orchestrator | AI live proposal/conformance may be unavailable; deterministic LifeOS gates remain meaningful | +| Plugin/external integration | That integration fails independently; no direct database repair/access | + +A degraded external provider is never permission to weaken tenant/auth/security validation. + +## 7. Backup and recovery + +**Current tier:** verified PostgreSQL logical dump/restore. + +Protected-main backup tooling/runbook provides: + +- private custom-format archive; +- checksum/non-secret metadata; +- restore checksum verification; +- explicit empty-target requirement; +- integration evidence for corruption and unsafe-target refusal. + +It does **not** claim: + +- point-in-time recovery; +- automatic backup scheduling; +- encrypted off-site replication; +- retention enforcement; +- continuous WAL archival; +- a universal RPO/RTO. + +Deployment owners establish independent encrypted storage and recovery objectives. If required RPO is shorter than logical dump cadence, implement/rehearse WAL/PITR at the operated PostgreSQL layer. + +See `docs/operations/backup-and-restore.md`. + +## 8. Migration and rollout + +- Database migrations are service owned. +- Production workflow migration/application order must be explicit. +- A completed forward migration is not automatically reversed by workload rollback. +- Changes that require compatibility windows document old/new application/schema coexistence. +- Migration failure evidence is credential-free and does not hide partial state. +- Destructive data migration requires explicit backup/recovery and review evidence. + +## 9. Deployment rollback + +The reference deployment workflow captures the existence/revision of covered workloads before apply, performs protected-environment dry-run/diff, and verifies rollback/deletion semantics for the workload resources it claims to recover. + +Do not overstate rollback: + +- completed DB migrations may persist; +- namespace/policy/external infrastructure may not be reversed; +- first-time workloads may require verified deletion rather than revision rollback; +- recovery failure is a distinct failure, not success-with-warning. + +See `docs/operations/production-deployment.md`. + +## 10. Secrets and key rotation + +- Secrets enter runtime through protected environment/secret-management boundaries, not Git. +- Browser/model/public artifacts never receive unrelated service credentials. +- AI/model live tests use `NVIDIA_NIM_API_KEY` only at the approved model boundary. +- Review-agent credentials are not repurposed for development agents. +- Signed service-context keys use explicit active/overlap/retirement semantics where implemented. +- Provider-specific token rotation/revocation belongs to the owning integration/identity service and operator configuration. + +Hosted Google Calendar unattended synchronization remains incomplete until durable per-user credential storage/refresh/revocation is implemented and operationally documented under issue #129. + +## 11. Incident categories + +At minimum classify incidents into: + +- authentication/session/provider failure; +- cross-tenant authorization suspicion; +- database availability/integrity; +- replay/duplicate side effect; +- notification/calendar provider degradation; +- model/provider/prompt-injection anomaly; +- privacy grant/access anomaly; +- backup/restore failure; +- deployment/rollback failure; +- CI/supply-chain/release compromise; +- data/credential leakage. + +Security-sensitive incidents follow private vulnerability/incident channels rather than public issue detail. + +## 12. Operational evidence for release + +Before a stable release, verify on one exact integrated protected head: + +- CI/security/dependency checks; +- package/container build/smoke evidence; +- owned-code coverage gates; +- supported PostgreSQL migration/integration behavior; +- backup/restore rehearsal; +- production-reference render/dry-run/deployment contract tests; +- accessibility/localization browser journeys; +- SBOM/provenance according to release workflow; +- runbook/doc links for changed operator behavior; +- no obsolete temporary repair workflow/source. + +## 13. Runbook index + +Canonical operator details live in existing runbooks, including: + +- `docs/operations/service-level-objectives.md` +- `docs/operations/planning-service-level-objectives.md` +- `docs/operations/backup-and-restore.md` +- `docs/operations/production-deployment.md` +- `docs/operations/notification-persistence.md` +- AI proposal/orchestrator assurance and key-rotation runbooks under `docs/operations/` + +Feature runbooks remain authoritative for exact commands/configuration they own; this document supplies the repository-wide operational boundary. diff --git a/docs/PRD.md b/docs/PRD.md new file mode 100644 index 00000000..414b1dad --- /dev/null +++ b/docs/PRD.md @@ -0,0 +1,238 @@ +# LifeOS Product Requirements Document + +**Version:** 1.0-draft +**Baseline date:** 2026-08-09 +**Product:** LifeOS +**Repository:** `ContextualWisdomLab/life-os` + +## 1. Product definition + +LifeOS is an open-source, privacy-first personal operating system that connects long-term goals to projects, tasks, habits, calendar commitments, reviews, and auditable AI assistance. It is designed for multi-user server-backed operation while remaining self-hostable and modular. + +LifeOS is not a flat checklist and AI is not an autonomous owner of user data. The product promise is: + +> **Turn intentions into an explainable, recoverable action loop while keeping the user in authority over personal data and AI-assisted decisions.** + +## 2. Status convention + +Every requirement uses one of these statuses: + +- **Implemented on protected main** +- **Implemented on active PR** +- **Partial** +- **Accepted architecture** +- **Planned** +- **Research only** +- **Superseded** +- **Out of scope** + +Protected-main source and tests, not this document, are authoritative for an `Implemented on protected main` claim. Active-PR status must name the current PR in evidence and must be revalidated whenever its head, state, or ancestry changes. + +## 3. Target users + +### Primary + +- Individuals who want goals, projects, tasks, habits, and reviews in one system. +- Knowledge workers who need to understand why a task exists and what objective it advances. +- Users who need cross-device durable state rather than a browser-only checklist. +- Self-hosters and organizations that want an inspectable open-source personal productivity platform. + +### Secondary + +- Small teams that may later share workspaces and planning objects. +- Developers building integrations or plugins against versioned LifeOS contracts. +- Researchers or advanced users using consented personal-history exports for longitudinal analysis outside the transactional product core. + +### Operators + +- Self-hosters and enterprise platform teams responsible for deployment, PostgreSQL, NATS, secrets, backup storage, identity-provider registration, network policy, monitoring, and incident response. + +## 4. Product principles + +1. **Goal-connected action.** Tasks and habits can be traced to projects/goals where relevant. +2. **User authority.** User-authored state is never silently changed by AI. +3. **Durable ownership.** Server-side domain services own authoritative persisted state; browser-local state is explicitly labeled draft/cache/offline state. +4. **Tenant safety.** Workspace ownership is derived from authenticated context, not trusted from arbitrary client identifiers. +5. **Modular MSA.** Each bounded service owns its contracts, persistence, migrations, tests, runtime, and failure behavior. +6. **Portable deployment.** Compose supports local/self-hosted composition and Kubernetes provides a provider-neutral production reference without hiding operator responsibilities. +7. **Auditable assistance.** AI output is stored as inert proposal/evidence with explicit accept/reject history where implemented. +8. **Recoverability.** Idempotency, optimistic concurrency, backup/restore, bounded retries, and failure evidence are product features rather than operational afterthoughts. +9. **Accessibility and localization.** Core journeys are keyboard operable and Korean/English capable; status is not conveyed by color alone. +10. **Evidence over claims.** Product maturity comes from code-current tests and protected-main evidence, not roadmap labels or a single aggregate score. + +## 5. Historical decisions + +### Superseded: login-free local-first product as primary architecture + +Early exploration considered IndexedDB-only storage with no account. The product later moved to Google/GitHub authentication, personal workspaces, PostgreSQL-backed services, and multi-device durability. Local browser state remains useful for explicit drafts/offline UX, but is not the system of record. + +### Superseded: single application as the durable architecture + +A single Docker application was considered as a fast deployment path. Current LifeOS uses domain-oriented independent services composed by the repository. Docker Compose is a deployment profile, not a collapse of service ownership. + +### Superseded: UUIDv7 internal identifiers + +The initial combined design proposed UUIDv7. Current protected-main repository contracts require opaque UUIDv4 internal identifiers. See ADR-0002. + +## 6. Core customer journey + +The target end-to-end customer journey is: + +1. Sign in with Google or GitHub. +2. Receive or access an authorized personal workspace. +3. Capture an intention/action quickly. +4. Organize durable Goals → Projects → Tasks and recurring Habits. +5. Select realistic Today priorities and scheduled actions. +6. Complete work and retain immutable/idempotent completion evidence where applicable. +7. Receive bounded reminders without duplicate delivery or notification fatigue. +8. Review progress and stalled work in daily/weekly review flows. +9. Synchronize selected commitments with an authorized calendar provider. +10. Optionally request an AI proposal based on bounded context. +11. Inspect proposal evidence and explicitly accept/reject; AI does not silently mutate planning data. +12. Exercise privacy/data-rights controls and recover data through supported export/backup paths. +13. Continue across phone/tablet/desktop through the responsive PWA and durable server state. + +A release should be judged by how many of these steps work end-to-end, not by the number of services, configured capability rows, or abstractions present. + +## 7. Functional requirements + +### Identity and workspace + +| ID | Requirement | Status | Representative evidence | +| --- | --- | --- | --- | +| PRD-ID-001 | Support Google and GitHub login through bounded OAuth/OIDC provider contracts. | Implemented on protected main | `identity.oauth-session`; identity integration tests | +| PRD-ID-002 | Issue revocable workspace-scoped browser sessions without exposing provider secrets to downstream services. | Implemented on protected main | identity runtime/session tests | +| PRD-ID-003 | Internal IDs are opaque UUIDv4 values; numeric provider IDs remain provider metadata only. | Implemented on protected main | `AGENTS.md`, `ARCHITECTURE.md`, migration/tests | +| PRD-ID-004 | Derive workspace/user authority from authenticated context and fail closed on cross-tenant access. | Implemented on protected main | identity/planning/AI/privacy tests | +| PRD-ID-005 | Support future non-personal workspace memberships without weakening personal-workspace isolation. | Accepted architecture | initial domain model; no team-workspace GA claim | + +### Goals, projects, tasks and Today + +| ID | Requirement | Status | Representative evidence | +| --- | --- | --- | --- | +| PRD-PLAN-001 | Persist goals, projects, milestones and tasks across process restarts. | Implemented on protected main | `planning.durable-data` | +| PRD-PLAN-002 | Provide tenant-safe search over durable planning objects with bounded Unicode-normalized behavior. | Implemented on protected main | `capture.search` | +| PRD-PLAN-003 | Provide a fast capture experience without presenting local drafts as durable records. | Implemented on protected main | quick-capture component/e2e | +| PRD-PLAN-004 | Provide a Today action loop with bounded priorities and completion workflow. | Implemented on protected main | `today.action-loop` | +| PRD-PLAN-005 | Prevent stale concurrent updates from silently overwriting newer durable Today state across devices. | Implemented on active PR | issue #121; PR #127 implements the versioned durable Today aggregate, explicit local-to-durable migration, strong preconditions, idempotency, PostgreSQL persistence, conflict reconciliation, and browser acceptance; it is not protected-main evidence until merge | +| PRD-PLAN-006 | Keep planning source-of-truth inside planning-service; review/search projections never become mutation authority. | Implemented on protected main | service ownership and repository tests | + +### Habits and reviews + +| ID | Requirement | Status | Representative evidence | +| --- | --- | --- | --- | +| PRD-HAB-001 | Define recurring habits and retain completion history independently from recurrence-definition changes. | Implemented on protected main | `habit.recurring-core` | +| PRD-HAB-002 | Make completion recording replay-safe and tenant scoped. | Implemented on protected main | habit PostgreSQL integration tests | +| PRD-REV-001 | Support guided daily/weekly review using durable planning/habit evidence. | Implemented on protected main | `review.guided-loop` | +| PRD-REV-002 | Preserve review output as projection/evidence rather than silently rewriting source planning objects. | Implemented on protected main | review service boundary | + +### Calendar and notifications + +| ID | Requirement | Status | Representative evidence | +| --- | --- | --- | --- | +| PRD-CAL-001 | Synchronize selected commitments through explicit provider adapters with idempotency/concurrency protection. | Implemented on protected main | `calendar.time-blocking`; CalDAV/Google tests; completed #51 | +| PRD-CAL-002 | Avoid duplicate/destructive provider updates through deterministic identifiers and strong preconditions. | Implemented on protected main | provider integration tests | +| PRD-CAL-003 | Store/refresh/revoke per-user Google Calendar credentials for hosted multi-user use. | Partial | issue #129; current adapter still uses an operator-supplied runtime token rather than a complete per-user credential lifecycle | +| PRD-NOT-001 | Deliver timezone-correct bounded reminders with fatigue controls. | Implemented on protected main | `notifications.reminders` | +| PRD-NOT-002 | Recover expired claims/retries without duplicate inbox delivery. | Implemented on protected main | notification integration tests | + +### AI assistance and development automation + +| ID | Requirement | Status | Representative evidence | +| --- | --- | --- | --- | +| PRD-AI-001 | Treat model output as an inert proposal, never an execution command. | Implemented on protected main | AI service architecture/tests | +| PRD-AI-002 | Persist proposal evidence before return and retain replay-safe accept/reject decision history. | Implemented on protected main | AI audit persistence/tests | +| PRD-AI-003 | Derive actor/workspace context from the authenticated web boundary and never forward browser credentials. | Implemented on protected main | same-origin AI BFF tests | +| PRD-AI-004 | Keep deterministic proposal-quality/safety evaluation independent of live-provider availability. | Implemented on protected main | proposal evaluator + NIM conformance split | +| PRD-AI-005 | Permit deeper orchestration only when measured quality/control evidence justifies it against a strong single-route baseline. | Accepted architecture | `ARCHITECTURE.md` and the protected-main NIM conformance harness provide evaluation support; architecture status does not claim every orchestration profile is shipped | +| PRD-AI-006 | Autonomous developer automation may create bounded reviewed work but may not become product data authority. | Implemented on protected main | PR #122 merged as `876850018a17323900844e79845ba395b7bf6a9a`; `.github/workflows/opencode-commercial-development.yml`; `packages/commercial-development-agent/`; PR #133 is an active hardening follow-up rather than protected-main evidence | + +### Privacy, security and data rights + +| ID | Requirement | Status | Representative evidence | +| --- | --- | --- | --- | +| PRD-PRIV-001 | Protect sensitive data through purpose-bound authorization, tenant scope, encryption/secret boundaries and auditable access rather than blanket masking. | Implemented on protected main | privacy-service work merged in #124 | +| PRD-PRIV-002 | Make privacy access decisions/evidence append-only and grants bounded/single-use where designed. | Implemented on protected main | privacy PostgreSQL/security tests | +| PRD-PRIV-003 | Keep credentials, raw prompts/responses, hidden reasoning and unbounded tenant content out of retained public artifacts. | Implemented on protected main | repository-wide contracts/security tests | +| PRD-PRIV-004 | Provide user-facing export/deletion lifecycle with durable job/audit evidence before claiming complete data-rights UX. | Partial | issue #55 remains open; PR #134 is the active bounded implementation for preserving real authentication age across session rotation, one prerequisite for recent-auth enforcement, while concrete domain participation, durable orchestration/reconciliation, retention, protected delivery, and audit lifecycle remain incomplete | + +### Integration and extensibility + +| ID | Requirement | Status | Representative evidence | +| --- | --- | --- | --- | +| PRD-INT-001 | Expose a versioned plugin contract without granting plugins direct cross-service database authority. | Implemented on protected main | plugin SDK + integration service | +| PRD-INT-002 | Validate untrusted plugin manifests/events and preserve tenant-scoped provenance. | Implemented on protected main | plugin contract tests | +| PRD-INT-003 | Add installation, secret persistence and outbound plugin delivery only behind separately reviewed authorization/SSRF/audit boundaries. | Planned | issue #130 tracks installation grants, encrypted secret lifecycle, SSRF-safe delivery, bounded retry/audit and revocation; the protected-main surface remains validation/preparation only | +| PRD-INT-004 | Compose with optional CWL services through stable interfaces while preserving standalone LifeOS operation. | Accepted architecture | architecture/agent contracts | + +### Web, PWA, accessibility and localization + +| ID | Requirement | Status | Representative evidence | +| --- | --- | --- | --- | +| PRD-WEB-001 | Provide responsive installable PWA behavior on phone/tablet/desktop. | Implemented on protected main | `mobile.pwa` | +| PRD-WEB-002 | Make core journeys keyboard operable with visible focus and non-color-only status. | Implemented on protected main | accessibility tests/e2e for current core flows | +| PRD-WEB-003 | Provide structurally complete Korean and English message catalogs. | Implemented on protected main | `accessibility.localization` / changelog | +| PRD-WEB-004 | Preserve/recover explicitly local offline drafts without silently uploading them. | Implemented on active PR | PR #127 covers the explicit durable Today migration/conflict/recheck flow for the current bounded Today slice; broader offline behavior must not be inferred beyond that reviewed scope | + +### Readiness and product governance + +| ID | Requirement | Status | Representative evidence | +| --- | --- | --- | --- | +| PRD-GOV-001 | Report configured capability-evidence maturity separately from canonical buyer-gap exhaustion so a 100% capability score cannot imply a complete product while accepted buyer journeys remain open or unknown. | Implemented on protected main | issue #128 is closed as completed; PR #131 merged as protected main `2ad45a935283e83d9bb0f3ea5c3d23b2669078b1` with repository-owned `product/buyer-gaps.json`, explicit `open`/`resolved`/`unknown` reconciliation, and separate report dimensions; ADR-0008 records the durable decision | +| PRD-GOV-002 | Treat arbitrary issue/review/model prose as untrusted evidence rather than executable product policy. | Implemented on protected main | PR #131 consumes repository-owned gap identity and bounded issue-state evidence while keeping issue body/comment/review/model text non-executable | + +### Backup, deployment and operations + +| ID | Requirement | Status | Representative evidence | +| --- | --- | --- | --- | +| PRD-OPS-001 | Provide verified PostgreSQL logical backup/restore with corruption and unsafe-target refusal. | Implemented on protected main | backup scripts/tests/runbook | +| PRD-OPS-002 | Provide provider-neutral production reference deployment with restricted runtime defaults and explicit operator-owned dependencies. | Implemented on protected main | Kustomize/deployment tests/runbook; `reference` describes scope, not status | +| PRD-OPS-003 | Expose bounded health/readiness/metrics and preserve monitoring data as an operator-only surface. | Implemented on protected main | service endpoints/runbooks | +| PRD-OPS-004 | Do not claim point-in-time recovery, managed cluster provisioning, or a fixed public SLA without measured/operator-specific evidence. | Accepted architecture | README/runbooks explicitly preserve these operator-owned/non-claimed boundaries | + +## 8. Non-functional requirements + +### Security and privacy + +- Fail closed on malformed ownership, identifiers, signatures, digests, timestamps, untrusted provider responses, and configuration. +- Keep SQL structure static and parameterize dynamic values. +- Protect service credentials and third-party access tokens from browser/model/artifact exposure. +- Preserve tenant isolation in every repository and integration test that crosses a service persistence boundary. +- Treat external responses, connector data, model output, calendar payloads and plugin manifests as untrusted. + +### Reliability and consistency + +- Mutations that may be replayed use idempotency controls. +- Concurrent/stale mutations use explicit version/digest/ETag/precondition mechanisms where loss is plausible. +- Durable workers support bounded retries/claims and expose sanitized failure states. +- Cross-service event consumers are idempotent where at-least-once delivery is possible. + +### Quality + +- Public production declarations have beginner-readable documentation. +- Packages that define exact coverage gates maintain 100% statement/branch/function/line coverage with meaningful tests. +- Core customer journeys include realistic PostgreSQL and browser integration evidence rather than mock-only success. +- Accessibility/localization/security/backup/release behavior is tested as product behavior. + +### Operability + +- A self-hoster can understand what LifeOS provides versus what the operator must provision. +- Failures expose credential-free diagnostics and correlation/evidence identifiers. +- Release claims require exact integrated CI/security/package/provenance/recovery evidence. + +## 9. Explicit non-goals + +LifeOS does not claim or provide as a product contract: + +- medical or psychological diagnosis or treatment; +- autonomous consequential employment, admission, insurance, credit, legal or similar decisions; +- silent AI mutation of user-owned state; +- guaranteed availability of external model/calendar/identity providers; +- a proprietary Google-operated backend hidden behind the open-source project; +- cross-service direct database writes; +- certification claims (for example SOC 2 or CSAP) without independent certification evidence; +- a public SLA/RPO/RTO value that has not been measured for a specific operated deployment. + +## 10. Release outcome + +The first stable release requires more than individual capability maturity. It requires one protected integrated head where the primary customer journey, canonical buyer-gap state, tenant/privacy boundaries, migrations, backup/restore, deployment, accessibility/localization, packaging/SBOM/provenance, security checks, and required review gates all pass together. Versioning and `CHANGELOG.md` release sections are updated only after that evidence exists. diff --git a/docs/PRIVACY_DATA_LIFECYCLE.md b/docs/PRIVACY_DATA_LIFECYCLE.md new file mode 100644 index 00000000..a3d845c5 --- /dev/null +++ b/docs/PRIVACY_DATA_LIFECYCLE.md @@ -0,0 +1,206 @@ +# LifeOS Privacy and Data Lifecycle Contract + +**Baseline:** protected `main` at `876850018a17323900844e79845ba395b7bf6a9a` + +## 1. Scope + +This document describes the **software data lifecycle and authority model**. `docs/legal/privacy.md` describes upstream-project privacy practices, not every deployment. Independent deployment operators determine their own legal basis, notice, retention schedule, subprocessors, residency, incident process, and regulatory obligations. + +LifeOS may contain sensitive goals, career, health, relationship, habit, calendar, review and AI-context data. The software therefore treats privacy as an access/lifecycle architecture problem, not a global masking toggle. + +## 2. Data classes + +| Class | Examples | Default handling | +| --- | --- | --- | +| Identity/authentication | internal user/workspace IDs, external identity mapping, sessions | identity-owned, minimum necessary, no provider credentials downstream | +| Personal planning | goals, projects, tasks, Today drafts/state | planning-owned, tenant-scoped, user-authorized | +| Behavioral history | habit completion, review evidence, notification outcomes | owning-service persistence; immutable/append-only where defined | +| Integration credentials | calendar access/refresh tokens, provider account identity | secret/encrypted lifecycle; current hosted per-user calendar lifecycle is Partial; tracking issue #129 | +| AI proposal evidence | proposal content/digest, explicit decisions, provider/model provenance | inert/auditable, bounded; no raw hidden reasoning/credential retention | +| Privileged privacy evidence | purpose decisions, grants, access events | privacy-owned, content-minimized, append-only where defined | +| Operational metadata | correlation IDs, latency/counts/errors | bounded labels/content; avoid personal text/secrets | +| Backup/export artifacts | logical database backups, future user export archives | private, integrity-verified, explicit access/expiry/retention controls | +| Public project data | GitHub commits/issues/reviews/synthetic fixtures | upstream-project privacy notice; never use real user/customer sensitive data as fixtures | + +## 3. Lifecycle states + +```mermaid +stateDiagram-v2 + [*] --> Collected: authorized product input/provider callback + Collected --> Validated: bounded + tenant/authority checked + Validated --> Durable: owning service accepts persistence + Validated --> Rejected: invalid/unauthorized/oversized + Durable --> Active: normal product use + Active --> Archived: backup/export according to explicit policy + Active --> Deleted: authorized erasure completed and verified + Archived --> Expired: operator/user retention policy disposes artifact + Rejected --> [*] + Deleted --> [*] + Expired --> [*] +``` + +A browser-local draft is not `Durable` merely because it exists on a device. An AI proposal is not an authorized planning mutation merely because the user viewed it. + +## 4. Collection minimization + +- Collect only fields required by the owning product contract. +- Do not turn external provider payloads into internal durable records wholesale; map validated required fields. +- Provider-native IDs remain external mappings, not internal primary keys. +- Public/open-source fixtures are synthetic and must not resemble live credentials or unnecessary personal records. +- Model prompts/context are bounded to the selected operation rather than arbitrary account history. + +## 5. Tenant and purpose authority + +### Ordinary product data + +Owning services derive workspace/actor authority from authenticated/signed context and scope every persistent query/mutation to that authority. + +### Privileged sensitive access + +Where unmasked/sensitive data access requires privilege beyond ordinary user ownership, privacy-service evaluates actor, workspace/resource, purpose, operation and lifetime, persists a decision, and may issue a bounded/single-use grant. Expired/reused/wrong-purpose/wrong-resource grants fail closed. + +Redaction/masking is applied when disclosure is unnecessary; it is not a substitute for authorization. + +## 6. Credentials and cryptographic material + +Credentials are never ordinary domain JSON. + +- OAuth/calendar/model/GitHub/review-agent credentials remain in their owning secret boundary. +- Browser session cookies are not forwarded to downstream model/domain providers. +- Model providers do not receive GitHub/review-agent credentials. +- Public logs/errors/artifacts do not contain tokens, cookies, authorization headers, private keys or raw provider error bodies. +- Hosted per-user calendar credentials require encrypted durable storage, refresh/revocation and provider-selection evidence before LifeOS claims the capability complete; tracking issue #129. +- Signing/encryption key rotation follows explicit active/overlap/retirement rules where implemented. + +## 7. Data retention + +The upstream repository does **not** prescribe one universal runtime retention period because independent deployments differ in legal/operational requirements. + +Software requirements: + +- every retention-sensitive artifact has an identifiable owner and purpose; +- public CI/model artifacts use bounded retention configured by repository policy; +- backups/exports require operator/user retention and disposal policy; +- privileged access audit evidence is retained independently from unnecessary sensitive payload; +- an operator can determine which service/artifact owns a record that must be retained or erased; +- a legal hold, where supported, blocks destructive erasure explicitly rather than silently retaining data after claiming deletion. + +A fixed retention number must not be added to canonical docs unless code/operator policy and legal basis support that number. + +## 8. Export lifecycle + +**Status:** Partial +**Tracking:** issue `#55`. + +The identity-owned data-rights core can deterministically coordinate contributor exports and produce digest evidence. Complete customer-facing export requires all required domain contributors and delivery/storage lifecycle. + +Target flow: + +```mermaid +sequenceDiagram + actor Owner as Workspace Owner + participant Gateway + participant Rights as Data Rights Coordinator + participant Domains as Registered Domain Contributors + participant Archive as Protected Export Delivery + + Owner->>Gateway: Authenticated export request + Gateway->>Rights: Trusted workspace/actor + request identity + Rights->>Domains: Export exact tenant-owned registered data + Domains-->>Rights: Versioned bounded sections + Rights->>Rights: Canonicalize + digest + completeness check + Rights->>Archive: Encrypt/store/stream according to reviewed delivery contract + Archive-->>Gateway: Expiring authorized handle + audit evidence + Gateway-->>Owner: Machine-readable export access +``` + +No export is called complete if a required registered domain is missing or a section silently fails. + +## 9. Erasure lifecycle + +**Status:** Partial +**Tracking:** issue `#55`. + +Current data-rights core supports preflight, deterministic contributor order, fail-closed execution, verification and bounded receipt semantics with test contributors. Complete productization requires concrete domain adapters, durable request/receipt/reconciliation, authenticated recent-owner confirmation, retention/legal-hold/backup-expiry evidence and operator-visible stuck-request recovery. + +Target invariants: + +- exact workspace owner/recent-auth authority; +- explicit destructive confirmation; +- canonical registered participant set; +- all participants preflight before destructive commit begins; +- exact request/idempotency identity; +- partial completion is never reported as full deletion; +- remaining partial state is observable/reconcilable; +- legal hold/retention blockers are explicit; +- backup copies are governed by documented expiry/recovery policy rather than falsely claimed erased immediately if immutable retention requires otherwise. + +## 10. Backups + +Protected-main logical backup/restore is an operator recovery capability, not a user export API. + +- Archives are private and checksum verified. +- Restore refuses a corrupted archive or unsafe non-empty target. +- Upstream does not claim automatic encrypted off-site storage, scheduling, retention or PITR. +- Operators must reconcile backup retention with deletion/hold policy for their deployment. + +## 11. Logs, metrics and diagnostics + +Allowed operational evidence should prefer: + +- opaque correlation/request/job IDs; +- bounded status/reason code; +- service/operation classification; +- duration/count/capacity measures; +- non-secret exact revision/digest where needed for debugging. + +Avoid: + +- goal/task/habit/review text; +- calendar event content; +- raw OAuth/provider/model response bodies; +- cookies/tokens/authorization headers; +- raw prompts/responses/hidden reasoning; +- database DSNs/passwords/private keys; +- unbounded stack traces in public responses/artifacts. + +## 12. AI data lifecycle + +- Bounded user context enters the AI proposal boundary only for the selected operation. +- Model output is untrusted and validated. +- Proposal evidence is persisted before it is returned where the audit contract requires it. +- Explicit accept/reject decisions are append-only/replay-safe evidence. +- A proposal does not become planning state automatically. +- Live conformance artifacts contain bounded metrics/provenance, not raw prompts/responses/secrets/hidden reasoning. + +## 13. Integration-provider lifecycle + +Provider state is isolated by provider/account/connection and user/workspace authority. Disconnect/revocation must make a connection unusable according to the provider contract. A process-global token used for a bounded development adapter is not acceptable evidence of a hosted multi-user credential product. + +## 14. Data-rights acceptance gate + +Upgrade `PRD-PRIV-004` from `Partial` only when exact protected-main evidence proves: + +- complete domain registry/coverage; +- authenticated owner/recent-auth boundary; +- durable request/export/deletion receipts; +- retry/reconciliation and partial-failure visibility; +- legal-hold/retention/backup-expiry behavior; +- protected export delivery, expiry and download audit; +- tenant isolation/replay/concurrency/restart tests; +- exact-head security/review/release gates. + +## 15. Operator extension + +Deployment operators must document and validate their actual: + +- purposes/legal bases; +- retention schedules and legal holds; +- encryption/KMS/key rotation; +- data residency/transfers/subprocessors; +- access-control/admin/support processes; +- backup/PITR/expiry; +- data subject/user request process; +- breach/incident response. + +Upstream architecture enables these controls but does not certify an independent deployment automatically. diff --git a/docs/RELEASE_AND_MIGRATION.md b/docs/RELEASE_AND_MIGRATION.md new file mode 100644 index 00000000..bb524a6c --- /dev/null +++ b/docs/RELEASE_AND_MIGRATION.md @@ -0,0 +1,211 @@ +# LifeOS Release, Migration, and Rollback Contract + +**Baseline:** protected `main` at `5c87a7ec3568a4ce47b25cad843f1bc5be91b294` + +## 1. Purpose + +This document defines repository-wide release and change-management invariants. Exact deployment and backup commands remain in `docs/operations/production-deployment.md` and `docs/operations/backup-and-restore.md`. + +A merged feature is not automatically a release. A passing old head is not release evidence for a newer head. + +## 2. Change classes + +| Class | Examples | Minimum evidence | +| --- | --- | --- | +| Documentation-only | canonical docs, runbook correction | links/status/evidence consistency; normal configured checks | +| Application-compatible | additive UI/domain logic without persistent schema change | tests, coverage, security, build, affected E2E | +| API/event contract | route/event/schema semantic change | old/new contract tests, versioning, consumer compatibility | +| Database migration | table/constraint/index/data transform | real prior-state migration test, compatibility/rollback or forward-fix plan | +| External integration | OAuth/calendar/model/plugin provider behavior | provider contract, timeout/retry/origin/credential tests | +| Deployment/runtime | image/config/probes/network/workload behavior | render/dry-run/rollout/rollback evidence | +| Destructive/data-rights | deletion, retention, bulk data transform | explicit authority, backup/recovery, partial failure, immutable audit evidence | +| Security/cryptography | keys, signatures, auth, privilege | threat-model update, negative/replay tests, rotation/recovery | + +## 3. Versioning policy + +Until release readiness is proven, buyer-visible work remains under `CHANGELOG.md` → `Unreleased`. + +A version/tag/release is created only when: + +- the exact protected-main commit is selected as the release candidate; +- all repository-required CI/security/review gates pass for that commit; +- package/container/install smoke tests pass where artifacts are distributed; +- migration/rollback or forward-fix compatibility is proven for changes since the prior release; +- backup/restore/recovery evidence is current where applicable; +- supported web accessibility/localization journeys pass; +- SBOM/provenance/reproducibility evidence required by release policy is produced; +- release notes match the actual integrated artifact. + +Do not increment version solely to retrigger checks or because one PR merged. + +## 4. Database migration contract + +Every schema/data migration identifies: + +1. owning service/schema; +2. exact starting supported state; +3. forward SQL/code transform; +4. constraints/indexes introduced or changed; +5. application compatibility window; +6. data volume/locking/timeout assumptions; +7. rollback or forward-fix decision; +8. backup/recovery prerequisites for destructive transformations; +9. real PostgreSQL migration tests; +10. operator-visible failure/recovery behavior. + +### Expand/contract preference + +For changes requiring mixed-version operation, prefer: + +```text +expand schema +→ deploy readers/writers compatible with old + new +→ backfill/reconcile with idempotent evidence +→ switch authority +→ observe +→ contract obsolete schema only after no supported consumer depends on it +``` + +Avoid requiring every service to deploy simultaneously unless the release explicitly proves that atomic coupling. + +## 5. Migration failure states + +A migration may fail: + +- before any change; +- after DDL but before backfill; +- during batch backfill; +- after data transform but before application rollout; +- after new application deploy with old schema still present; +- after contract/removal. + +The migration plan must say which states are transactionally rolled back, forward-fixed, retried, or restored. “Run rollback” is not sufficient if PostgreSQL/app behavior cannot actually reverse the completed transformation safely. + +## 6. Backup prerequisite + +Before a destructive/high-risk production migration, the operator verifies the recovery tier appropriate to the risk. + +Current upstream LifeOS proves logical PostgreSQL dump/restore behavior, including checksum and unsafe-target refusal. It does not prove PITR or a deployment-specific RPO/RTO. Operators requiring tighter recovery use rehearsed managed/PostgreSQL WAL/PITR outside the current upstream logical-dump tier. + +## 7. API and event migration + +Breaking public/internal semantics require a new version or compatibility adapter. + +During a compatibility window: + +- old version remains explicitly supported or fails predictably; +- producer/consumer tests cover both sides; +- event replay/at-least-once behavior is understood; +- unknown versions fail closed; +- removed fields/operations are not silently reinterpreted; +- client migration and removal conditions are documented. + +See `docs/API_CONTRACTS.md`. + +## 8. Key/credential migration + +Key rotation is staged: + +```text +introduce new key identifier +→ begin signing/encrypting with new active key +→ retain narrowly bounded previous-key verification/decryption where required +→ migrate/refresh dependent state +→ verify usage has moved +→ retire previous key +→ reject retired identifier immediately +``` + +Never overwrite a key in place while callers cannot distinguish old/new material. Provider credential migrations preserve account/tenant binding and revocation semantics. + +## 9. Application rollout + +A rollout must distinguish: + +- health (process/runtime alive); +- readiness (safe to receive the service's production traffic); +- migration success; +- external provider availability; +- actual customer-journey success. + +A green liveness probe alone is not release acceptance. + +The Kubernetes reference uses bounded reviewed inputs, immutable image identity where configured, server-side validation/diff, protected production environment, and workload rollback verification according to its runbook. + +## 10. Rollback semantics + +### Application rollback + +Restores a prior compatible application workload revision/image and verifies readiness. + +### First-time workload rollback + +May require deletion of the newly created workload followed by absence verification rather than “roll back” to a nonexistent prior revision. + +### Database rollback + +Not automatically implied by application rollback. Completed migrations may remain and require a compatible prior application or explicit reverse/forward-fix migration. + +### External side effects + +Calendar messages/notifications/provider writes are not automatically reversible. Provider operations must define compensation or explicitly state that reversal is not supported. + +### Data erasure + +Successful erasure cannot be “rolled back” by restoring user data casually. Backup retention/legal recovery requirements must be reconciled with deletion policy and user/operator expectations. + +## 11. Release candidate verification order + +1. freeze/select exact candidate commit; +2. verify dependency lock/artifact source; +3. run deterministic unit/integration/E2E/coverage; +4. run security/SAST/dependency/supply-chain gates; +5. run migration compatibility/recovery evidence; +6. build distributable artifacts/images; +7. smoke test installed/built artifacts outside source assumptions; +8. verify Compose/Kubernetes reference as applicable; +9. verify backup/restore and operator runbooks affected by the release; +10. run bounded live-provider conformance where required as release evidence, without making provider availability a fabricated deterministic pass; +11. verify SBOM/provenance/signing; +12. verify changelog/release notes/version; +13. create tag/release from the unchanged accepted commit; +14. verify published artifact digest/provenance and rollback path. + +If the commit changes, exact-head evidence is reacquired. + +## 12. Release failure / abort + +Abort or defer the release when: + +- candidate head moves unexpectedly; +- required check/review is absent/stale/failing; +- migration cannot be rehearsed or has an unresolved destructive ambiguity; +- backup/restore/provenance artifact cannot be verified; +- critical security finding remains; +- package/container smoke test differs from source-tree test result; +- operator docs would require undocumented/manual knowledge; +- release notes claim behavior not present in the artifact. + +A release abort does not prevent continued branch/product work. + +## 13. Emergency security release + +An emergency release may reduce nonessential scope but does not permit fabricated checks, unreviewed secret leakage, stale-head publication, or an unsafe migration. Security fixes still preserve exact source/artifact identity, minimal review, regression evidence, provenance and operator remediation guidance. + +## 14. Changelog contract + +`CHANGELOG.md` records externally meaningful additions/fixes/security changes under `Unreleased` until release. It must not become an internal commit log or claim a release date/version not represented by an immutable tag/artifact. + +## 15. Documentation update rule + +A release/migration change updates all affected canonical layers: + +- PRD for customer outcome/scope; +- TRD for shared runtime contract; +- Architecture/ADR for authority/ownership decisions; +- API/data/UML for contract/persistence flow; +- threat/privacy for security/data lifecycle; +- test/operability for acceptance/recovery; +- traceability and changelog for evidence/status. + +Documentation-only completion does not replace executable migration/release evidence. diff --git a/docs/STANDARDS_TRACEABILITY.md b/docs/STANDARDS_TRACEABILITY.md new file mode 100644 index 00000000..2396a80e --- /dev/null +++ b/docs/STANDARDS_TRACEABILITY.md @@ -0,0 +1,107 @@ +# LifeOS Standards and Research Traceability + +**Baseline:** protected `main` at `5c87a7ec3568a4ce47b25cad843f1bc5be91b294` + +## 1. Purpose + +LifeOS has feature-level standards/research documents under `docs/research/` and approved specifications. This index prevents those references from becoming disconnected citations by mapping **why a standard/research family matters, which product boundary uses it, and where implementation evidence lives**. + +This file does not copy full bibliographies from scoped research documents. The scoped document remains authoritative for exact APA 7 citation, publication status, version/date and evidence limitations. + +## 2. Evidence classes + +Use these labels when adding a source: + +- **Normative standard/specification** — protocol/security/accessibility semantics that implementation claims to follow. +- **Authoritative guidance** — operational/security practice from a primary maintainer/body. +- **Peer-reviewed research** — evidence informing product/statistical/model design. +- **Preprint/technical report** — useful but not equivalent to peer-reviewed/final standard evidence. +- **Repository experiment** — LifeOS fixture/ablation/benchmark evidence; product-specific and reproducible. + +Do not present a preprint or internal benchmark as a normative standard. + +## 3. Standards / research map + +| Area | Source family | Product decision / use | Repository evidence | +| --- | --- | --- | --- | +| OAuth/OIDC | OAuth/OIDC/provider specifications and provider docs | bounded Google/GitHub login, callback/state, provider identity mapping | identity-service source/tests and feature plans | +| HTTP conditional/replay semantics | HTTP conditional request semantics | calendar ETag conflict safety, stale write classification | calendar provider adapters/tests; issue #51 completion evidence | +| iCalendar/CalDAV | iCalendar + CalDAV standards | deterministic VEVENT/resource identity and non-destructive sync | calendar integration source/tests | +| Accessibility | WCAG/WAI platform semantics | keyboard/focus/non-color-only/localized core journeys | web accessibility E2E and scoped plans | +| Time zones | IANA zone semantics/runtime libraries | reminder quiet hours, DST-safe local-calendar behavior | notification scheduler tests/runbook | +| CloudEvents/event envelope concepts | versioned event-envelope practices | bounded immutable domain-event identity/correlation/causation | `packages/contracts/`, plugin SDK/tests | +| Kubernetes/container hardening | Kubernetes/runtime/security guidance | restricted non-root/read-only reference workloads, probes, network/disruption/topology boundaries | `infra/kubernetes/`, infra tests, production deployment runbook | +| Software supply chain | SBOM/provenance/action/dependency pinning guidance | exact source/artifact identity, pinned external actions, release evidence | workflows/security/commercial-readiness tests | +| OWASP application security | application/security threat classes | tenant/auth/SSRF/secret/subprocess/prompt-injection regressions | AppGuardrail/Semgrep/GHAS, threat model, scoped security docs | +| NIST AI risk guidance | AI governance/evaluation | deterministic validation, provenance, provider/live-evidence separation | AI quality/live-conformance specs/tests | +| Fugu / Conductor / TRINITY / strong-single-agent research | agent compute-allocation evidence | strong single-route baseline; deeper orchestration only when measured benefit justifies it | AI live-conformance design/research docs and tests | +| PostgreSQL semantics | PostgreSQL documentation/behavior | transactions, unique/idempotency constraints, migrations, dump/restore | service integration tests, backup/recovery runbook | +| Cryptographic primitives | platform cryptography/API guidance | SHA-256 digests, HMAC context, key rotation, encrypted secret boundaries | identity/AI/privacy/data-rights tests and runbooks | + +## 4. Current scoped research documents + +Representative current evidence includes: + +- `docs/research/2026-08-04-ai-gateway-key-rotation-standards.md` +- `docs/research/2026-08-05-contextual-orchestrator-proposal-transport-standards.md` +- `docs/research/2026-08-05-ai-proposal-quality-evaluation-standards.md` +- `docs/superpowers/specs/2026-08-06-ai-nim-live-conformance-design.md` +- privacy/security/operations feature specifications that contain exact standards references. + +When a scoped research document is superseded, update this index and the owning ADR/spec rather than deleting historical rationale silently. + +## 5. AI research traceability rule + +For model routing/orchestration claims: + +1. record source publication status; +2. state the claim actually supported by the source; +3. define the LifeOS fixture/metric that tests whether the claim transfers to this product; +4. retain a strong single-route baseline; +5. measure quality/safety, not only latency/token use; +6. do not turn a paper's agent count/topology into a repository requirement without measured product evidence; +7. keep provider/model availability separate from deterministic policy validity. + +## 6. Security standards traceability rule + +A security citation is not a mitigation. Every cited security requirement maps to: + +- a trust boundary/threat in `docs/THREAT_MODEL.md`; +- a deterministic validator/policy where applicable; +- a negative exploit/regression test; +- least-privilege/runtime/deployment configuration where applicable; +- exact-head scanner/review evidence. + +## 7. Protocol standards traceability rule + +For OAuth/calendar/events/API protocols: + +- record the protocol/version/date in the scoped design or contract when semantics depend on it; +- add real serialization/parsing/conditional-request tests; +- distinguish provider extension from protocol standard; +- preserve unknown-version fail-closed behavior; +- record compatibility/migration when the protocol contract changes. + +## 8. Accessibility traceability rule + +Accessibility requirements must map to actual interaction behavior and tests. A design-system token or ARIA attribute by itself does not prove a user can complete the journey. Core journeys combine semantic controls, keyboard flow, visible focus, status announcements, localization, reduced-motion behavior and responsive layout evidence. + +## 9. Research and standards update process + +When adding or changing a material evidence source: + +1. prefer primary normative or peer-reviewed sources; +2. verify the current version/status when the decision is time-sensitive; +3. record APA 7 citation and stable link in the scoped research/spec document; +4. state whether it is final standard, peer-reviewed article, preprint/technical report or internal experiment; +5. map it to a product/technical/ADR decision and test oracle; +6. update this index if the repository-wide boundary changes; +7. do not cite a source merely to decorate an already-decided implementation. + +## 10. Mathematical and psychometric future modules + +No production psychometric compute module is currently claimed. If added, research traceability must additionally specify estimand/model equations, data-generating assumptions, true-parameter recovery design, multilevel/multiple-membership/temporal requirements, numerical precision/convergence, CPU/GPU parity and publication status for methodological sources. + +## 11. Release criterion + +A stable release does not require every research question to be closed. It does require every **material product claim based on standards/research** to have a clear source class, scoped reference, implementation/test mapping and no known contradiction presented as established fact. diff --git a/docs/TEST_STRATEGY.md b/docs/TEST_STRATEGY.md new file mode 100644 index 00000000..1e14b4d5 --- /dev/null +++ b/docs/TEST_STRATEGY.md @@ -0,0 +1,251 @@ +# LifeOS Test Strategy + +**Baseline:** protected `main` at `5c87a7ec3568a4ce47b25cad843f1bc5be91b294` + +## 1. Objective + +LifeOS tests prove customer-visible behavior, authority boundaries, durability, failure/recovery and release evidence. A test suite is not considered strong merely because it invokes code or achieves a numerical coverage threshold. + +## 2. Test pyramid by boundary + +### Unit/domain tests + +Use for pure invariants and deterministic transformations: + +- goal/project/task/habit state rules; +- recurrence/local-time policy; +- UUID/signature/digest/schema validation; +- provider/model response parsing; +- proposal quality scoring; +- privacy purpose/grant validation; +- projection/rendering/serialization contracts. + +Tests assert outcomes and failures, not implementation call counts alone. + +### Repository/PostgreSQL integration tests + +Required when persistence semantics are material: + +- tenant-scoped reads/writes; +- transaction rollback; +- uniqueness/idempotency; +- concurrent winners; +- immutable/append-only evidence; +- claim expiry/recovery; +- exact expiry boundaries; +- restart durability; +- migration constraints and unsafe mutation rejection. + +Use disposable databases/isolated schemas and exercise real SQL. + +### HTTP/service integration tests + +Required for: + +- OAuth/session callback/error handling; +- session-derived tenant/actor scope; +- signed private service context; +- bounded request/response parsing; +- problem/error mapping; +- stale conflict/idempotency behavior; +- provider failure sanitization; +- service bootstrap/shutdown. + +### Browser/E2E tests + +Cover primary journeys with the real web boundary: + +- onboarding/login surface where practical with bounded provider stubs; +- Today capture, priorities and completion; +- planning search and stale-request protection; +- Korean/English localization; +- keyboard/focus/accessibility behavior; +- responsive/mobile/PWA installation/offline-draft state; +- proposal view/decision UX where exposed. + +### Provider/contract tests + +- Google/GitHub OAuth response contracts with safe fixtures; +- Google Calendar/CalDAV identifiers/ETags/preconditions; +- plugin manifest/event schemas; +- contextual-orchestrator/OpenAI-compatible response bounds; +- NATS/event schema and idempotent consumer behavior. + +Live external providers are never the only deterministic merge oracle. + +## 3. Required negative testing + +Every ownership/security-sensitive feature includes realistic negative cases for applicable classes: + +- other-workspace identifier; +- malformed/non-v4/sequential internal ID; +- missing/expired/replayed session or signed context; +- wrong HTTP method/path signature; +- duplicate idempotency key with same/different payload; +- stale revision/digest/ETag; +- concurrent workers/writers; +- oversized body/collection/text; +- malformed Unicode/JSON/provider row; +- provider redirect/origin confusion; +- secret-shaped content/logging; +- prompt injection/untrusted-model text; +- expired/reused privacy grant; +- backup checksum corruption/non-empty restore target; +- deployment rollback failure; +- stale browser response after query/navigation/unmount. + +## 4. Coverage policy + +Packages that declare exact coverage gates maintain **100% statement, branch, function and line coverage for owned production code where technically meaningful**. + +Coverage is not accepted when achieved by: + +- empty assertions; +- source rewriting in CI; +- excluding reachable production branches without rationale; +- replacing a failing behavioral test with a looser check; +- invoking code without validating the result; +- treating skipped/ignored required tests as pass. + +A coverage gap should usually be resolved by clarifying behavior and adding a realistic assertion; unreachable code should be removed or justified rather than hidden. + +## 5. Deterministic versus live AI evidence + +### Deterministic merge gates + +Must run without requiring a live model provider: + +- proposal schema/operation validation; +- ownership/signed-context validation; +- prompt-injection fixtures; +- forbidden/leakage text detection; +- proposal persistence/decision semantics; +- provider response bounding/error classification; +- orchestration policy configuration and artifact sanitation. + +### Bounded live conformance + +May use `NVIDIA_NIM_API_KEY` through the approved model boundary to measure: + +- real proposal validity/grounding/utility; +- prompt-injection resistance; +- strong-single-route versus bounded orchestration behavior; +- provider/model provenance and bounded usage metrics. + +Provider unavailable/rate-limit/malformed response is recorded as unavailable/failure evidence. It never fabricates a score and does not silently disable deterministic gates. + +## 6. Security test gates + +Applicable exact-head PRs must pass configured repository gates such as: + +- AppGuardrail; +- Semgrep/SAST; +- GitHub Advanced Security/code scanning; +- secret/dependency/supply-chain checks; +- repository-specific commercial-readiness/security contracts. + +A status/check is valid only for the exact current head and correct checkout semantics. Queued, absent, cancelled, skipped-required, action-required, stale-head or predecessor-head evidence is not promoted to success. + +## 7. Accessibility and localization testing + +Core journeys test: + +- keyboard-only operation; +- visible focus and semantic controls; +- accessible names/live regions; +- reduced motion where relevant; +- no color-only status; +- Korean and English catalog key parity; +- interpolation/fallback behavior; +- narrow/mobile viewport interactions; +- print/export exact values where a reporting feature requires them. + +Automated checks supplement rather than replace manual assistive-technology review for release-critical interaction changes. + +## 8. Time and concurrency testing + +Use deterministic clocks/fixtures for: + +- IANA timezone offsets and DST transitions; +- quiet hours crossing local-day boundaries; +- claim/grant/session expiry at the exact boundary; +- stale request ordering; +- duplicate/replayed commands; +- concurrent PostgreSQL transactions; +- optimistic-concurrency conflicts. + +Do not use arbitrary sleeps where a deterministic barrier/clock/transaction control can prove the behavior. + +## 9. Backup, migration and deployment testing + +### Backup/restore + +Exercise real PostgreSQL tooling and verify: + +- archive/checksum creation; +- exact expected tenant records after restore; +- checksum-corruption refusal; +- non-empty-target refusal; +- bounded credential-free failure evidence. + +### Migrations + +For schema changes verify as applicable: + +- migration applies on supported prior state; +- existing valid records remain valid; +- new constraints reject invalid state; +- rollback or forward-fix strategy is explicit; +- application compatibility during rollout is understood. + +### Deployment reference + +Verify rendered Kubernetes/Compose artifacts, immutable image/source requirements, probe/security/network-policy contracts, dry-run behavior and claimed workload rollback semantics without claiming infrastructure provisioning not supplied by the repository. + +## 10. Documentation consistency tests + +The canonical documentation graph should receive machine-checkable regression coverage for: + +- required canonical file presence; +- README/documentation index links; +- ADR index entries and allowed statuses; +- balanced Markdown code fences and parseable Mermaid blocks where tooling permits; +- current service/bounded-context names; +- UUIDv4 and service-data-ownership invariants; +- no canonical statement that UUIDv7 or browser-only local storage is current architecture; +- `Implemented on active PR` not being presented as protected-main behavior; +- traceability entries pointing to existing paths or explicit open gaps. + +These tests should compare against repository source/config/migrations rather than checking prose existence only. + +## 11. Test-first defect workflow + +For a valid product defect: + +1. reproduce/locate the first failing boundary; +2. write the smallest realistic regression that fails for the defect; +3. run it and observe RED for the intended reason; +4. implement the narrowest causal fix; +5. run focused GREEN; +6. run affected package and repository gates; +7. refetch exact PR head and external checks; +8. resolve only review threads whose underlying cause is fixed. + +For pure documentation drift where no runtime behavior changes, the RED oracle is a documentation consistency check or direct exact-source comparison; do not invent product code merely to satisfy TDD ceremony. + +## 12. Release test evidence + +A stable release requires one exact protected integrated head with applicable: + +- repository CI; +- security/SAST/dependency gates; +- exact owned-code coverage; +- browser/accessibility/localization journeys; +- PostgreSQL migration/integration evidence; +- backup/restore evidence; +- Compose/Kubernetes reference validation; +- package/container build and smoke tests; +- SBOM/provenance/reproducibility evidence under release policy; +- no valid unresolved review/security finding. + +Tests from an older head/base, synthetic merge only, or a temporary repair branch are not a substitute for final-head evidence. diff --git a/docs/THREAT_MODEL.md b/docs/THREAT_MODEL.md new file mode 100644 index 00000000..30899ffe --- /dev/null +++ b/docs/THREAT_MODEL.md @@ -0,0 +1,284 @@ +# LifeOS Threat Model + +**Baseline:** protected `main` at `876850018a17323900844e79845ba395b7bf6a9a` +**Scope:** upstream LifeOS software and documented reference deployment boundaries. Independent operators must extend this model for their infrastructure, providers, geography and legal obligations. + +`SECURITY.md` defines vulnerability reporting and safe harbor. This file defines architecture threats and controls. + +## 1. Security objectives + +1. A user cannot read or mutate another workspace's data without explicit authorized membership/purpose. +2. Provider/browser/model credentials never cross into unrelated service, model, log, artifact or public-error boundaries. +3. Untrusted model/provider/plugin/calendar/browser input cannot become authority merely by being parsed. +4. Replay, stale state and concurrent workers do not create duplicate or unauthorized side effects. +5. AI proposals remain inert and auditable. +6. Privileged sensitive-data access is purpose/resource/actor bounded and auditable. +7. Backups, migrations, deployments and automation cannot silently bypass protected source/review/tenant boundaries. +8. A compromised or unavailable optional provider degrades only the capability that needs it where practical. +9. A compromised service database credential cannot legitimately traverse into another service's tables merely because multiple schemas share one physical PostgreSQL cluster. + +## 2. Assets + +- user identity and workspace membership; +- goals, projects, tasks, habits, reviews and personal text; +- browser sessions and OAuth/provider credentials; +- calendar integration credentials/resource identifiers; +- notification delivery state/idempotency data; +- AI proposal/evidence/decision history; +- privacy access decisions/grants/events; +- per-service database credentials and encryption/signing keys; +- source code, CI credentials, review evidence, releases and provenance; +- backups and deployment artifacts; +- correlation/operational metadata that may become sensitive when joined. + +## 3. Trust boundaries + +```mermaid +flowchart TB + Browser[Untrusted/partially trusted browser input] + Web[Web / BFF boundary] + Identity[Identity service] + Planning[Planning service] + Habit[Habit service] + Review[Review service] + Notification[Notification service] + AI[AI proposal service] + Privacy[Privacy access service] + Integrations[Calendar / Plugin services] + + IdentityDB[(Identity-owned DB schema / role)] + PlanningDB[(Planning-owned DB schema / role)] + HabitDB[(Habit-owned DB schema / role)] + ReviewDB[(Review-owned DB schema / role)] + NotificationDB[(Notification-owned DB schema / role)] + AIDB[(AI-owned DB schema / role)] + PrivacyDB[(Privacy-owned DB schema / role)] + + External[Identity / Calendar / Model Providers] + Model[Untrusted model output] + Plugin[Untrusted plugin manifests/events] + CI[GitHub CI / autonomous agents] + Artifacts[Logs / metrics / artifacts / backups] + + Browser --> Web + Web --> Identity + Web --> Planning + Web --> Habit + Web --> Review + Web --> AI + Web --> Privacy + Web --> Integrations + + Identity --> IdentityDB + Planning --> PlanningDB + Habit --> HabitDB + Review --> ReviewDB + Notification --> NotificationDB + AI --> AIDB + Privacy --> PrivacyDB + + Identity --> External + Integrations --> External + AI --> External + External --> Identity + External --> Integrations + External --> AI + AI --> Model + Model --> AI + Plugin --> Integrations + CI --> Identity + CI --> Planning + CI --> Habit + CI --> Review + CI --> Notification + CI --> AI + CI --> Privacy + CI --> Integrations + + IdentityDB --> Artifacts + PlanningDB --> Artifacts + HabitDB --> Artifacts + ReviewDB --> Artifacts + NotificationDB --> Artifacts + AIDB --> Artifacts + PrivacyDB --> Artifacts + Identity --> Artifacts + Planning --> Artifacts + Habit --> Artifacts + Review --> Artifacts + Notification --> Artifacts + AI --> Artifacts + Privacy --> Artifacts + Integrations --> Artifacts +``` + +Each database node is an independent service-owned persistence boundary even when an operator co-locates schemas on one PostgreSQL cluster. A service receives only its own DSN/role/schema privileges. Cross-service data movement must cross a reviewed HTTP/event/saga/plugin/MCP boundary; it cannot shortcut through the shared physical cluster. + +No arrow implies unconditional trust. Each boundary revalidates the fields it depends on. + +## 4. Threat catalogue + +### TM-001: OAuth callback/login substitution + +**Threat:** attacker forges/replays state, substitutes provider identity, abuses redirect handling or links an identity to the wrong LifeOS user. + +**Controls:** fixed provider/redirect contracts, bounded callback parsing, state/session verification, explicit external-identity mapping, provider IDs not used as internal authorization IDs, secure session cookies/revocation, provider-specific regression tests. + +**Residual risk:** provider compromise/misconfiguration is operator/external-provider dependent. + +### TM-002: Tenant/workspace scope injection + +**Threat:** client submits another `workspace_id`/actor header or manipulates a path to access cross-tenant data. + +**Controls:** derive authority from session/signed private context, tenant-scoped SQL, UUID validation, cross-tenant negative tests, no cross-service direct DB access. + +**Residual risk:** a missing scope predicate in new repository code remains a critical review target. + +### TM-003: Session replay/fixation/theft + +**Threat:** stolen/replayed browser session accesses personal data. + +**Controls:** secure HttpOnly/SameSite cookie/session handling, rotation/revocation/lifetime checks, no browser credential forwarding to domain/model services, bounded public errors. + +### TM-004: Signed service-context replay/confusion + +**Threat:** signed context for one actor/method/path is replayed against another operation. + +**Controls:** bind workspace, actor, exact method/path, key identifier and issuance/lifetime; bounded active/previous key overlap; reject retired/unknown key IDs; tests for method/path replay. + +### TM-005: SQL injection / unsafe persistence decoding + +**Threat:** external text or identifiers alter SQL structure or malformed database rows become trusted objects. + +**Controls:** static/parameterized SQL, bounded validators at persistence adapters, strict UUID/timestamp/enumeration parsing, credential-free failures. + +### TM-006: Cross-service database privilege confusion + +**Threat:** one service is given a cluster-wide database role or uses a shared DSN and reads/mutates another bounded context's tables, bypassing API/event authorization, audit and ownership logic. + +**Controls:** per-service database role/DSN/schema ownership, no cross-service table access in architecture/tests, service-owned migrations/repositories, explicit API/event/saga/plugin/MCP boundaries, least-privilege deployment secrets, logical ERD diagrams that never imply shared-table authority. + +**Residual risk:** operator-side PostgreSQL role misconfiguration can violate the intended boundary and must be tested/monitored by the deployment owner. + +### TM-007: Stale/lost update + +**Threat:** multi-device or asynchronous stale state overwrites newer durable state. + +**Controls:** ETag/revision/digest/idempotency semantics where implemented, stale async response ownership checks, explicit local-draft/durable distinction. Complete durable Today aggregate conflict resolution remains issue #121. + +### TM-008: Duplicate side effects / replay + +**Threat:** retries or concurrent workers create duplicate habit completion, notification delivery, calendar events or AI decisions. + +**Controls:** idempotency keys/digests, database uniqueness/transaction serialization, expiring claims/fencing, deterministic calendar identifiers/preconditions, proposal revision/digest-bound decisions. + +### TM-009: Calendar/provider SSRF or credential leakage + +**Threat:** malicious provider URL/redirect/body sends tokens to untrusted hosts or causes unbounded fetch/response buffering. + +**Controls:** fixed/validated provider origins, bounded redirects/response sizes/timeouts, provider-specific credential scoping, sanitized failures, deterministic resource/ETag handling. Hosted credential lifecycle issue #129 and plugin delivery issue #130 require their own strengthened credential/SSRF evidence. + +### TM-010: Model prompt injection / data exfiltration + +**Threat:** untrusted planning context instructs a model to leak secrets, mutate state or fabricate authority. + +**Controls:** model receives bounded context and no browser/GitHub/review-agent credentials, output is structured untrusted data, deterministic validation, proposal is inert, retained artifacts exclude raw prompt/response/hidden reasoning/credential material, prompt-injection fixtures. + +### TM-011: Model/provider unavailability or fabricated evaluation + +**Threat:** CI/product claims success when live provider is missing, rate-limited or returns malformed output. + +**Controls:** deterministic validators/gates are authoritative; live conformance returns sanitized unavailable/failure evidence; null metrics are not fabricated; exact provider result shape is bounded. + +### TM-012: Plugin manifest/event injection + +**Threat:** plugin manifest/event creates arbitrary commands, cross-tenant access, secrets or network activity. + +**Controls:** versioned SDK/schema validation, bounded tenant-scoped event preparation, no direct DB access, no current generic installation/secret/outbound-delivery authority. Issue #130 tracks signed grants, encrypted secrets, SSRF control, retry/audit and revocation for the runtime last mile. + +### TM-013: Sensitive-data overexposure + +**Threat:** broad operator/service access exposes personal goals, health, relationship or AI context beyond purpose. + +**Controls:** purpose-bound privacy decisions/grants/events, least privilege, tenant scope, encryption/secret manager boundary, content-minimized logs/metrics/errors, time-bounded/single-use grants where required. + +### TM-014: Privacy grant replay/expiry race + +**Threat:** a consumed/expired/wrong-purpose grant is reused. + +**Controls:** signed grant context, exact expiry semantics, atomic consumption, immutable decision/event evidence, concurrent winner tests, rollback when consumption cannot persist. + +### TM-015: Log/metric/artifact data leakage + +**Threat:** raw personal text, credentials, stack traces, prompts/model output or provider bodies enter retained public artifacts. + +**Controls:** bounded structured errors, sanitized CI/model evidence, metrics dimension restrictions, no raw prompt/response retention in live conformance, synthetic fixtures in public repository. + +### TM-016: Malicious or corrupted backup/restore + +**Threat:** corrupted/tampered archive, restore into wrong/non-empty target, hidden destructive overwrite. + +**Controls:** checksum verification, selected custom-format archive, empty-target refusal, real restore integration tests, private metadata/evidence, explicit statement that logical dumps are not PITR. + +### TM-017: Deployment privilege or rollback failure + +**Threat:** deployment applies unreviewed mutable image/config or rollback claim does not restore workload state. + +**Controls:** digest-pinned images/reference renderer, protected production environment, server-side dry run/diff, capture existing deployment/revision, verify rollback/deletion result, least-privilege workload/network defaults and per-service database credentials. + +### TM-018: CI/autonomous-agent supply-chain compromise + +**Threat:** mutable action/dependency, broad token, encoded patch or model output writes unsafe code/workflow, or automation bypasses review. + +**Controls:** exact-head/base/blob guards, immutable action/dependency pins where required, least privilege, separate model/review credentials, AppGuardrail/Semgrep/GHAS/review gates, no fabricated approval, temporary repair scaffolding removed when obsolete, bounded autonomous writer lease. + +### TM-019: Concurrency between repository writers + +**Threat:** two agents overwrite/race the same PR branch based on stale evidence. + +**Controls:** writer lease, pre-write refetch, freeze only conflicting target, no force-push/admin bypass, exact-head verification after writes. + +## 5. Privacy misuse cases + +- An operator searches personal text without a valid support/security/legal purpose. +- An AI feature receives more user history than needed for the selected proposal. +- Calendar/plugin provider tokens are reused for unrelated providers/tasks. +- An exported artifact contains internal audit/credential material. +- A support/debug log retains personal action text indefinitely. + +Mitigations must combine purpose, resource/tenant scope, bounded retention, encryption, explicit export contracts and immutable access evidence. Redaction alone is insufficient. + +## 6. Availability and resource exhaustion + +Threats include oversized HTTP/model/provider payloads, slow subprocess/network calls, worker claim leakage, high-cardinality telemetry, repeated invalid OAuth/provider requests and expensive live-model orchestration. + +Controls include size/count/time bounds, abort/timeout semantics, bounded worker concurrency, fatigue/rate policies, no unbounded telemetry labels and provider-unavailable degradation. + +## 7. Security acceptance evidence + +A security-sensitive change is not complete until applicable evidence exists: + +- realistic exploit/regression test; +- tenant-negative test; +- malformed/oversized input test; +- replay/concurrency test; +- exact permission/credential boundary verification; +- AppGuardrail/Semgrep/GHAS/current-head security checks; +- no unresolved valid security review thread; +- migration/rollback/incident evidence for persistence/deployment changes. + +## 8. Out-of-scope / operator extension + +This upstream threat model does not prove security of an independent operator's: + +- cloud account/IAM; +- cluster/control plane; +- DNS/TLS/ingress/WAF; +- managed database/NATS; +- secret manager/KMS; +- employee/admin devices; +- identity/calendar/model provider tenant configuration; +- data-retention/legal process; +- SOC 2/CSAP/ISO certification. + +Operators must extend and validate these boundaries for their deployment. diff --git a/docs/TRACEABILITY.md b/docs/TRACEABILITY.md new file mode 100644 index 00000000..0df224cc --- /dev/null +++ b/docs/TRACEABILITY.md @@ -0,0 +1,129 @@ +# LifeOS Requirements and Evidence Traceability + +**Baseline:** protected `main` at `be5e6d91dca81e76223a2846e74ce1de8e22a2f2` + +## 1. Purpose + +This file maps canonical requirements and architecture decisions to representative protected-main code/tests/runbooks/capability evidence and current active implementation PRs. It is an index, not an exhaustive substitute for source control, migrations, exact-head checks, or `product/capabilities.json`. + +An entry marked `Implemented on active PR` is not protected-main evidence. PR identity is recorded separately from status so a closed, replaced, rebased, or merged PR cannot silently preserve stale maturity. + +## 2. Product requirement traceability + +| PRD requirement | Status | Capability / authority | Representative implementation | Representative verification / operations | +| --- | --- | --- | --- | --- | +| PRD-ID-001 Google/GitHub login | Implemented on protected main | `identity.oauth-session` | `apps/identity-service/src/` | `oauth-http.integration.test.ts` | +| PRD-ID-002 revocable scoped session | Implemented on protected main | identity service | identity runtime/session repositories | identity/security tests | +| PRD-ID-003 UUIDv4 internal IDs | Implemented on protected main | ADR-0002 / `AGENTS.md` | service migrations/domain validators | UUID constraint/validation tests | +| PRD-ID-004 tenant-derived authority | Implemented on protected main | identity + owning service | signed/authenticated context boundaries | cross-tenant integration tests | +| PRD-PLAN-001 durable goals/projects/tasks | Implemented on protected main | `planning.durable-data` | `apps/planning-service/src/postgres-planning-repository.ts` | `postgres-planning-repository.integration.test.ts` | +| PRD-PLAN-002 durable planning search | Implemented on protected main | `capture.search` | `apps/planning-service/src/search.ts` | planning/web search tests | +| PRD-PLAN-003 quick capture/local draft distinction | Implemented on protected main | `capture.search` / Today UX | `apps/web/app/components/quick-capture.tsx` | `quick-capture-search.spec.ts` | +| PRD-PLAN-004 Today action loop | Implemented on protected main | `today.action-loop` | web Today components/gateway composition | `apps/web/e2e/today-flow.spec.ts` | +| PRD-PLAN-005 full durable Today optimistic multi-device sync | Implemented on active PR | issue #121 / PR #127 | versioned Today aggregate, strong revision preconditions, idempotency, planning-owned PostgreSQL persistence and explicit local-to-durable UX on PR #127 | exact #127 head must pass planning/PostgreSQL/browser/conflict/offline acceptance before protected-main upgrade | +| PRD-HAB-001 recurring habit + history | Implemented on protected main | `habit.recurring-core` | `apps/habit-service/` | habit service/PostgreSQL integration tests | +| PRD-REV-001 guided review | Implemented on protected main | `review.guided-loop` | `apps/review-service/` | review service integration tests | +| PRD-CAL-001 calendar synchronization | Implemented on protected main | `calendar.time-blocking`; completed issue #51 | `apps/integration-calendar-service/` | calendar sync integration tests | +| PRD-CAL-003 hosted per-user Google token lifecycle | Partial | issue #129 | current adapter uses operator-supplied runtime token in documented slice | encrypted per-user storage/refresh/revocation/provider-selection gap is explicit | +| PRD-NOT-001 bounded reminders | Implemented on protected main | `notifications.reminders` | `apps/notification-service/src/reminder-scheduler.ts` | scheduler/integration tests; `docs/operations/notification-persistence.md` | +| PRD-NOT-002 durable claim/retry/idempotency | Implemented on protected main | notification persistence | `postgres-reminder-repository.ts` + runtime | PostgreSQL/concurrency integration tests | +| PRD-AI-001 inert proposals | Implemented on protected main | ADR-0004 / AI service | proposal service/audit domain | no-silent-mutation + proposal tests | +| PRD-AI-002 proposal evidence + explicit decisions | Implemented on protected main | AI audit capability | AI PostgreSQL proposal audit repository/application | proposal audit HTTP/PostgreSQL tests | +| PRD-AI-003 authenticated same-origin AI boundary | Implemented on protected main | AI gateway trust boundary | web AI proposal BFF/routes + signed context | scope/identity-stream/routes integration tests | +| PRD-AI-004 deterministic quality independent of live provider | Implemented on protected main | quality/live-conformance split | proposal quality evaluator + optional live model adapter | quality fixture tests + scheduled bounded live evidence | +| PRD-AI-005 bounded deeper orchestration | Accepted architecture | strong single-route baseline + explicit orchestration evidence | protected-main quality/conformance harness and #133 model/Compose hardening | explicit NVIDIA model catalog and credential-free digest-pinned Compose runtime verification are now protected-main evidence | +| PRD-AI-006 autonomous OpenCode development loop | Implemented on protected main | issue #120 / merged PRs #122 and #133 | `.github/workflows/opencode-commercial-development.yml`, `packages/commercial-development-agent/` | exact-head CI/security/review gates preceded both protected merges; model account has no Docker authority | +| PRD-PRIV-001 purpose-bound sensitive access | Implemented on protected main | ADR-0005 | `apps/privacy-service/` | privacy service PostgreSQL/security tests | +| PRD-PRIV-002 bounded grants + append-only evidence | Implemented on protected main | privacy access domain | privacy decision/grant/event persistence | exact expiry/concurrency/immutability tests | +| PRD-PRIV-004 end-user export/deletion UX | Partial | issue #55 / active PR #138 | protected main preserves authentication age and recent-auth ownership prerequisites from #134-#137; PR #138 implements a durable request/receipt ledger | concrete domain contributors, authenticated orchestration wiring, recovery events, retention/legal-hold, protected export delivery/download audit and destructive replay drills remain required | +| PRD-INT-001 versioned plugin contract | Implemented on protected main | plugin integration | `packages/plugin-sdk/`, `apps/integration-service/` | plugin contract integration tests | +| PRD-INT-003 plugin installation/secrets/outbound delivery | Planned | issue #130 | validation/preparation surface exists; runtime authority is intentionally absent | installation grants, encrypted secrets, SSRF-safe delivery, retry/audit/revocation tests required | +| PRD-WEB-001 installable responsive PWA | Implemented on protected main | `mobile.pwa` | manifest/service worker/web app | `mobile-pwa.spec.ts` | +| PRD-WEB-002 accessibility | Implemented on protected main | `accessibility.localization` | semantic web components/design tokens | accessibility E2E/current core-flow tests | +| PRD-WEB-003 Korean/English localization | Implemented on protected main | `accessibility.localization` | `apps/web/messages/en.json`, `ko.json` | localization/catalog tests | +| PRD-WEB-004 Today offline/durable reconciliation | Implemented on active PR | issue #121 / PR #127 | explicit local draft migration and conflict/recheck flow for the bounded Today aggregate | broader offline behavior must not be inferred beyond PR #127 acceptance scope | +| PRD-GOV-001 capability maturity separated from buyer-gap exhaustion | Implemented on protected main | completed issue #128 / ADR-0008 / merged PR #131 | `product/buyer-gaps.json` and separate readiness dimensions | malformed/duplicate/unknown/fetch-failure and open/resolved/unknown contract tests integrated with exact-PR-head audit binding | +| PRD-GOV-002 untrusted issue/review/model prose is non-executable governance input | Implemented on protected main | ADR-0008 / security boundary | canonical gap identity is repository-owned and live reconciliation consumes bounded issue state only | raw issue/review/model prose does not define policy | +| PRD-OPS-001 verified logical backup/restore | Implemented on protected main | backup/recovery capability | `infra/backup/` | restore/deployment tests + `backup-and-restore.md` | +| PRD-OPS-002 production reference deployment | Implemented on protected main | deployment capability | `infra/kubernetes/`, deployment workflow | infra tests + `production-deployment.md`; `reference` describes scope only | +| PRD-OPS-003 health/metrics operator surface | Implemented on protected main | observability capability | gateway/planning/service observability | SLO/runbook/observability tests | + +## 3. Architecture decision traceability + +| ADR | Current invariant/evidence | Regression expectation | +| --- | --- | --- | +| ADR-0001 multi-user/self-hostable MSA | README, service layout, Compose/Kubernetes, PostgreSQL | local browser state never masquerades as durable system of record | +| ADR-0002 UUIDv4 | AGENTS/CLAUDE/ARCHITECTURE, validators/migrations | reject non-v4 internal IDs where the domain requires the shared invariant | +| ADR-0003 service-owned persistence | root architecture, independent service repositories/migrations | no cross-service table access; contract/event integration only | +| ADR-0004 inert AI proposal | AI architecture, proposal persistence/decision domain | no planning mutation dependency; stale/replay/prompt-injection tests | +| ADR-0005 purpose-bound sensitive access | privacy-service | actor/resource/purpose/grant expiry/replay/audit tests | +| ADR-0006 work-conserving maintenance | AGENTS waiting/PR loop + protected-main OpenCode automation | exact-head evidence, blocker-local routing, no fabricated approval, no documentation-only stopping | +| ADR-0007 canonical documentation graph | PR #126 until merged | exact status vocabulary, real link targets, ADR stable-filename index targets, source-bound claims, active-PR/protected-main status review | +| ADR-0008 capability maturity vs buyer gaps | Implemented on protected main | 100% configured capability evidence can coexist with open/unknown buyer gaps; issue prose cannot redefine gap policy | + +## 4. Current active implementation ledger + +This ledger is intentionally separate from protected-main evidence and must be live-refetched before use. + +| PR | Scope | Canonical interpretation | +| --- | --- | --- | +| #127 | durable Today synchronization / issue #121 | product requirement implementation is `Implemented on active PR` until exact reviewed head merges | +| #138 | durable data-rights request/receipt ledger / issue #55 | bounded identity persistence implementation is `Implemented on active PR`; the overall export/deletion requirement remains `Partial` | + +PRs #131, #133 and #134 are no longer active implementation items: their accepted work is integrated on protected main. Later #135-#137 also advanced the same recent-auth/data-rights ownership boundary on protected main before #138 began. + +## 5. Canonical evidence hierarchy + +When sources disagree, use this order: + +1. current protected-main source/migrations/tests and configured branch/ruleset behavior; +2. exact current active-PR source explicitly labeled as active-PR evidence; +3. accepted/implemented ADRs and root `ARCHITECTURE.md` for durable decisions, without upgrading active code to protected-main status; +4. canonical PRD/TRD/data model/UML/threat/test/operability docs; +5. repository-owned capability and buyer-gap registries plus `CHANGELOG.md` as governed evidence; +6. current feature specs/runbooks/research documents for bounded detail; +7. historical specs/plans/closed PRs/conversation as rationale, not current implementation truth. + +A resolved review thread does not outrank later exact-source regression evidence. + +## 6. Known gaps exposed by this baseline + +The maintenance loop must refetch issues/PRs and protected main before selecting work. The live product queue demonstrates why configured capability maturity cannot stand in for complete customer journeys. + +### Highest-impact known gaps and active paths + +1. **Issue #121 — durable Today workspace synchronization / optimistic concurrency.** PR #127 is the active implementation path. The requirement is not protected-main evidence until its exact reviewed head is integrated. +2. **Issue #55 — complete tenant export and deletion orchestration.** Recent-auth/authentication-age prerequisites are protected-main behavior. PR #138 actively adds durable request/receipt persistence while concrete domain participation, authenticated orchestration wiring, operational recovery evidence, retention/legal-hold/backup-expiry, encrypted delivery and download-audit follow-ups remain incomplete. +3. **Issue #129 — hosted per-user calendar credential lifecycle.** Conflict-safe CalDAV and Google provider adapters exist; encrypted per-user credential persistence/refresh/revocation/discovery/selection remains incomplete. The original CalDAV issue #51 is closed as completed rather than being kept open to represent a different product gap. +4. **Issue #130 — plugin runtime last mile.** Versioned manifest/event validation exists; installation grants, encrypted secret lifecycle, SSRF-safe outbound delivery, bounded retry/audit and revocation are explicitly tracked rather than implied by the validation-only contract. +5. **Stable release evidence.** Active development remains under `Unreleased`; integrated release gates must pass together before version/tag/release claims. + +### Resolved governance and runtime defects + +- **Issue #128 — readiness accounting defect** is completed through protected PR #131. Configured capability maturity and canonical buyer-gap state are separate dimensions. +- **PR #133 OpenCode/model/Compose hardening** is protected-main behavior, not an active follow-up. Explicit model resolution and digest-pinned PostgreSQL/NATS runtime probes retain least-authority separation. +- **PR #134 authentication-age preservation and subsequent #135-#137 recent-auth work** are protected-main prerequisites for data-rights operations; they do not by themselves complete #55. + +### Documentation engineering status + +**Implemented on active PR:** PR #126 supplies the canonical documentation graph and `packages/commercial-readiness/src/documentation-contract.test.mjs`. The contract fails on non-canonical status values, missing stable ADR filename targets, broken/escaping README links, selected source/configuration evidence drift, and stale active-PR/protected-main traceability. This remains active-PR evidence until PR #126 itself merges. + +## 7. Commercial-readiness interpretation + +Configured capability maturity and canonical buyer-gap state are separate protected-main dimensions under ADR-0008 and merged PR #131. A 100% configured maturity score is still not equivalent to a complete product: registered gaps #55, #121, #129 and #130 remain independently visible until their own product outcomes close. + +If live state for a registered gap is unavailable or ambiguous, the buyer-gap dimension reports `unknown`, not silently resolved. Readiness consumers must combine capability maturity, canonical buyer-gap state, end-to-end PRD journey coverage, operator/release acceptance, and fresh source evidence. + +## 8. Updating traceability + +When a requirement changes status: + +1. verify exact protected-main or active-PR evidence; +2. update PRD status using only the canonical vocabulary; +3. update this row/path/test/issue/PR evidence; +4. update architecture/ADR if authority, ownership or readiness interpretation changed; +5. update data/UML/threat/operability docs if their boundary changed; +6. add/modify regression evidence; +7. update `CHANGELOG.md` for buyer-visible behavior; +8. do not upgrade to `Implemented on protected main` before merge; +9. if an active PR closes/replaces/rebases, immediately revalidate or downgrade its documentation status rather than preserving stale evidence. diff --git a/docs/TRD.md b/docs/TRD.md new file mode 100644 index 00000000..8ef44654 --- /dev/null +++ b/docs/TRD.md @@ -0,0 +1,324 @@ +# LifeOS Technical Requirements Document + +**Version:** 1.0-draft +**Baseline:** protected `main` at `876850018a17323900844e79845ba395b7bf6a9a` + +## 1. Purpose + +This document defines repository-wide technical requirements for LifeOS. It does not replace owning-service source, migrations, versioned API/event schemas, exact active-PR source, or scoped runbooks. It defines the common runtime, authority, data, security, failure, quality, deployment, readiness, and release contracts those artifacts must satisfy. + +## 2. Architecture status + +LifeOS is a TypeScript-first monorepo containing a Next.js web/PWA, gateway/BFF, and independently bounded services. PostgreSQL is the durable system of record for persisted domains. NATS JetStream is available for versioned asynchronous events. Optional external providers include Google/GitHub identity, Google Calendar/CalDAV, and model providers through approved AI boundaries. + +Protected main currently includes: + +- web/PWA and gateway; +- identity/OAuth/session boundaries; +- durable planning/search and Today UX foundations; +- recurring habit persistence/completion; +- guided review; +- calendar provider adapters; +- durable notification scheduling/delivery; +- AI proposal persistence, decisions, evaluation, contextual-orchestrator transport, and bounded NVIDIA NIM conformance; +- plugin contract discovery/validation/preparation; +- purpose-bound privacy access; +- PostgreSQL logical backup/restore; +- provider-neutral Kubernetes reference deployment; +- AppGuardrail/security/commercial-readiness gates; +- the bounded hourly OpenCode commercial-development workflow and deterministic commercial-development package merged from PR #122 as `876850018a17323900844e79845ba395b7bf6a9a`. + +Current active implementation evidence additionally includes: + +- PR #127 for the versioned durable Today aggregate, explicit local-to-durable migration, strong revision preconditions, idempotency, PostgreSQL persistence and conflict/recheck browser flow for issue #121; +- PR #131 for repository-owned buyer-gap identity and independent capability-maturity versus buyer-gap reporting for issue #128; +- PR #133 for explicit NVIDIA model-catalog resolution and real digest-pinned PostgreSQL/NATS Compose verification of the protected-main OpenCode development loop. + +These active PRs are not protected-main evidence until their exact reviewed heads merge. The OpenCode workflow is repository automation, not product-data mutation authority. + +## 3. Technology baseline + +| Concern | Requirement | +| --- | --- | +| Web | Next.js + React + TypeScript | +| Gateway/domain services | Existing NestJS/TypeScript-compatible service patterns | +| Workspace | pnpm + Turborepo | +| Durable database | PostgreSQL | +| Event transport | NATS JetStream when asynchronous durability is required | +| Local composition | Docker Compose | +| Production reference | Kubernetes/Kustomize provider-neutral reference | +| Observability | structured logs, correlation IDs, Prometheus-compatible metrics and scoped SLO/OpenTelemetry contracts where implemented | +| AI/model access | approved OpenCode/contextual-orchestrator boundary using `NVIDIA_NIM_API_KEY` where model access is required | + +New dependencies require demonstrated product or operational value and cannot create a competing authority for an existing bounded context. + +## 4. Bounded contexts and authority + +### Web / PWA + +Owns interaction state and explicitly browser-local drafts/caches. It never becomes a direct database client or durable domain authority. + +### Gateway / BFF + +Owns public composition/authentication routing. It may derive and sign context, but it is not a hidden shared domain database or generic mutation bus. + +### Identity service + +Owns LifeOS user identity, external identity mapping, browser sessions, workspace membership/authorization context, and provider login boundaries. External numeric/provider IDs never become internal primary keys. + +### Planning service + +Owns goals, projects, milestones, tasks, planning search, Today-related durable planning state, and planning mutation rules. Review, AI, notification, and integration services cannot update planning tables directly. + +### Habit service + +Owns habit definitions, recurrence behavior, completion evidence, and habit persistence. + +### Review service + +Owns review snapshots/projections/observations. It consumes evidence without becoming planning/habit source-of-truth authority. + +### Calendar integration service + +Owns provider adaptation and calendar synchronization state. Hosted per-user encrypted Google credential lifecycle/provider selection is `Partial` and tracked by issue #129. + +### Notification service + +Owns reminder occurrences, expiring claims, immutable outcomes, in-app delivery, timezone/fatigue/retry behavior, and worker recovery. + +### AI proposal service + +Owns bounded proposal generation, immutable proposal evidence, explicit accept/reject history, and deterministic proposal-quality evaluation. It has no generic planning mutation repository. + +### Privacy service + +Owns purpose-bound access decisions, grants, events, signed sensitive-access boundaries, and their persistence contract. + +### Plugin integration service / SDK + +Owns plugin contract discovery, manifest/event validation, event preparation, and SDK types. Generic plugin installation, durable secrets, outbound delivery, retry/audit/revocation, and arbitrary commands are `Planned` under issue #130 rather than implied by the validation-only surface. + +## 5. Data and identity requirements + +### TRD-DATA-001 — Service-owned persistence + +Each bounded service owns its database schema/migrations and credentials. Direct cross-service table reads/writes are prohibited. Shared IDs may form logical relationships but do not create hidden cross-service foreign-key authority. + +### TRD-DATA-002 — Internal identifiers + +Internal durable identifiers are opaque UUIDv4 under the current protected-main contract. Provider/native identifiers are stored only through explicit mappings. + +### TRD-DATA-003 — Database naming + +Product-owned database objects use descriptive multiword `snake_case` names unless an external protocol requires another spelling. + +### TRD-DATA-004 — Time + +Persist instants in UTC and retain IANA timezone/local-calendar values where civil-time semantics matter. Notification/habit logic cannot assume every local day is 24 hours. + +### TRD-DATA-005 — Immutability and concurrency + +Audit/decision/completion/outcome evidence defined as immutable/append-only rejects unauthorized mutation. Mutable data uses explicit revision, digest, ETag, idempotency, or fencing evidence wherever silent stale overwrite or duplicate side effects are plausible. + +### TRD-DATA-006 — Active-PR persistence claims + +A persistence entity or migration that exists only on an active PR is labeled `Implemented on active PR` in canonical documentation and logical ERD notes. It may be modeled as a planned/logical relationship for architecture discussion but cannot be called protected-main persisted state until exact merge evidence exists. + +## 6. Authentication and authorization + +- OAuth callbacks validate provider/state/redirect boundaries. +- Browser sessions are revocable and server-verifiable according to identity-service. +- Browser-selected workspace/actor headers are never trusted as authority. +- Signed private context binds the exact actor/workspace/method/path and bounded lifetime where required. +- Sensitive operations require purpose/resource/tenant authorization in addition to authentication. +- Integration credentials are distinct from login credentials and need independent storage, refresh, rotation, and revocation lifecycles. + +## 7. HTTP/API requirements + +1. Bound public/internal input before retaining untrusted bodies where practical. +2. Derive ownership from authenticated/signed context. +3. Use idempotency or equivalent replay protection for realistically repeatable mutations. +4. Use explicit stale-write preconditions where silent overwrite is unacceptable. +5. Return bounded credential-free problems/errors. +6. Do not expose stack traces, dependency bodies, tokens, or secrets. +7. Explicitly control redirects/origins for security-sensitive provider/model transport. +8. Version breaking semantics instead of silently changing a shared schema. +9. Active-PR routes/contracts are not documented as protected-main API until their exact implementation is merged. + +See `docs/API_CONTRACTS.md`. + +## 8. Event requirements + +When domain events are used: + +- event IDs are opaque and unique; +- type/version is explicit; +- actor/workspace/correlation/causation context is bounded and validated; +- payloads are immutable after publication; +- consumers are idempotent under replay/at-least-once delivery; +- database-to-event publication uses an outbox/equivalent reliability boundary where atomic effect is required; +- receiving an event never grants direct mutation authority over the producer's database. + +## 9. Concurrency and idempotency by domain + +### Planning / Today + +Protected main already requires durable planning writes to reject or reconcile stale state and keeps browser-local drafts visibly distinct from durable state. PR #127 is `Implemented on active PR` for the complete bounded versioned Today aggregate under issue #121: session-derived workspace authority, `If-None-Match` creation, strong `If-Match` update, monotonic opaque revision evidence, idempotency-key replay semantics, PostgreSQL persistence, and explicit local draft migration/conflict recheck. It remains non-protected-main evidence until merge. + +### Habit completion + +Use tenant-scoped idempotency and concurrency-safe persistence so duplicates/concurrent submissions resolve to one valid completion lifecycle. + +### Notifications + +Claims are fenced/expiring, delivery IDs replay-safe, and duplicate delivery or immutable-outcome mutation is rejected. + +### Calendar + +Use deterministic provider identifiers and strong preconditions/ETags where available. Provider responses are untrusted and bounded. + +### AI decisions + +Bind accept/reject to exact proposal revision/digest, actor, workspace, and idempotency identity. Stale proposals cannot be silently accepted as current. + +### Privacy grants + +Time-bounded/single-use grants fail closed at exact expiry/consumption and cannot be repurposed across actor/purpose/resource. + +## 10. AI and autonomous-development requirements + +### Product AI authority + +- Model output is untrusted structured data. +- Deterministic validators/product authorization remain authoritative. +- AI cannot silently mutate user-owned planning data. +- Proposal evaluation independently covers validity, operation conformance, grounding, utility, leakage, and prompt-injection resistance as implemented. + +### Live provider separation + +- Live provider availability is not fabricated into a deterministic merge pass. +- Provider absence/outage returns sanitized unavailable evidence. +- `NVIDIA_NIM_API_KEY` is scoped only to the approved model boundary. +- Browser/session/GitHub/review-agent credentials never become model inputs or retained model artifacts. + +### Orchestration + +A strong single-route model is the mandatory baseline. Deeper orchestration exposes workflow stage, reasoning effort, decomposition, recursion, roles, and access topology and is justified by measured quality/control benefit. + +### Repository autonomous development + +The merged OpenCode scheduler: + +- runs on an hourly/manual cadence; +- selects bounded repository-owned work under deterministic policy; +- isolates untrusted issue/model content; +- verifies exact base/diff/path/content/test authority before remote mutation; +- may create one draft PR through its reviewed credentialed boundary; +- cannot treat model output as merge/release/repository-administration authority; +- remains subject to normal CI, AppGuardrail, Semgrep, Security Scan, CodeRabbit/review, and exact-head merge policy. + +PR #133 is an `Implemented on active PR` hardening path for that scheduler. Its scope is to resolve explicit NVIDIA model catalog selection and prove real Compose verification with digest-pinned PostgreSQL/NATS without introducing broad Docker/network authority for the model. Its checks/evidence do not transfer to protected main until merge. + +## 11. Readiness and buyer-gap governance + +### TRD-GOV-001 — Separate readiness dimensions + +Configured capability-evidence maturity and canonical buyer-gap exhaustion are independent dimensions under ADR-0008. The system must not derive whole-product completeness solely from `product/capabilities.json` maturity. + +### TRD-GOV-002 — Repository-owned gap identity + +Buyer-gap identity, linked capability/requirement, and expected issue mapping are versioned repository-owned data. Arbitrary issue title/body/comment/review/model text is untrusted and cannot redefine executable readiness policy. + +### TRD-GOV-003 — Fail-closed reconciliation + +Live issue-state evidence may reconcile a registered gap as `open`, `resolved`, or `unknown`. Fetch failure, missing evidence, malformed registry content, duplicate identity or unknown capability references fail closed; unknown does not become resolved. + +Issue #128 tracks the defect and PR #131 is the `Implemented on active PR` path. The protected-main capability maturity calculation remains its own dimension until the active implementation is integrated. + +## 12. External integration requirements + +### Identity providers + +Provider registration/redirect policy/secrets are operator-owned inputs. Provider outage degrades new login/link operations, not unrelated authenticated domain state where a valid session remains usable. + +### Calendar providers + +CalDAV/Google adapters validate provider origin/response/ETag and expose only reviewed operations. Hosted unattended multi-user Google synchronization cannot be called complete until #129 proves encrypted per-user credential storage/refresh/revocation/discovery/selection. + +### Plugins + +Manifests/events are bounded, versioned, tenant scoped, and have no direct DB authority. Installation/secrets/outbound network delivery requires a separate least-authority/SSRF/audit boundary tracked by issue #130. + +## 13. Security and privacy requirements + +- Treat all external responses, model output, stored JSON, environment values, connector results, calendar/plugin payloads, issue/review content, and decoded database rows as untrusted until validated. +- Parameterize SQL; never interpolate untrusted values into SQL structure. +- Use least-privilege GitHub/runtime/database permissions. +- Bound subprocess/network/file operations by time and size when exhaustion is plausible. +- Retained CI/model/public artifacts exclude credentials, browser cookies, raw prompts/responses, hidden reasoning, and unbounded tenant content. +- `SECURITY.md` governs reporting; `docs/THREAT_MODEL.md` governs architecture threats. +- Sensitive access uses purpose/resource/actor/lifetime control and auditable evidence rather than blanket masking. +- Export/delete lifecycle is `Partial` under issue #55 until concrete contributors, durable orchestration/reconciliation, recent-auth, retention/legal-hold, protected delivery, and audit evidence are complete. + +See `docs/PRIVACY_DATA_LIFECYCLE.md`. + +## 14. Web/PWA/accessibility/localization + +- Core journeys support keyboard navigation and visible focus. +- Essential status is not color-only. +- Reduced-motion preferences are respected where relevant. +- Korean/English catalogs remain structurally aligned. +- PWA/local drafts never imply successful durable synchronization until server acceptance is proven. +- Stale asynchronous requests cannot replace the latest owned UI state after query/navigation/unmount changes. +- PR #127 may be used as active-PR evidence for its bounded Today sync journey but not as evidence for unrelated offline-first behavior. + +## 15. Observability and diagnostics + +- Services expose bounded health/readiness appropriate to their actual responsibility. +- Metrics are an operator surface and production ingress restricts them. +- Correlation IDs propagate where implemented. +- Logs are structured, bounded, credential-free, and avoid unnecessary personal text. +- Errors distinguish validation/authentication/authorization/conflict/rate-limit/dependency/unexpected classes without leaking dependency internals. +- Numeric SLA/SLO objectives exist only in measured/operator-specific scoped documents. + +## 16. Backup, migration, rollback, and deployment + +- Schema changes include migration compatibility plus rollback or forward-fix evidence appropriate to risk. +- Logical backup produces checksum/integrity evidence and restore rejects corruption/unsafe non-empty targets. +- Logical dumps do not imply PITR; WAL/archive/replication are operator work until explicitly implemented. +- Workload rollback claims cover only state explicitly captured/verified by the deployment workflow; completed DB migrations/external infrastructure are not represented as automatically reversible. +- Compose provides local/self-hosted composition. +- Kubernetes artifacts are a provider-neutral hardened reference and deliberately do not provision cluster, DB, NATS, ingress/TLS/DNS, registry, or secret manager. + +See `docs/OPERABILITY.md` and `docs/RELEASE_AND_MIGRATION.md`. + +## 17. Test and documentation requirements + +- Unit tests cover deterministic domain invariants and malformed inputs. +- PostgreSQL tests cover durability, tenancy, transactions, replay, concurrency, expiry, and recovery. +- HTTP integration covers actual authority/problem boundaries. +- Browser E2E covers core journey, localization, accessibility, and PWA behavior where applicable. +- Security regressions cover AppGuardrail/GHAS classes and model/provider/trust boundaries. +- Backup/restore/deployment references have executable contract tests. +- Deterministic gates are separate from bounded live-provider evidence. +- Owned packages declaring exact gates maintain meaningful 100% statement/branch/function/line coverage. +- Documentation consistency validates real canonical link targets, exact canonical status vocabulary, ADR index targets/status, source/configuration/migration evidence for key claims, active-PR versus protected-main state, and live buyer-gap traceability. +- A resolved review thread is historical evidence only; current exact source is revalidated after every branch mutation. + +See `docs/TEST_STRATEGY.md`. + +## 18. Release requirements + +A stable release requires one exact protected integrated head with applicable: + +- required CI/security/dependency/review gates; +- exact owned-code coverage; +- package/container build/smoke evidence; +- migration/recovery/backup evidence; +- accessibility/localization/browser evidence; +- canonical buyer-gap state plus configured capability-evidence maturity; +- SBOM/provenance/reproducibility evidence required by policy; +- no unresolved valid security/review finding; +- release notes/changelog matching the artifact. + +Do not bump product version merely because one PR, documentation set, configured capability score, or intermediate product slice is complete. diff --git a/docs/UML.md b/docs/UML.md new file mode 100644 index 00000000..0fa7ced1 --- /dev/null +++ b/docs/UML.md @@ -0,0 +1,391 @@ +# LifeOS UML and Interaction Views + +**Baseline:** protected `main` at `876850018a17323900844e79845ba395b7bf6a9a` + +These diagrams are architecture documentation, not proof that every target path is fully implemented. Sections explicitly state current status. Service names and authority boundaries must remain synchronized with protected-main code and current active-PR evidence. + +## 1. Component / bounded-context view + +**Status:** Implemented on protected main + +The bounded contexts and service-owned persistence ports shown here exist on protected main. Some external-credential and complete buyer journeys remain partial and are called out in the scoped sections below. + +```mermaid +flowchart TB + User[User] + Web[Next.js Web / PWA] + Gateway[Gateway / BFF] + Identity[Identity Service] + Planning[Planning Service] + Habit[Habit Service] + Review[Review Service] + Calendar[Calendar Integration Service] + Notification[Notification Service] + AI[AI Proposal Service] + Privacy[Privacy Access Service] + Plugin[Plugin Integration Service] + NATS[(NATS JetStream)] + + User --> Web + Web --> Gateway + Gateway --> Identity + Gateway --> Planning + Gateway --> Habit + Gateway --> Review + Gateway --> AI + Gateway --> Privacy + Gateway --> Calendar + Gateway --> Plugin + + Planning -. publishes domain events .-> NATS + Habit -. publishes domain events .-> NATS + Review -. publishes projection/events .-> NATS + NATS -. delivers reminder/event inputs .-> Notification + + Identity --> IdentityDB[(Identity-owned PostgreSQL schema / role)] + Planning --> PlanningDB[(Planning-owned PostgreSQL schema / role)] + Habit --> HabitDB[(Habit-owned PostgreSQL schema / role)] + Review --> ReviewDB[(Review-owned PostgreSQL schema / role)] + Notification --> NotificationDB[(Notification-owned PostgreSQL schema / role)] + AI --> AIDB[(AI-owned PostgreSQL schema / role)] + Privacy --> PrivacyDB[(Privacy-owned PostgreSQL schema / role)] + + Calendar --> ExternalCalendar[Google Calendar / CalDAV] + Identity --> IdentityProvider[Google / GitHub Identity] + AI --> ModelBoundary[Local rule model or approved contextual-orchestrator/model boundary] +``` + +Each database edge above represents only the owning service's persistence port. Co-location on one PostgreSQL cluster does not imply shared credentials, cross-service table access, or foreign-key authority. + +## 2. Login and workspace authorization sequence + +**Status:** Implemented on protected main + +Identity OAuth/session behavior is implemented. Exact personal-workspace provisioning details follow current identity protected-main source and tests rather than this diagram alone. + +```mermaid +sequenceDiagram + actor User + participant Web + participant Gateway + participant Identity + participant Provider as Google/GitHub + participant Domain as Planning or other domain service + + User->>Web: Start sign-in + Web->>Gateway: Auth start request + Gateway->>Identity: Begin provider flow + Identity->>Provider: OAuth/OIDC authorization + Provider-->>Identity: Callback data + Identity->>Identity: Validate provider/state and map external identity + Identity->>Identity: Establish revocable LifeOS session/workspace authority + Identity-->>Web: Secure browser session boundary + User->>Web: Open tenant-scoped feature + Web->>Gateway: Browser request + session + Gateway->>Identity: Validate session / derive workspace+actor + Identity-->>Gateway: Opaque LifeOS authority context + Gateway->>Domain: Bounded request with derived/signed context + Domain-->>Gateway: Tenant-scoped result + Gateway-->>Web: Credential-free response +``` + +Provider credentials and browser cookies do not become arbitrary downstream-service inputs. + +## 3. Goal → Project → Task and Habit lifecycle + +**Status:** Implemented on protected main + +Goal/project/task and recurring-habit persistence are implemented. This is a logical domain flow; physical relationships follow planning/habit service implementations. Milestones/task dependencies from historical planning are not shown as protected-main persisted states; see `docs/DATA_MODEL.md`. + +```mermaid +stateDiagram-v2 + [*] --> Captured + Captured --> Goal: classify as objective + Captured --> Project: classify as coordinated outcome + Captured --> Task: classify as actionable work + Captured --> Habit: classify as recurring behavior + + Goal --> Project: create/support project + Project --> Task: project action + Habit --> HabitScheduled: recurrence creates due behavior + + Task --> TaskCompleted: explicit completion + HabitScheduled --> HabitCompleted: explicit completion + TaskCompleted --> ReviewEvidence + HabitCompleted --> ReviewEvidence + ReviewEvidence --> [*] +``` + +Review evidence does not directly rewrite planning/habit source-of-truth records. + +## 4. Today planning and stale-state boundary + +**Status:** Implemented on active PR + +Protected main already has the Today action loop and local-draft/durable-record distinction. PR #127 implements the complete bounded durable Today aggregate, explicit local-to-durable synchronization, optimistic concurrency, idempotency and conflict/recheck browser journey for issue #121. This sequence must not be treated as protected-main evidence until that exact PR head merges. + +```mermaid +sequenceDiagram + actor User + participant Browser as Web/PWA + participant Gateway + participant Planning + + User->>Browser: Capture / select Today priorities + Browser->>Browser: Maintain explicitly labeled local draft state + Browser->>Gateway: Check current durable Today state + Gateway->>Planning: Tenant-scoped GET / durable aggregate lookup + Planning-->>Gateway: Durable aggregate + strong revision evidence + Gateway-->>Browser: Current durable evidence + Browser->>Browser: Discard stale async response if ownership/query/navigation changed + User->>Browser: Explicit save/load/reconcile action + Browser->>Gateway: Mutation + If-Match/If-None-Match + idempotency evidence + Gateway->>Planning: Authorized write derived from session workspace + alt preconditions current + Planning-->>Gateway: Accepted durable revision + Gateway-->>Browser: Confirm durable state + else stale/conflicting + Planning-->>Gateway: Credential-free conflict/current revision evidence + Gateway-->>Browser: Recheck and reconcile explicitly; no silent overwrite + end +``` + +Issue #121 remains open until the reviewed implementation is protected-main evidence and its acceptance gates are complete. + +## 5. Reminder delivery sequence + +**Status:** Implemented on protected main + +This sequence describes durable PostgreSQL reminder scheduling and in-app delivery behavior. + +```mermaid +sequenceDiagram + participant Scheduler + participant Store as Notification-owned PostgreSQL Repository + participant Gateway as In-app Delivery Gateway + + Scheduler->>Store: Claim due occurrence with bounded lease + alt claim acquired + Store-->>Scheduler: Fenced claim token + Scheduler->>Scheduler: Apply timezone quiet-hour and fatigue policy + alt deliver now + Scheduler->>Gateway: Deliver with idempotent delivery key + Gateway-->>Scheduler: Delivery outcome + Scheduler->>Store: Append immutable outcome + else defer + Scheduler->>Store: Persist next due instant + immutable defer outcome + end + else already claimed/completed + Store-->>Scheduler: No duplicate authority + end +``` + +## 6. Calendar synchronization sequence + +**Status:** Implemented on protected main + +CalDAV/Google provider adapters are implemented. Hosted per-user Google credential persistence, refresh, revocation and provider/calendar selection remain `Partial` under issue #129. + +```mermaid +sequenceDiagram + actor User + participant Web + participant Gateway + participant Calendar + participant Provider as Google Calendar / CalDAV + + User->>Web: Synchronize selected commitment + Web->>Gateway: Authorized sync intent + Gateway->>Calendar: Tenant-scoped bounded command + Calendar->>Calendar: Validate provider configuration + deterministic remote identity + Calendar->>Provider: Create/update with strong precondition where supported + alt provider accepts + Provider-->>Calendar: Remote ID / ETag + Calendar-->>Gateway: Bounded sync result + Gateway-->>Web: Success evidence + else precondition/provider failure + Provider-->>Calendar: Conflict/error + Calendar-->>Gateway: Sanitized classified failure + Gateway-->>Web: Recover/retry/reconcile state + end +``` + +## 7. AI proposal and explicit decision sequence + +**Status:** Implemented on protected main + +Proposal generation, persistence, evidence and decision history are implemented. This diagram does not imply automatic planning mutation. + +```mermaid +sequenceDiagram + actor User + participant Web as Authenticated Web/BFF + participant Identity + participant AI as AI Proposal Service + participant Audit as AI-owned Proposal Store + + User->>Web: Request proposal + Web->>Identity: Resolve session authority + Identity-->>Web: Workspace + actor + Web->>AI: Signed method/path/workspace/actor + bounded context + AI->>AI: Treat context/model output as untrusted and validate + AI->>Audit: Persist immutable proposal evidence + Audit-->>AI: Proposal revision/digest + AI-->>Web: Inert proposal + Web-->>User: Present proposal + evidence + User->>Web: Explicit accept or reject + Web->>AI: Decision bound to exact proposal revision/digest + AI->>Audit: Append replay-safe decision event + Audit-->>AI: Decision evidence + AI-->>Web: Decision recorded +``` + +The AI service is not a generic planning command bus. + +## 8. Purpose-bound privacy access sequence + +**Status:** Implemented on protected main + +The privacy-service authorization/grant/evidence core is implemented. Complete user-facing export/deletion orchestration remains `Partial` under issue #55. + +```mermaid +sequenceDiagram + participant Caller as Authorized Service/Operator Boundary + participant Privacy as Privacy Access Service + participant Store as Privacy-owned PostgreSQL Repository + + Caller->>Privacy: Signed actor/resource/purpose request + Privacy->>Privacy: Validate context, purpose, bounds and policy + Privacy->>Store: Append access decision + alt denied + Store-->>Privacy: Denial evidence + Privacy-->>Caller: Sanitized denial + else allowed + Privacy->>Store: Persist bounded single-use/time-limited grant + Store-->>Privacy: Grant evidence + Privacy-->>Caller: Signed grant token/handle + Caller->>Privacy: Consume grant for exact authorized operation + Privacy->>Store: Atomically validate/consume and append event + Privacy-->>Caller: Authorized result boundary + end +``` + +No other bounded service is permitted to use this `Store`; other services interact with the privacy service through its reviewed API/context boundary. + +## 9. Backup and restore state flow + +**Status:** Implemented on protected main + +The verified logical dump/restore tier is implemented. PITR is not claimed. + +```mermaid +stateDiagram-v2 + [*] --> BackupRequested + BackupRequested --> ArchiveCreated + ArchiveCreated --> ChecksumRecorded + ChecksumRecorded --> BackupComplete + BackupComplete --> RestoreRequested + RestoreRequested --> IntegrityVerified + IntegrityVerified --> TargetChecked + TargetChecked --> RestoreRunning: target deliberately empty + TargetChecked --> RestoreRefused: non-empty/unsafe target + RestoreRunning --> RestoreVerified + RestoreVerified --> [*] + RestoreRefused --> [*] + IntegrityVerified --> RestoreRefused: checksum/corruption failure +``` + +## 10. Deployment topology + +**Status:** Implemented on protected main + +Compose and Kubernetes provider-neutral reference artifacts exist. Cluster, DB/NATS managed services, ingress/TLS/DNS, registry pipeline and secret manager remain operator-owned; `reference` describes scope rather than a separate implementation status. + +```mermaid +flowchart TB + Client[Browser / PWA] + Ingress[Operator-owned HTTPS ingress] + Web[Web workload] + Gateway[Gateway workload] + Identity[Identity workload] + Planning[Planning workload] + Habit[Habit workload] + Review[Review workload] + Notification[Notification workload] + AI[AI workload] + Privacy[Privacy workload] + Integrations[Calendar / Plugin workloads] + PG[(Operator-owned PostgreSQL cluster)] + NATS[(Operator-owned NATS JetStream)] + Secrets[Operator-owned Secret Manager / protected environment] + Providers[Identity / Calendar / Model Providers] + Metrics[Operator monitoring network] + + Client --> Ingress + Ingress --> Web + Ingress --> Gateway + Web --> Gateway + + Gateway --> Identity + Gateway --> Planning + Gateway --> Habit + Gateway --> Review + Gateway --> AI + Gateway --> Privacy + Gateway --> Integrations + + Identity -->|identity-only DSN/role/schema| PG + Planning -->|planning-only DSN/role/schema| PG + Habit -->|habit-only DSN/role/schema| PG + Review -->|review-only DSN/role/schema| PG + Notification -->|notification-only DSN/role/schema| PG + AI -->|ai-only DSN/role/schema| PG + Privacy -->|privacy-only DSN/role/schema| PG + + Planning -. publishes .-> NATS + Habit -. publishes .-> NATS + Review -. publishes .-> NATS + NATS -. delivers bounded inputs .-> Notification + + Secrets --> Gateway + Secrets --> Identity + Secrets --> Planning + Secrets --> Habit + Secrets --> Review + Secrets --> Notification + Secrets --> AI + Secrets --> Privacy + Secrets --> Integrations + Identity --> Providers + AI --> Providers + Integrations --> Providers + + Gateway -. metrics .-> Metrics + Identity -. metrics .-> Metrics + Planning -. metrics .-> Metrics + Habit -. metrics .-> Metrics + Review -. metrics .-> Metrics + Notification -. metrics .-> Metrics + AI -. metrics .-> Metrics + Privacy -. metrics .-> Metrics + Integrations -. metrics .-> Metrics +``` + +A shared physical PostgreSQL cluster is only a deployment co-location choice: every service uses its own authorized DSN/role/schema boundary and may not traverse into another service's tables. + +## 11. Failure and degraded-mode view + +```mermaid +flowchart LR + Request[User intent] --> Auth{Session/authority valid?} + Auth -->|No| AuthFail[Credential-free auth failure] + Auth -->|Yes| Domain{Owning domain available?} + Domain -->|No| DomainFail[Classified dependency unavailable; no fabricated success] + Domain -->|Yes| External{External provider required?} + External -->|No| Complete[Complete local/domain operation] + External -->|Yes| Provider{Provider available + preconditions valid?} + Provider -->|Yes| Complete + Provider -->|No| Degraded[Preserve local truth; return bounded retry/conflict/unavailable evidence] +``` + +A calendar/model/provider outage does not authorize LifeOS to fabricate successful synchronization, proposal quality, or external side effects. diff --git a/docs/adr/0001-opaque-non-numeric-identifiers.md b/docs/adr/0001-opaque-non-numeric-identifiers.md index 9b900e25..1dd0fc39 100644 --- a/docs/adr/0001-opaque-non-numeric-identifiers.md +++ b/docs/adr/0001-opaque-non-numeric-identifiers.md @@ -1,7 +1,7 @@ # ADR 0001: Opaque non-numeric identifiers -- **Status:** Accepted -- **Date:** 2026-08-02 +**Status:** Superseded +**Date:** 2026-08-02 ## Context @@ -9,25 +9,52 @@ Sequential numeric identifiers expose record counts, creation order, and easily LifeOS is a public multi-user service, so identifiers visible in APIs, events, URLs, logs, exports, and database relationships must not reveal sequence or cardinality. +## Drivers + +- avoid sequential/enumerable internal identifiers; +- keep third-party provider identifiers outside LifeOS primary-key authority; +- retain tenant-scoped authorization independently from locator opacity; +- use a repository-wide identifier contract that tests and migrations can enforce. + +## Alternatives + +1. Sequential database-generated integer identifiers. +2. Time-ordered UUID variants. +3. Cryptographically random UUIDv4 identifiers with explicit provider mappings. + ## Decision 1. Internal entity identifiers use cryptographically random UUIDv4 values represented as strings in application code and as PostgreSQL `uuid` columns in persistence. 2. Numeric primary keys, auto-increment columns, database sequences, and numeric-only public identifiers are prohibited. 3. Workspace, user, session, goal, project, task, habit, review, event, correlation, causation, export-job, and integration identifiers follow the same rule. -4. Client-supplied identifiers are validated as non-empty, non-numeric opaque strings before repository access. -5. Third-party identifiers are never reused as LifeOS primary keys. Provider identity is stored separately as `(provider, provider_subject)` text and mapped to an independent LifeOS UUIDv4 user ID. -6. OAuth provider subjects that happen to be numeric, such as some GitHub account IDs, remain external attributes only and are never exposed as internal resource IDs. -7. Public pagination uses opaque signed or encrypted cursors rather than offsets or row IDs. -8. IDs are authorization locators, not authorization evidence. Every lookup remains workspace- and actor-scoped. - -## Why UUIDv4 - -UUIDv4 is preferred over sequential integers and time-ordered identifiers because it does not reveal creation time or ordering through the identifier itself. The collision probability is negligible for this system when generated with a cryptographically secure source. +4. Client-supplied identifiers are validated before repository access. +5. Third-party identifiers are never reused as LifeOS primary keys. Provider identity is stored separately and mapped to an independent LifeOS UUIDv4 identifier. +6. IDs are authorization locators, not authorization evidence. Every lookup remains workspace- and actor-scoped. ## Consequences - Database indexes are larger than integer indexes. - Logs and URLs are less human-readable. -- Tests must verify generated IDs are UUIDv4 and reject numeric-only supplied identifiers. +- Tests verify generated IDs are UUIDv4 and reject numeric/sequential supplied identifiers at protected boundaries. - Foreign keys remain explicit and tenant-aware; opaque IDs do not replace authorization or tenant isolation. -- The earlier design note proposing UUIDv7 is superseded by this ADR. +- Chronological ordering uses explicit temporal/revision fields instead of identifier order. + +## Failure and recovery + +Malformed or nonconforming internal identifiers fail closed before ownership-sensitive repository operations. Existing valid UUIDv4 identifiers require no data migration. + +## Security and privacy impact + +Opaque identifiers reduce enumeration and provider-identity leakage, but do not weaken the requirement for authenticated, tenant-scoped authorization. + +## Acceptance evidence + +This historical decision is reflected by protected-main UUIDv4 validators, migrations, security tests, and repository-wide agent/architecture contracts. ADR-0002 restates the same current invariant in the canonical documentation graph with explicit migration and supersession guidance. + +## Migration / rollback + +No data migration is required by this record because the protected-main data model already uses UUIDv4. Any future identifier-version change requires a separately reviewed compatibility and migration decision. + +## Supersession + +**Superseded by ADR-0002, `Opaque UUIDv4 internal identifiers`.** ADR-0002 is the canonical current formulation; this file is retained because ADR identifiers and historical rationale must not disappear when the documentation graph is consolidated. diff --git a/docs/adr/0002-internal-identifiers-uuidv4.md b/docs/adr/0002-internal-identifiers-uuidv4.md new file mode 100644 index 00000000..e8e0efc1 --- /dev/null +++ b/docs/adr/0002-internal-identifiers-uuidv4.md @@ -0,0 +1,60 @@ +# ADR-0002: Opaque UUIDv4 internal identifiers + +**Status:** Accepted architecture +**Date:** 2026-08-09 + +## Context + +The original 2026-08-02 combined design proposed UUIDv7 identifiers. Protected-main `AGENTS.md`, `CLAUDE.md`, `ARCHITECTURE.md`, migrations and regression tests now establish opaque UUIDv4 internal identifiers and reject sequential/non-v4 values in security-sensitive domains. Provider-native numeric identifiers must not become LifeOS primary keys. + +Leaving UUIDv7 in an apparently authoritative design creates contradictory schema and API guidance. + +## Drivers + +- one repository-wide identifier invariant; +- opacity across tenant and provider boundaries; +- resistance to accidental exposure of provider identity/order; +- compatibility with existing protected-main migrations/tests; +- no numeric internal primary keys. + +## Alternatives + +1. Keep UUIDv7 from the historical design. +2. Use mixed UUID versions by service. +3. Standardize current internal IDs on UUIDv4 and treat external IDs as provider metadata. + +## Decision + +All new LifeOS internal object identifiers use **UUIDv4** unless a separately reviewed external protocol requires another identifier form at its boundary. Numeric provider IDs and external resource IDs are mapped through explicit provider records and never reused as internal primary keys. + +The historical UUIDv7 statement is superseded. + +## Consequences + +- APIs/database migrations/tests share one current identifier contract. +- Chronological ordering must use explicit timestamp/revision fields rather than ID ordering. +- Existing UUIDv4 rows need no migration. +- External sortable IDs remain an adapter concern, not an internal schema shortcut. + +## Failure and recovery + +Malformed, non-v4, sequential or provider-native identifiers at internal boundaries fail closed before ownership-sensitive operations. Public errors do not echo sensitive rows or credentials. + +## Security/privacy impact + +Opaque IDs reduce information leakage and make provider-identity confusion less likely, but UUIDs are not authorization. Every operation still enforces authenticated workspace/resource authority. + +## Acceptance evidence + +- `AGENTS.md` UUIDv4 invariant; +- `CLAUDE.md` UUIDv4 invariant; +- root `ARCHITECTURE.md` UUIDv4 invariant; +- protected-main tests/migrations rejecting non-v4 identifiers in relevant services. + +## Migration / rollback + +No migration is required for the accepted protected-main baseline. A future change to another UUID version requires explicit cross-service migration and API compatibility evidence; it must not reinterpret existing UUIDv4 rows. + +## Supersession + +Supersede only through an ADR that proves product benefit, cross-service migration safety, tenant/security effects, indexing/performance effects and backwards compatibility. diff --git a/docs/adr/0002-oauth-transactions-and-session-tokens.md b/docs/adr/0002-oauth-transactions-and-session-tokens.md index 5d18cb60..53db1d11 100644 --- a/docs/adr/0002-oauth-transactions-and-session-tokens.md +++ b/docs/adr/0002-oauth-transactions-and-session-tokens.md @@ -1,7 +1,8 @@ # ADR 0002: OAuth transactions and session tokens -- **Status:** Accepted -- **Date:** 2026-08-03 +**Status:** Implemented on protected main +**Date:** 2026-08-03 +**Identity note:** This historical ADR already used number `0002` before the canonical documentation baseline introduced `0002-internal-identifiers-uuidv4.md`. The repository therefore treats the full ADR filename, not the four-digit prefix alone, as the stable index identity for pre-baseline collisions. ## Context @@ -9,6 +10,21 @@ LifeOS accepts Google and GitHub sign-in while maintaining provider-neutral inte The repository already contained provider authorization and token-exchange builders. This decision hardens the shared `auth-security` transaction and session layer rather than introducing a second implementation. +## Drivers + +- prevent authorization response injection, replay, redirect substitution, and authorization-server mix-up; +- keep bearer/session credentials out of durable plaintext storage and public diagnostics; +- retain provider-neutral internal identity and workspace authority; +- make browser sessions revocable and safely rotatable; +- preserve exact callback and PKCE/OIDC transaction binding with bounded lifetime. + +## Alternatives + +1. Trust provider callbacks without a durable one-time transaction record. +2. Persist raw `state`, session bearer tokens, PKCE verifiers, or provider tokens as ordinary application values. +3. Share one ambiguous callback/issuer path for every provider without explicit provider binding. +4. Keep the provider adapters but centralize the transaction/session primitives in the identity boundary, as selected here. + ## Decision ### Authorization transactions @@ -48,3 +64,23 @@ The repository already contained provider authorization and token-exchange build - Existing callers must supply the initiating browser-session identifier and the exact redirect URI when creating and consuming transactions. - Existing sessions are backfilled to their owners' personal workspaces by migration `0003_oauth_binding_and_session_rotation.sql`. - Provider callback adapters remain responsible for network exchange, provider response validation, ID-token validation for Google, and profile retrieval; this ADR supplies the transaction and session primitives they must use. + +## Failure and recovery + +Malformed, expired, replayed, provider-mismatched, browser-session-mismatched, or redirect-mismatched authorization transactions fail closed before token exchange or identity mutation. A failed session rotation does not make a new bearer token authoritative until its replacement session has been persisted under the identity-service transaction boundary. Provider/network failures remain sanitized dependency failures rather than exposing token responses. + +## Security and privacy impact + +Only digests of one-time `state` and session bearer values are durable. Provider credentials, PKCE verifier material, OIDC nonces, and browser session tokens stay within their reviewed identity/provider boundary. Opaque identifiers do not replace workspace authorization, and a valid provider identity never grants access to an unrelated LifeOS workspace. + +## Acceptance evidence + +Protected main contains the identity-service transaction/session implementation, OAuth callback/provider integration tests, UUIDv4/session ownership migrations, replay/expiry/redirect/provider-binding regressions, and credential-safe browser-session handling. Exact source and migration evidence remains authoritative over this prose. + +## Migration / rollback + +Migration `0003_oauth_binding_and_session_rotation.sql` backfilled existing sessions to their owners' personal workspaces and established the rotation/ownership constraints used by the implemented boundary. Future changes to token storage, authentication-age semantics, provider binding, or credential encryption require forward-compatible migration evidence and must not silently reinterpret existing session provenance. + +## Supersession + +This decision remains implemented on protected main. It is superseded only by a later indexed ADR that preserves or explicitly migrates the transaction anti-replay, PKCE/OIDC, provider-binding, credential-storage, workspace-authority, and session-rotation security properties. The duplicated historical `0002` numeric prefix is not itself a reason to discard or rewrite this ADR's decision history; canonical indexing uses the full filename for identity. diff --git a/docs/adr/0003-domain-oriented-service-data-ownership.md b/docs/adr/0003-domain-oriented-service-data-ownership.md new file mode 100644 index 00000000..8af6cbd3 --- /dev/null +++ b/docs/adr/0003-domain-oriented-service-data-ownership.md @@ -0,0 +1,56 @@ +# ADR-0003: Domain-oriented service and data ownership + +**Status:** Accepted architecture +**Date:** 2026-08-09 + +## Context + +LifeOS contains strongly related planning, habit, review, notification, identity, integration, AI and privacy capabilities. A shared database or arbitrary service-to-service table access would make tenancy, migrations, incident containment and independent deployment difficult to reason about. + +## Drivers + +- explicit business authority; +- tenant isolation; +- independent migrations/tests/runtime; +- replaceable service implementations; +- safe event/API evolution; +- bounded failure and rollback. + +## Alternatives + +1. Shared relational schema with cross-service joins/writes. +2. One service per entity regardless of consistency boundary. +3. Domain-oriented bounded services with owned persistence and versioned contracts. + +## Decision + +Use domain-oriented bounded services. Each service owns its persistence, migrations, runtime configuration, observability and shutdown behavior. Cross-service interaction uses versioned HTTP, event, saga, plugin or MCP contracts. Direct cross-service table reads/writes are prohibited. + +Planning keeps goals/projects/milestones/tasks together while they form one consistency boundary; splitting is driven by measured ownership/load, not entity count. + +## Consequences + +- Duplicate/projection data may exist where needed for read models. +- Cross-service workflows require explicit idempotency and failure handling. +- Logical ERD links are not physical foreign-key authority. +- Local Compose may run one PostgreSQL cluster while retaining schema/credential ownership. + +## Failure/recovery + +When a dependency is unavailable, the caller returns/defer/compensates according to its contract rather than directly repairing another service's tables. Event consumers are idempotent under replay. + +## Security/privacy impact + +Separate authority limits blast radius and supports minimum database privileges. Shared IDs are correlation values, not authorization. + +## Acceptance evidence + +`ARCHITECTURE.md`, `AGENTS.md`, service-specific repositories/migrations, PostgreSQL integration tests and versioned integration/plugin boundaries. + +## Migration/rollback + +Moving ownership requires a dedicated migration/saga plan, dual-read/write only when explicitly bounded, reconciliation evidence and removal of the old authority after cutover. + +## Supersession + +Requires evidence that another ownership model improves reliability/product delivery without weakening tenancy, migration safety or independent operation. diff --git a/docs/adr/0004-inert-auditable-ai-proposals.md b/docs/adr/0004-inert-auditable-ai-proposals.md new file mode 100644 index 00000000..47ef77dc --- /dev/null +++ b/docs/adr/0004-inert-auditable-ai-proposals.md @@ -0,0 +1,55 @@ +# ADR-0004: AI proposals are inert and auditable + +**Status:** Accepted architecture +**Date:** 2026-08-09 + +## Context + +LifeOS can use models for planning assistance, but personal planning state is user-owned. Allowing model output to directly mutate tasks/goals/habits would merge probabilistic generation, authorization and transactional authority into one unsafe boundary. + +## Drivers + +- preserve user agency; +- prompt-injection/model-error containment; +- durable provenance and review; +- provider portability; +- replay-safe explicit decisions; +- deterministic product validation independent of provider availability. + +## Alternatives + +1. Let the model call arbitrary mutation tools. +2. Let the AI service own a planning repository/command bus. +3. Persist inert proposals, validate them deterministically and record explicit accept/reject decisions separately. + +## Decision + +AI/model output is untrusted, inert proposal data. The AI service may generate, validate, persist and retrieve proposal evidence and append explicit decision events. It does not receive generic planning mutation authority. Browser credentials/provider secrets are not forwarded to the model. + +A strong single-model route is the evaluation baseline; deeper orchestration requires measured benefit without safety regression. + +## Consequences + +- Proposal acceptance does not implicitly mean planning mutation exists; any future execution capability requires a separately authorized contract. +- The product can audit what was proposed and what the user decided. +- Live provider outages do not require weakening deterministic CI/product gates. + +## Failure/recovery + +Malformed model output, stale proposal revisions, unavailable providers and validation failures produce bounded classified evidence. They never fabricate a successful proposal/decision or silently mutate planning data. + +## Security/privacy impact + +Reduces prompt-injection blast radius and credential exposure. Retained artifacts exclude raw prompts/responses, hidden reasoning, credentials and unbounded tenant data. + +## Acceptance evidence + +Protected-main AI proposal persistence, same-origin signed-context tests, append-only/replay-safe decision tests, proposal-quality evaluator, and root architecture boundary. + +## Migration/rollback + +Future execution actions must map accepted proposal intent to narrowly authorized domain commands and preserve proposal/decision provenance. Rollback disables execution without corrupting proposal history. + +## Supersession + +Only a reviewed ADR with an equally strong authorization, provenance, user-consent, prompt-injection and rollback model may change this authority split. diff --git a/docs/adr/0005-purpose-bound-sensitive-data-access.md b/docs/adr/0005-purpose-bound-sensitive-data-access.md new file mode 100644 index 00000000..510e4e51 --- /dev/null +++ b/docs/adr/0005-purpose-bound-sensitive-data-access.md @@ -0,0 +1,55 @@ +# ADR-0005: Purpose-bound sensitive-data access + +**Status:** Accepted architecture +**Date:** 2026-08-09 + +## Context + +LifeOS may contain highly sensitive personal information. Blanket masking alone can destroy product utility while still failing to define who may access unmasked data, for what purpose, for how long, and with what audit evidence. + +## Drivers + +- useful user-controlled personal data; +- least privilege and tenant separation; +- explicit purpose limitation; +- short-lived/single-use privileged access; +- auditable decisions without copying sensitive payloads into logs; +- fail-closed authorization and replay behavior. + +## Alternatives + +1. Mask all sensitive values globally. +2. Rely only on broad service/account roles. +3. Authorize exact actor/resource/purpose combinations and issue bounded auditable grants where privileged access is required. + +## Decision + +Sensitive-data access uses purpose-bound authorization in addition to authentication and workspace ownership. Privacy-service records append-only decisions/events and, where needed, time-bounded/single-use grants. Encryption and secret-management boundaries protect stored/transport credentials; public logs/errors/artifacts remain content-minimized. + +Masking/redaction is used where disclosure is unnecessary, not as a substitute for authorization. + +## Consequences + +- Privileged flows require explicit purpose/resource policy and evidence. +- Audit facts are retained separately from the sensitive payload whenever possible. +- Product/reporting features can still operate on authorized data rather than degraded masked substitutes. + +## Failure/recovery + +Expired, reused, malformed, wrong-purpose, wrong-actor or wrong-resource grants fail closed. Partial persistence failures roll back/compensate according to the privacy repository contract and never claim access was consumed successfully without durable evidence. + +## Security/privacy impact + +This is a primary privacy/security control. It reduces standing privilege and creates evidence for misuse investigation while minimizing sensitive logging. + +## Acceptance evidence + +Protected-main purpose-bound privacy service, signed context/grant tests, PostgreSQL integration tests and append-only mutation constraints merged in PR #124. + +## Migration/rollback + +New sensitive domains map operations to explicit purpose codes and policy before enabling privileged access. Disabling a purpose/grant path must not delete historical decision/event evidence. + +## Supersession + +Requires an access-control model that provides at least equivalent tenant, purpose, lifetime, replay, audit and recovery guarantees. diff --git a/docs/adr/0006-work-conserving-autonomous-maintenance.md b/docs/adr/0006-work-conserving-autonomous-maintenance.md new file mode 100644 index 00000000..7264f9ae --- /dev/null +++ b/docs/adr/0006-work-conserving-autonomous-maintenance.md @@ -0,0 +1,60 @@ +# ADR-0006: Work-conserving autonomous maintenance + +**Status:** Accepted architecture +**Date:** 2026-08-09 + +## Context + +LifeOS uses automated commercial-readiness/review/development loops. A naive automation loop can waste an entire invocation describing one queued check, failed mutation path, missing CLI, reviewer delay, or writer conflict even when other safe work exists. + +## Drivers + +- maximize validated repository/product progress per finite run; +- avoid concurrent branch writers; +- preserve exact-head/base evidence; +- distinguish a blocked action from a blocked repository; +- avoid routine status narration as an output substitute; +- continue buyer-visible development after PR queues drain. + +## Alternatives + +1. Stop the whole run at the first blocker. +2. Retry one blocker indefinitely. +3. Maintain a live executable queue, localize blockers, preserve a writer lease, and continue non-conflicting work. + +## Decision + +Autonomous maintenance is work-conserving. Each run refetches live PR/head/base/review/check state, performs evidence-backed RCA, verifies remedy feasibility, executes the highest-value safe action, and immediately selects the next action while run budget remains. + +A writer lease prevents races on a branch, but a conflict freezes only that target. Queued checks, provider latency, missing local tools, review delay or a failed first remedy do not by themselves terminate the run or disable the hourly scheduler. + +A successful commit, documentation update, PR creation, merge, or check dispatch is an intermediate result rather than automatic completion. Documentation completion must transition into the next executable implementation, validation, merge, cleanup, operability, or buyer-gap action when one exists. + +## Consequences + +- Runs may combine one coherent source-write root cause with additional read-only/review/merge/cleanup/product-preparation work. +- Operational state must be refetched rather than trusted from memory or resolved review threads. +- User-visible notification becomes exceptional and action-oriented. +- Scheduler prompts are updated when a repeated premature-stop or semantic-documentation-regression pattern is discovered. + +## Failure/recovery + +If another writer moves the same target, discard stale assumptions, reconcile read-only, and continue elsewhere. If one mutation mechanism is unavailable, probe alternative connector/Git/ref/artifact/workflow paths before declaring user action necessary. + +If a previously resolved review defect reappears in the exact current source, current source wins over historical review state: restore a regression test, fix the current defect, and revalidate the new head. + +## Security/governance impact + +Exact-head/base/blob guards, no fabricated approval/evidence, branch-protection compliance and bounded permissions remain mandatory. Work-conserving execution increases activity, not authority. + +## Acceptance evidence + +Protected-main `AGENTS.md` waiting/escalation rules and commercial-readiness loop define the repository policy. PR #122 is now protected-main evidence as squash commit `876850018a17323900844e79845ba395b7bf6a9a`, adding the bounded OpenCode commercial-development loop while retaining ordinary exact-head CI/security/review/merge authority. The enabled external LifeOS hourly maintainer extends this policy and must remain a single non-duplicative writer lease. + +## Migration/rollback + +Automation may be paused/removed without changing product data. Duplicate writer loops are consolidated rather than stacked. A failed scheduler mechanism falls back to another safe audited maintenance path without weakening merge gates. + +## Supersession + +Supersede only if a different scheduling/concurrency model provides equal or better exact-head safety, throughput, observability, work-conserving continuation, and protection against documentation/status drift. diff --git a/docs/adr/0007-canonical-documentation-graph.md b/docs/adr/0007-canonical-documentation-graph.md new file mode 100644 index 00000000..da070f88 --- /dev/null +++ b/docs/adr/0007-canonical-documentation-graph.md @@ -0,0 +1,92 @@ +# ADR-0007: Canonical documentation graph with code-current status + +**Status:** Accepted architecture +**Date:** 2026-08-09 + +## Context + +LifeOS accumulated valuable feature specifications, plans, research notes and runbooks, but the whole product could not be reconstructed without reconciling a stale initial design, protected-main source, capability evidence, changelog entries, old PRs and conversation history. Several canonical document families were absent. + +A later failure mode showed that even a previously resolved review thread is insufficient proof of current correctness: subsequent branch movement can reintroduce invalid composite status values or weaken documentation-contract tests. Therefore exact-current source, not review history or file presence, is the documentation authority. + +## Drivers + +- acquirer/operator/contributor comprehension from the repository alone; +- explicit separation of shipped versus planned behavior; +- traceability from requirement/decision to code/test/runbook; +- preservation of superseded rationale without parallel sources of truth; +- machine-checkable semantic documentation consistency; +- exact-current-head verification after every documentation mutation; +- documentation findings that feed implementation rather than ending work. + +## Alternatives + +1. Keep feature plans/specs as the only documentation. +2. Replace all historical docs with one monolithic document. +3. Check only that canonical document files exist. +4. Maintain a small canonical documentation spine that indexes scoped evidence, records exact status/supersession explicitly, and validates key claims against current source/configuration/migrations. + +## Decision + +LifeOS maintains: + +- `docs/PRD.md`; +- `docs/TRD.md`; +- root `ARCHITECTURE.md`; +- `docs/adr/README.md` and material ADRs; +- `docs/DATA_MODEL.md`; +- `docs/UML.md`; +- `docs/API_CONTRACTS.md`; +- `SECURITY.md` and `docs/THREAT_MODEL.md`; +- `docs/PRIVACY_DATA_LIFECYCLE.md`; +- `docs/TEST_STRATEGY.md`; +- `docs/OPERABILITY.md`; +- `docs/RELEASE_AND_MIGRATION.md`; +- `docs/STANDARDS_TRACEABILITY.md`; +- `docs/TRACEABILITY.md`; +- `docs/DOCUMENTATION_ASSESSMENT.md`; +- scoped research, runbooks, legal docs, specs/plans, capability manifest and changelog as supporting evidence. + +Canonical documents and ADRs use only these exact status categories: `Implemented on protected main`, `Implemented on active PR`, `Partial`, `Accepted architecture`, `Planned`, `Research only`, `Superseded`, and `Out of scope`. PR numbers, scope qualifiers such as `reference`, implementation notes and other evidence belong outside the status value. + +Machine-checkable documentation tests validate more than presence. They verify canonical local link targets, exact ADR index targets and material ADR set, exact status vocabulary, balanced fences, key service/data/AI authority claims against real repository evidence, and live buyer-gap/active-PR traceability that would otherwise silently drift. + +A resolved review comment is historical evidence only. If current exact source contradicts the accepted contract, repair the current source and regression test even if the old thread remains resolved. + +Historical files remain available but are marked/indexed as historical when superseded. + +## Consequences + +- Documentation changes require evidence review, not merely prose review. +- Product status can be read without treating roadmaps or active PRs as shipped behavior. +- Architecture drift such as UUID version, service ownership, active buyer-gap implementation, or readiness semantics is visible sooner. +- API/event, privacy/data lifecycle, migration/release, and standards/research claims have explicit canonical indexes instead of being inferred from unrelated feature prose. +- The canonical docs stay concise by linking scoped evidence rather than copying every feature spec. +- Documentation tests may intentionally fail when current docs lag a new active PR; that failure is a synchronization defect, not a reason to weaken the contract. + +## Failure/recovery + +If docs disagree with protected-main code/tests, the implemented claim is downgraded/corrected immediately; documentation is not used to override behavior. If docs lag a current active implementation, mark it `Implemented on active PR` with exact PR evidence rather than prematurely upgrading protected-main status. + +If a doc reveals a genuine implementation gap, the maintenance loop creates/advances the smallest executable product/test/migration change rather than declaring completion at the audit. If a later commit reintroduces a documentation defect, current-source revalidation reopens the causal repair regardless of historical thread resolution. + +## Security/privacy impact + +Canonical docs must not embed credentials, private user data, exploit details that belong in private security reports, raw model traces or unnecessary sensitive examples. Threat/security/privacy lifecycle boundaries become easier to audit. Link and evidence validation must not fetch arbitrary external content as part of a deterministic merge gate. + +## Acceptance evidence + +- presence and review of the canonical files; +- real local links from README/architecture/agent contracts; +- status/evidence mapping in `docs/TRACEABILITY.md`; +- exact-current-head machine-checkable documentation consistency tests; +- source/configuration/migration evidence for key architecture claims; +- active PRs and canonical buyer gaps represented without being promoted to protected-main implementation. + +## Migration/rollback + +No product-data migration is required. Existing specs/plans are retained. Incorrect canonical content can be reverted independently while source/test truth remains intact. Documentation-contract strengthening is additive; if a new check is wrong, correct the check and its documented contract rather than weakening unrelated product gates. + +## Supersession + +Supersede if the repository adopts another documentation system that preserves equivalent authority, exact status vocabulary or an explicitly migrated replacement, API/data/privacy/release/standards traceability, historical rationale, active-PR/protected-main distinction, and machine-verifiable semantic consistency. diff --git a/docs/adr/0008-separate-capability-maturity-from-buyer-gap-exhaustion.md b/docs/adr/0008-separate-capability-maturity-from-buyer-gap-exhaustion.md new file mode 100644 index 00000000..10de4761 --- /dev/null +++ b/docs/adr/0008-separate-capability-maturity-from-buyer-gap-exhaustion.md @@ -0,0 +1,108 @@ +# ADR-0008: Separate capability maturity from buyer-gap exhaustion + +**Status:** Implemented on protected main + +## Context + +LifeOS commercial-readiness evidence historically summarized configured capability maturity and rendered unresolved gaps from the same configured capability set. That is useful for determining whether a registered capability has its required files, tests, controls, and evidence, but it is not equivalent to proving that the whole product has no remaining buyer-visible gaps. + +Protected-main evidence can therefore report every configured capability at target while accepted customer journeys remain incomplete. End-to-end tenant export/deletion (#55), durable multi-device Today synchronization (#121), hosted per-user calendar credentials (#129), and generic plugin runtime delivery (#130) remain distinct product gaps even when their related core capability evidence is mature. + +Issue #128 recorded this semantic defect. PR #131 implemented the repository-owned buyer-gap registry and explicit issue-state reconciliation and was squash-merged to protected main as `2ad45a935283e83d9bb0f3ea5c3d23b2669078b1` on 2026-08-09; issue #128 is closed as completed. The implementation therefore belongs to protected-main evidence, while the registered buyer gaps remain independently open until their own product outcomes are integrated. + +## Drivers + +- Commercial-readiness reporting must not convert configured evidence completeness into a claim of whole-product completeness. +- Buyer-gap identity must be deterministic and reviewable. +- Arbitrary issue, review, model, or comment prose is untrusted data and cannot become executable product policy. +- Missing or ambiguous live issue evidence must fail closed rather than silently resolve a gap. +- Release and autonomous-development decisions need separate views of capability maturity and remaining customer/operator outcomes. + +## Alternatives + +### Keep one scalar capability score + +Rejected. A single configured score is compact but loses the distinction between mature foundations and incomplete end-to-end journeys. + +### Treat every open GitHub issue as a buyer gap + +Rejected. Issue text and labels are mutable, heterogeneous, and may represent bugs, chores, research, duplicates, or untrusted content. This would make arbitrary repository prose executable governance input. + +### Maintain buyer gaps manually only in prose + +Rejected. A prose-only ledger is hard to validate, can drift from live issue state, and is unsuitable for deterministic automation. + +### Repository-owned gap registry plus bounded live state reconciliation + +Selected. The repository owns stable gap identifiers and their intended issue/capability linkage; automation consumes only bounded issue state needed to decide `open`, `resolved`, or `unknown`. + +## Decision + +LifeOS maintains **two independent readiness dimensions**: + +1. **Configured capability-evidence maturity** — whether each registered capability has reached its evidence target under the existing capability contract. +2. **Canonical buyer-gap state** — whether repository-owned durable product/operator gaps are open, resolved, or unknown according to a versioned gap registry and bounded live evidence. + +The two dimensions may be correlated but are never interchangeable. A 100% capability maturity result must not imply zero buyer gaps. + +Canonical buyer-gap identity is defined in repository-owned versioned data, not by arbitrary issue title/body/comment/review/model text. Live GitHub issue state may reconcile that identity but cannot redefine it. + +If a registered gap cannot be fetched or its state is ambiguous, the report records `unknown`; it does not silently treat the gap as resolved. Closing a gap in product governance requires the expected issue state plus code/test/operational evidence appropriate to the linked PRD requirement. + +Release readiness consumes both dimensions together with exact protected-head CI, security, review, migration, backup/recovery, accessibility, packaging/provenance, and operator evidence. + +## Consequences + +### Positive + +- Commercial-readiness reports distinguish mature capability evidence from remaining buyer journeys. +- Autonomous development can prioritize explicit product gaps without scraping arbitrary issue prose into policy. +- Unknown evidence remains visible instead of being converted into optimistic success. +- Capability evidence remains stable and useful without being overloaded into a whole-product completeness metric. + +### Trade-offs + +- Readiness output contains more than one headline dimension and therefore requires clearer operator/product interpretation. +- The buyer-gap registry is another versioned governance artifact that must be reviewed when gaps are created, superseded, split, or resolved. +- Gap closure requires evidence reconciliation rather than simply closing an issue or raising a capability score. + +## Failure/recovery + +If the buyer-gap registry is malformed, duplicated, refers to unknown capabilities, exceeds bounded limits, or cannot be reconciled safely, readiness evaluation fails closed for the buyer-gap dimension. + +If the GitHub issue-state dependency is temporarily unavailable, affected gap states become `unknown`; deterministic capability evidence continues to be computed independently. A later run may reconcile the same stable gap identity after live evidence returns. + +If a gap is split or superseded, update the registry and traceability in a reviewed change. Do not rewrite historical readiness artifacts in place. + +## Security/privacy impact + +- Issue bodies, comments, review text, and model output remain untrusted and non-executable. +- The live collector requests only bounded state required for registered gaps and does not retain arbitrary issue bodies as policy. +- Repository-owned gap IDs and capability links are reviewable source artifacts. +- Unknown/fetch-failure behavior is fail-closed and cannot manufacture a zero-gap result. +- This ADR changes readiness interpretation, not branch protection, merge authority, reviewer identity, or product-data authorization. + +## Acceptance evidence + +Protected main at `2ad45a935283e83d9bb0f3ea5c3d23b2669078b1` includes: + +- versioned `product/buyer-gaps.json` policy data; +- rejection of malformed IDs, duplicate gap ownership, unknown capability references, duplicate capability links, excessive collections, and malformed snapshots; +- deterministic `open`, `resolved`, and `unknown` reconciliation; +- fetch failure and ambiguous evidence becoming `unknown` rather than `resolved`; +- no raw issue body/comment/review text retained as executable gap policy; +- backward-compatible capability maturity as its own dimension; +- separate report fields for capability-evidence gaps and buyer-gap/unknown state; +- exact-PR-head checkout binding in commercial-readiness CI. + +PR #131 passed its applicable CI, AppGuardrail, Semgrep, Security Scan, Commercial Readiness, current CodeRabbit status, and resolved-thread gates before the guarded squash merge used in this run. Exact protected-main source remains authoritative over this summary. + +## Migration/rollback + +The implementation is additive to the existing capability evidence model. Existing capability identifiers and maturity targets keep their previous meaning. + +If the buyer-gap reporting path must be rolled back, retain the capability calculation and repository gap registry/history so no historical product-gap lineage is lost. Do not reinterpret historical 100% capability reports as historical proof of zero buyer gaps. + +## Supersession + +Supersede this ADR only with a reviewed decision that provides an equally deterministic separation between configured capability evidence and whole-product/customer-outcome completeness, preserves untrusted-text boundaries, defines fail-closed unknown semantics, and includes migration of existing gap identity/history. diff --git a/docs/adr/0009-product-hosting-and-data-evolution.md b/docs/adr/0009-product-hosting-and-data-evolution.md new file mode 100644 index 00000000..e0daaeb3 --- /dev/null +++ b/docs/adr/0009-product-hosting-and-data-evolution.md @@ -0,0 +1,63 @@ +# ADR-0009: Product hosting and data architecture evolution + +**Status:** Accepted architecture +**Date:** 2026-08-09 + +## Context + +LifeOS exploration initially considered a private/login-free local-first PWA whose primary store was browser IndexedDB. A later option considered a single Docker application. The product subsequently became a public multi-user system with Google/GitHub authentication, account/workspace isolation, PostgreSQL durability, cross-device use, and independently bounded services. + +Keeping all three models as apparently current creates contradictory privacy, synchronization, deployment and ownership assumptions. + +## Drivers + +- durable cross-device user state; +- explicit tenant/workspace isolation; +- self-hostability without provider lock-in; +- independent service ownership and testability; +- auditable AI/integration/privacy boundaries; +- local/PWA usability without making browser storage the hidden source of truth. + +## Alternatives + +1. **Browser local-first only.** Lowest server/privacy burden, but no authoritative multi-device synchronization or server-side integration lifecycle. +2. **One monolithic Docker application.** Simple deployment, but weakens independent bounded-context ownership and makes future service isolation harder. +3. **Domain-oriented modular MSA with Compose/Kubernetes composition.** More explicit operations, but preserves portable deployment and independent authority. + +## Decision + +LifeOS is a **multi-user, server-backed, self-hostable modular MSA product**. PostgreSQL-backed bounded services own durable user/domain state. The web/PWA may keep local drafts, caches and offline state, but local state is explicitly labeled and becomes durable only through authorized service contracts. + +Docker Compose is a supported composition/development profile. It does not collapse service ownership. The Kubernetes artifacts are a provider-neutral reference rather than a managed-cloud product claim. + +## Consequences + +- Cross-device experiences have an authoritative server-side state boundary. +- Browser-local data requires explicit migration/reconciliation semantics. +- Operators own deployment infrastructure/secrets and legal/retention obligations for their deployment. +- Service contracts and migrations are first-class product interfaces. +- Local-only/private usage remains possible through self-hosting and local deployment, but is not a separate incompatible product architecture. + +## Failure and recovery + +If a server dependency is unavailable, the PWA may preserve an explicit local draft but must not claim it is synchronized. Recovery revalidates current server state and resolves conflict explicitly rather than overwriting silently. + +## Security and privacy impact + +Server persistence increases operator responsibility and attack surface, so tenant isolation, least privilege, purpose-bound access, encrypted secrets, export/deletion controls and private audit evidence are mandatory. It also enables stronger centralized access control and durable audit than unmanaged browser-only state. + +## Acceptance evidence + +- protected-main README describes multi-user self-hostable SaaS; +- gateway/domain services and PostgreSQL persistence exist; +- identity OAuth/session and tenant isolation tests exist; +- Compose and Kubernetes reference paths exist; +- web code distinguishes local Today drafts from durable records. + +## Migration / rollback + +Historical local-first and single-app proposals require no data migration because they were design alternatives, not the protected-main system of record. Any actual future browser-local import must remain explicit and idempotent. + +## Supersession + +Supersede this ADR only if LifeOS intentionally changes its primary authority model (for example to local-first CRDT sync or a hosted single-process architecture) with migration, offline, security, tenancy and rollback evidence. diff --git a/docs/adr/README.md b/docs/adr/README.md new file mode 100644 index 00000000..d8fde73e --- /dev/null +++ b/docs/adr/README.md @@ -0,0 +1,56 @@ +# LifeOS Architecture Decision Records + +ADRs capture durable decisions whose rationale would otherwise be reconstructed from old chat, PR bodies, feature plans, or code archaeology. Protected-main code and tests remain the implementation evidence. + +## Status meanings + +ADR status uses the same exact vocabulary as the canonical product documentation: + +- **Implemented on protected main** — the decision and its required implementation evidence are present on protected main. +- **Implemented on active PR** — implementation exists on a current PR but is not protected-main evidence. +- **Partial** — important parts exist, but the complete decision contract is not implemented. +- **Accepted architecture** — current reviewed architecture/governance decision; implementation may be partial or pending. +- **Planned** — accepted work direction without an implementation claim. +- **Research only** — exploratory evidence outside the production contract. +- **Superseded** — replaced by a later decision and retained for rationale. +- **Out of scope** — intentionally excluded from the current product contract. + +Qualifiers, implementation evidence, PR numbers, and scope notes belong in ADR prose rather than being appended to the status value. + +## ADR identity + +The repository contained historical ADR numbers before this canonical index was introduced. In particular, two durable records already carry the `0002` prefix: the protected-main OAuth/session decision and the later canonical UUIDv4 decision. Renumbering an accepted historical decision would break provenance, so the **full filename is the stable ADR identity**. The numeric prefix remains an ordering aid only and is not assumed globally unique across pre-baseline history. + +New ADRs must use an unused numeric prefix. Existing collisions are indexed with unambiguous filename-derived labels and must not be silently dropped or rewritten. + +## Index + +| ADR identity | Status | Decision | +| --- | --- | --- | +| [0001-opaque-non-numeric-identifiers](0001-opaque-non-numeric-identifiers.md) | Superseded | Historical UUIDv4/non-numeric identifier decision, retained and superseded by the canonical UUIDv4 formulation | +| [0002-oauth-transactions-and-session-tokens](0002-oauth-transactions-and-session-tokens.md) | Implemented on protected main | OAuth anti-replay, PKCE/OIDC transaction binding, provider separation, and revocable workspace-scoped session-token handling | +| [0002-internal-identifiers-uuidv4](0002-internal-identifiers-uuidv4.md) | Accepted architecture | Internal identifiers are opaque UUIDv4; old UUIDv7 design language is superseded | +| [0003-domain-oriented-service-data-ownership](0003-domain-oriented-service-data-ownership.md) | Accepted architecture | Domain services own persistence and communicate through versioned contracts, never cross-service table access | +| [0004-inert-auditable-ai-proposals](0004-inert-auditable-ai-proposals.md) | Accepted architecture | AI output is an inert auditable proposal; deterministic/user authority remains separate | +| [0005-purpose-bound-sensitive-data-access](0005-purpose-bound-sensitive-data-access.md) | Accepted architecture | Sensitive data uses purpose-bound authorization and auditable grants rather than blanket masking | +| [0006-work-conserving-autonomous-maintenance](0006-work-conserving-autonomous-maintenance.md) | Accepted architecture | Autonomous maintenance is exact-head, work-conserving, and blocker-local | +| [0007-canonical-documentation-graph](0007-canonical-documentation-graph.md) | Accepted architecture | Canonical docs require explicit as-built/planned/superseded status and code/test traceability | +| [0008-separate-capability-maturity-from-buyer-gap-exhaustion](0008-separate-capability-maturity-from-buyer-gap-exhaustion.md) | Implemented on protected main | Configured capability maturity and canonical buyer-gap exhaustion are independent readiness dimensions | +| [0009-product-hosting-and-data-evolution](0009-product-hosting-and-data-evolution.md) | Accepted architecture | Multi-user server-backed, self-hostable LifeOS supersedes local-first-only and single-app primary architectures | + +## ADR quality contract + +Every material ADR, including historical records retained in the canonical index, includes: + +1. context and problem; +2. decision drivers; +3. considered alternatives; +4. decision and scope; +5. consequences/trade-offs; +6. failure and recovery behavior; +7. security/privacy/governance impact; +8. acceptance/test evidence; +9. migration/rollback path; +10. supersession conditions. + +A feature plan is not a substitute for an ADR when the decision affects repository-wide authority, ownership, identity, persistence, security, deployment, interoperability, governance interpretation, or release criteria. diff --git a/docs/superpowers/plans/2026-08-09-canonical-documentation-baseline.md b/docs/superpowers/plans/2026-08-09-canonical-documentation-baseline.md new file mode 100644 index 00000000..cba8e525 --- /dev/null +++ b/docs/superpowers/plans/2026-08-09-canonical-documentation-baseline.md @@ -0,0 +1,143 @@ +# Canonical Product Architecture Baseline Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox syntax for tracking. + +**Goal:** Make LifeOS product, technical, architectural, data, security, privacy, operational, API, release, research, and decision boundaries reconstructable from the repository without relying on chat history or scattered feature plans. + +**Architecture:** Keep root `ARCHITECTURE.md` as the durable system boundary, add a canonical documentation graph under `docs/`, and add an ADR index with explicit supersession history. Every document distinguishes protected-main behavior from active-PR, accepted architecture, planned, superseded, research-only, and out-of-scope material. + +**Tech Stack:** Markdown, Mermaid diagram-as-code, existing TypeScript/NestJS/Next.js/PostgreSQL/NATS contracts, GitHub documentation links, Node built-in tests for documentation consistency. + +## Global Constraints + +- Protected-main behavior is authoritative for claims marked implemented. +- Internal identifiers are opaque UUIDv4 strings on current protected main. +- Services never read or mutate another service's database tables directly. +- AI proposals are inert until an explicitly authorized domain path acts on them. +- Database object names are descriptive multiword `snake_case` unless an external protocol requires otherwise. +- Do not invent persistence, SLO values, provider capabilities, certifications, or release status. +- Historical conversation/design choices are recorded as superseded rather than silently deleted. + +--- + +### Task 1: Audit and canonical documentation spine + +**Files:** +- Create: `docs/DOCUMENTATION_ASSESSMENT.md` +- Create: `docs/PRD.md` +- Create: `docs/TRD.md` + +**Interfaces:** +- Consumes: protected-main `README.md`, `ARCHITECTURE.md`, `AGENTS.md`, `CLAUDE.md`, `CHANGELOG.md`, `product/capabilities.json`, existing feature specs/runbooks. +- Produces: canonical product requirements and technical requirements referenced by all later documentation. + +- [x] Verify claimed implemented capabilities against protected-main source/evidence and downgrade unsupported whole-journey claims. +- [x] Record missing canonical document families and contradictory historical choices. +- [x] Write PRD requirements with explicit status and representative evidence. +- [x] Write TRD service, protocol, security, concurrency, degraded-mode, deployment, and release requirements. +- [x] Reconcile the documents after PR #122 merged so the OpenCode loop is protected-main rather than active-PR evidence. + +### Task 2: Data model and UML views + +**Files:** +- Create: `docs/DATA_MODEL.md` +- Create: `docs/UML.md` + +**Interfaces:** +- Consumes: PRD/TRD, protected-main migrations and service ownership. +- Produces: conceptual/logical entity model and code-current component/sequence/state/deployment diagrams. + +- [x] Model service-owned entities and explicitly label persisted, projection, logical, partial, and planned records. +- [x] Prohibit cross-service database foreign-key/table-access coupling in the logical model. +- [x] Verify the protected-main Planning migration and present only Goal → Project → Task as current persisted hierarchy; keep milestone/task-dependency concepts planned until implemented. +- [x] Model a future task dependency as exactly one predecessor plus one successor task per dependency record. +- [x] Add identity/workspace, Today/planning, habit, review, notification/calendar, AI decision, privacy access, backup/deployment flows. +- [x] Correct NATS event direction and make per-service PostgreSQL role/schema ownership explicit in UML/deployment diagrams. +- [x] Add degraded/failure-state diagrams where behavior is materially different. + +### Task 3: Architecture decisions and historical supersession + +**Files:** +- Create: `docs/adr/README.md` +- Create: `docs/adr/0001-product-hosting-and-data-evolution.md` +- Create: `docs/adr/0002-internal-identifiers-uuidv4.md` +- Create: `docs/adr/0003-domain-oriented-service-data-ownership.md` +- Create: `docs/adr/0004-inert-auditable-ai-proposals.md` +- Create: `docs/adr/0005-purpose-bound-sensitive-data-access.md` +- Create: `docs/adr/0006-work-conserving-autonomous-maintenance.md` +- Create: `docs/adr/0007-canonical-documentation-graph.md` + +**Interfaces:** +- Consumes: historical design choices plus protected-main implementation. +- Produces: status-bearing decisions with alternatives, consequences, recovery, acceptance, migration, rollback, and supersession rules. + +- [x] Record local-first/private and single-Docker ideas as superseded by current multi-user/self-hostable modular MSA. +- [x] Explicitly supersede the old UUIDv7 design statement with the current UUIDv4 invariant. +- [x] Capture service-owned persistence, AI authority, privacy access, autonomous-loop, and canonical-documentation decisions. +- [x] Index every material ADR with status and scope. + +### Task 4: Security, privacy, contracts, testing, operations, release, research, and traceability + +**Files:** +- Create: `docs/API_CONTRACTS.md` +- Create: `docs/THREAT_MODEL.md` +- Create: `docs/PRIVACY_DATA_LIFECYCLE.md` +- Create: `docs/TEST_STRATEGY.md` +- Create: `docs/OPERABILITY.md` +- Create: `docs/RELEASE_AND_MIGRATION.md` +- Create: `docs/STANDARDS_TRACEABILITY.md` +- Create: `docs/TRACEABILITY.md` + +**Interfaces:** +- Consumes: current security policy, service/API/event source, privacy/data-rights code, runbooks, CI, capability manifest, migrations, and research specs. +- Produces: API/event ownership, threat/privacy, quality, operator, release/migration, research and requirement-to-evidence contracts. + +- [x] Enumerate assets, trust boundaries, threats, mitigations, residual risks, and incident evidence. +- [x] Separate deterministic merge gates from bounded live-provider conformance. +- [x] Document Compose and Kubernetes reference deployment boundaries without inventing infrastructure ownership or SLA/RPO/RTO values. +- [x] Define API/event/provider contract ownership/evolution without duplicating exact source schemas. +- [x] Define sensitive-data lifecycle, purpose-bound access, retention/export/erasure/backup boundaries and keep incomplete data-rights UX explicitly Partial. +- [x] Define release/migration/rollback rules without implying automatic database/external-side-effect reversibility. +- [x] Classify normative standards, authoritative guidance, peer-reviewed research, preprints/reports and repository experiments. +- [x] Map representative PRD requirements to capabilities, modules, tests, runbooks, active PRs, and live gaps. +- [x] Prove commercial-readiness configured maturity does not imply whole-product gap exhaustion; create issue #128 for the audit defect. +- [x] Split completed CalDAV issue #51 from hosted credential lifecycle issue #129. +- [x] Register plugin runtime installation/secrets/SSRF-safe delivery gap as issue #130. + +### Task 5: Root documentation alignment and executable contract + +**Files:** +- Modify: `ARCHITECTURE.md` +- Modify: `README.md` +- Modify: `AGENTS.md` +- Modify: `CLAUDE.md` +- Modify: `CHANGELOG.md` +- Create: `packages/commercial-readiness/src/documentation-contract.test.mjs` + +**Interfaces:** +- Consumes: Tasks 1-4. +- Produces: one discoverable documentation entry point, consistent repository-wide instructions and machine-checkable regressions. + +- [x] Add notification/privacy/current automation bounded contexts and status to the architecture topology. +- [x] Link every canonical document from README. +- [x] Mark the 2026-08-02 design as historical rather than current product source of truth. +- [x] Require documentation status/evidence/no-early-stop discipline in agent handoffs. +- [x] Record the documentation baseline under `Unreleased` without changing product version. +- [x] Add deterministic tests for canonical file/index links, ADR status/index, fences, UUIDv4/MSA authority, service-owned persistence, inert AI authority, API/privacy/release/research boundaries, and live gap traceability. + +### Task 6: Verification and continuation + +**Files:** +- Review all changed Markdown/test files. + +**Interfaces:** +- Consumes: complete documentation baseline. +- Produces: reviewable documentation PR and an executable next-gap queue. + +- [x] Open one reviewable PR #126 against `main`. +- [x] Request CodeRabbit review and inspect all published review threads. +- [x] Address CodeRabbit findings for historical-status clarity, task-dependency cardinality/persistence truth, NATS event direction, per-service database ownership, and this plan's completion state. +- [x] Continue beyond documentation by merging gate-clean PR #122, closing superseded issue #119, closing completed #51, and creating #128/#129/#130. +- [ ] Observe exact-current-head PR #126 CI/AppGuardrail/Semgrep/Security Scan/Commercial Readiness completion after the latest fixes. +- [ ] Reconcile any new exact-head review findings and merge #126 only when configured gates accept the unchanged head. +- [ ] After #126 merges, use the maintenance loop to implement the highest-value executable live gap; documentation completion is not the run/scheduler exit condition. diff --git a/packages/commercial-readiness/src/documentation-contract.test.mjs b/packages/commercial-readiness/src/documentation-contract.test.mjs new file mode 100644 index 00000000..996b4347 --- /dev/null +++ b/packages/commercial-readiness/src/documentation-contract.test.mjs @@ -0,0 +1,333 @@ +import { existsSync, readFileSync, readdirSync } from 'node:fs'; +import { dirname, join, relative, resolve, sep } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import test from 'node:test'; +import assert from 'node:assert/strict'; + +const REPOSITORY_ROOT = fileURLToPath(new URL('../../../', import.meta.url)); +const REQUIRED_DOCUMENTS = Object.freeze([ + 'docs/PRD.md', + 'docs/TRD.md', + 'ARCHITECTURE.md', + 'docs/adr/README.md', + 'docs/DATA_MODEL.md', + 'docs/UML.md', + 'docs/API_CONTRACTS.md', + 'SECURITY.md', + 'docs/THREAT_MODEL.md', + 'docs/PRIVACY_DATA_LIFECYCLE.md', + 'docs/TEST_STRATEGY.md', + 'docs/OPERABILITY.md', + 'docs/RELEASE_AND_MIGRATION.md', + 'docs/STANDARDS_TRACEABILITY.md', + 'docs/TRACEABILITY.md', + 'docs/DOCUMENTATION_ASSESSMENT.md', +]); +const CANONICAL_STATUSES = Object.freeze([ + 'Implemented on protected main', + 'Implemented on active PR', + 'Partial', + 'Accepted architecture', + 'Planned', + 'Research only', + 'Superseded', + 'Out of scope', +]); +const REQUIRED_ADR_FILES = Object.freeze([ + '0001-opaque-non-numeric-identifiers.md', + '0002-oauth-transactions-and-session-tokens.md', + '0002-internal-identifiers-uuidv4.md', + '0003-domain-oriented-service-data-ownership.md', + '0004-inert-auditable-ai-proposals.md', + '0005-purpose-bound-sensitive-data-access.md', + '0006-work-conserving-autonomous-maintenance.md', + '0007-canonical-documentation-graph.md', + '0008-separate-capability-maturity-from-buyer-gap-exhaustion.md', + '0009-product-hosting-and-data-evolution.md', +]); +const REQUIRED_ADR_SECTIONS = Object.freeze([ + /^## Context$/mu, + /^## Drivers$/mu, + /^## Alternatives$/mu, + /^## Decision$/mu, + /^## Consequences$/mu, + /^## Failure(?: and |\/)recovery$/mu, + /^## Security(?: and |\/)privacy impact$/mu, + /^## Acceptance evidence$/mu, + /^## Migration(?: \/|\/)rollback$/mu, + /^## Supersession$/mu, +]); + +/** Reads one repository-owned UTF-8 file. */ +function readRepositoryText(relativePath) { + return readFileSync(join(REPOSITORY_ROOT, relativePath), 'utf8'); +} + +/** Counts Markdown code-fence openings and closings irrespective of language tag. */ +function countCodeFences(text) { + return text.split('\n').filter((line) => line.trimStart().startsWith('```')) + .length; +} + +/** Returns local Markdown link targets from one document. */ +function localMarkdownTargets(text) { + return [...text.matchAll(/\[[^\]]+\]\(([^)]+)\)/gu)] + .map((match) => match[1].trim()) + .filter( + (target) => + !target.startsWith('#') && + !/^[a-z][a-z0-9+.-]*:/iu.test(target), + ) + .map((target) => target.split('#', 1)[0].split('?', 1)[0]) + .filter((target) => target.length > 0); +} + +/** Resolves and verifies one local documentation link without allowing repository escape. */ +function assertLocalLinkExists(sourcePath, target) { + const sourceDirectory = dirname(join(REPOSITORY_ROOT, sourcePath)); + const resolvedTarget = resolve(sourceDirectory, target); + const relativeTarget = relative(REPOSITORY_ROOT, resolvedTarget); + assert.ok( + relativeTarget !== '..' && !relativeTarget.startsWith(`..${sep}`), + `${sourcePath} link escapes repository: ${target}`, + ); + assert.equal( + existsSync(resolvedTarget), + true, + `${sourcePath} link target does not exist: ${target}`, + ); +} + +/** Splits a Markdown table row into trimmed cells. */ +function markdownCells(line) { + return line + .slice(1, -1) + .split('|') + .map((cell) => cell.trim()); +} + +/** Asserts PRD requirement table rows use the exact canonical status vocabulary. */ +function assertPrdStatuses(text, sourcePath) { + for (const line of text.split('\n')) { + if (!/^\| PRD-[A-Z0-9-]+ \|/u.test(line)) continue; + const cells = markdownCells(line); + assert.ok( + CANONICAL_STATUSES.includes(cells[2]), + `${sourcePath} has non-canonical status for ${cells[0]}: ${cells[2]}`, + ); + } +} + +/** Asserts traceability requirement rows use the exact canonical status vocabulary. */ +function assertTraceabilityStatuses(text, sourcePath) { + for (const line of text.split('\n')) { + if (!/^\| PRD-[A-Z0-9-]+/u.test(line)) continue; + const cells = markdownCells(line); + assert.ok( + CANONICAL_STATUSES.includes(cells[1]), + `${sourcePath} has non-canonical status for ${cells[0]}: ${cells[1]}`, + ); + } +} + +/** Asserts the API boundary registry's Status column uses canonical values only. */ +function assertApiRegistryStatuses(text, sourcePath) { + const section = text.split('## 3. HTTP bounded-context registry')[1]?.split('## 4.')[0]; + assert.ok(section, `${sourcePath} is missing the HTTP boundary registry`); + for (const line of section.split('\n')) { + if (!line.startsWith('|') || /^\|\s*[-:]+/u.test(line)) continue; + const cells = markdownCells(line); + if (cells[0] === 'Boundary') continue; + assert.ok( + CANONICAL_STATUSES.includes(cells[3]), + `${sourcePath} has non-canonical API status for ${cells[0]}: ${cells[3]}`, + ); + } +} + +/** Asserts every explicit Markdown status field uses one exact canonical value. */ +function assertStatusFields(text, sourcePath) { + for (const match of text.matchAll(/^\*\*Status:\*\* ([^\r\n]+)$/gmu)) { + const status = match[1].trim().replace(/\s{2}$/u, ''); + assert.ok( + CANONICAL_STATUSES.includes(status), + `${sourcePath} has non-canonical status field: ${status}`, + ); + } +} + +test('canonical product and architecture documents remain discoverable', () => { + for (const relativePath of REQUIRED_DOCUMENTS) { + assert.equal( + existsSync(join(REPOSITORY_ROOT, relativePath)), + true, + `missing canonical document: ${relativePath}`, + ); + } +}); + +test('README canonical links resolve to real repository targets', () => { + const readme = readRepositoryText('README.md'); + const targets = new Set(localMarkdownTargets(readme)); + for (const relativePath of REQUIRED_DOCUMENTS) { + assert.ok(targets.has(relativePath), `README does not link ${relativePath}`); + } + for (const target of targets) { + assertLocalLinkExists('README.md', target); + } +}); + +test('canonical Markdown documents keep balanced fenced blocks', () => { + for (const relativePath of REQUIRED_DOCUMENTS) { + const fenceCount = countCodeFences(readRepositoryText(relativePath)); + assert.equal( + fenceCount % 2, + 0, + `${relativePath} has an unbalanced Markdown code fence`, + ); + } +}); + +test('canonical requirement, API, and status fields use one exact vocabulary', () => { + assertPrdStatuses(readRepositoryText('docs/PRD.md'), 'docs/PRD.md'); + assertTraceabilityStatuses( + readRepositoryText('docs/TRACEABILITY.md'), + 'docs/TRACEABILITY.md', + ); + assertApiRegistryStatuses( + readRepositoryText('docs/API_CONTRACTS.md'), + 'docs/API_CONTRACTS.md', + ); + for (const relativePath of [ + 'docs/UML.md', + 'docs/OPERABILITY.md', + 'docs/PRIVACY_DATA_LIFECYCLE.md', + 'docs/adr/README.md', + ]) { + assertStatusFields(readRepositoryText(relativePath), relativePath); + } +}); + +test('ADR index targets every material ADR by stable filename and every ADR uses canonical status', () => { + const adrDirectory = join(REPOSITORY_ROOT, 'docs/adr'); + const adrIndex = readRepositoryText('docs/adr/README.md'); + const indexTargets = new Set(localMarkdownTargets(adrIndex)); + const adrFiles = readdirSync(adrDirectory) + .filter((name) => /^\d{4}-.+\.md$/u.test(name)) + .sort(); + + for (const fileName of REQUIRED_ADR_FILES) { + assert.ok(adrFiles.includes(fileName), `missing material ADR ${fileName}`); + } + + for (const fileName of adrFiles) { + const adr = readRepositoryText(`docs/adr/${fileName}`); + const status = /^\*\*Status:\*\* ([^\r\n]+)$/mu.exec(adr)?.[1]?.trim(); + assert.ok(indexTargets.has(fileName), `${fileName} missing from ADR index`); + assertLocalLinkExists('docs/adr/README.md', fileName); + assert.ok( + status !== undefined && CANONICAL_STATUSES.includes(status), + `${fileName} has unsupported or missing ADR status: ${String(status)}`, + ); + for (const sectionPattern of REQUIRED_ADR_SECTIONS) { + assert.match(adr, sectionPattern, `${fileName} missing required ADR section`); + } + } +}); + +test('current canonical architecture resolves historical identifier and hosting drift', () => { + const architecture = readRepositoryText('ARCHITECTURE.md'); + const assessment = readRepositoryText('docs/DOCUMENTATION_ASSESSMENT.md'); + const prd = readRepositoryText('docs/PRD.md'); + + assert.match(architecture, /opaque UUIDv4/u); + assert.match(architecture, /multi-user, server-backed, self-hostable modular MSA/u); + assert.match(architecture, /superseded/iu); + assert.match(assessment, /UUIDv7 proposal → UUIDv4 protected-main invariant/u); + assert.match(prd, /Superseded: login-free local-first product as primary architecture/u); +}); + +test('canonical architecture claims remain bound to real source/configuration evidence', () => { + for (const evidencePath of [ + 'apps/planning-service/src/postgres-planning-repository.ts', + '.github/workflows/opencode-commercial-development.yml', + 'apps/privacy-service/src', + 'infra/backup', + 'infra/kubernetes', + ]) { + assert.equal( + existsSync(join(REPOSITORY_ROOT, evidencePath)), + true, + `documented evidence path disappeared: ${evidencePath}`, + ); + } + + const architecture = readRepositoryText('ARCHITECTURE.md'); + const dataModel = readRepositoryText('docs/DATA_MODEL.md'); + const threatModel = readRepositoryText('docs/THREAT_MODEL.md'); + const agentWorkflow = readRepositoryText( + '.github/workflows/opencode-commercial-development.yml', + ); + + assert.match(architecture, /never read or mutate another service's database tables directly/u); + assert.match(dataModel, /do \*\*not\*\* authorize cross-service SQL joins/u); + assert.match(architecture, /AI output is an inert proposal/u); + assert.match(threatModel, /AI proposals remain inert and auditable/u); + assert.match(agentWorkflow, /NVIDIA_NIM_API_KEY/u); + assert.doesNotMatch(agentWorkflow, /COPILOT_GITHUB_TOKEN/u); +}); + +test('data and UML views preserve protected-main planning and event-direction truth', () => { + const dataModel = readRepositoryText('docs/DATA_MODEL.md'); + const uml = readRepositoryText('docs/UML.md'); + + assert.match( + dataModel, + /Protected-main Planning migrations currently create only `planning\.goals`, `planning\.projects`, and `planning\.tasks`/u, + ); + assert.match(dataModel, /`milestone_record` — planned\/logical/u); + assert.match(dataModel, /`task_dependency` — planned\/logical/u); + assert.match(uml, /Planning -\. publishes domain events \.-> NATS/u); + assert.match(uml, /NATS -\. delivers reminder\/event inputs \.-> Notification/u); + assert.match(uml, /planning-only DSN\/role\/schema/u); + assert.match(uml, /privacy-only DSN\/role\/schema/u); +}); + +test('canonical contracts keep data lifecycle and release gaps explicit', () => { + const apiContracts = readRepositoryText('docs/API_CONTRACTS.md'); + const privacyLifecycle = readRepositoryText('docs/PRIVACY_DATA_LIFECYCLE.md'); + const releaseContract = readRepositoryText('docs/RELEASE_AND_MIGRATION.md'); + const standards = readRepositoryText('docs/STANDARDS_TRACEABILITY.md'); + + assert.match(apiContracts, /planning\.task\.completed\.v1/u); + assert.match(apiContracts, /issue #129/u); + assert.match(apiContracts, /Durable Today workspace synchronization/u); + assert.match(privacyLifecycle, /\*\*Status:\*\* Partial/u); + assert.match(privacyLifecycle, /\*\*Tracking:\*\* issue `#55`/u); + assert.match(privacyLifecycle, /issue #129/u); + assert.match(releaseContract, /A merged feature is not automatically a release/u); + assert.match(releaseContract, /Application rollback/u); + assert.match(standards, /Normative standard\/specification/u); + assert.match(standards, /Peer-reviewed research/u); +}); + +test('traceability distinguishes protected main, active implementation, and buyer gaps', () => { + const traceability = readRepositoryText('docs/TRACEABILITY.md'); + + assert.match(traceability, /PRD-AI-006 autonomous OpenCode development loop \| Implemented on protected main/u); + assert.match(traceability, /PR #127/u); + assert.match(traceability, /PR #131/u); + assert.match(traceability, /PR #133/u); + assert.match(traceability, /PR #134/u); + assert.match(traceability, /PR #138/u); + assert.match(traceability, /Issue #129/u); + assert.match(traceability, /Issue #130/u); +}); + +test('canonical PRD does not keep merged hardening PRs classified as active', () => { + const prd = readRepositoryText('docs/PRD.md'); + + assert.doesNotMatch(prd, /PR #133 is an active hardening follow-up/u); + assert.doesNotMatch(prd, /PR #134 is the active bounded implementation/u); + assert.match(prd, /PR #138/u); +});