diff --git a/.gitignore b/.gitignore index 764f7142..6e795b0d 100644 --- a/.gitignore +++ b/.gitignore @@ -1,6 +1,5 @@ # --- Private exploratory history (kept local, not published) --- .grill/ -DESIGN.md # The current design, ADR, security, glossary, and agent charter are intended # to be versioned with the open-source implementation. Historical drafts remain diff --git a/DESIGN.md b/DESIGN.md new file mode 100644 index 00000000..1d79b0bb --- /dev/null +++ b/DESIGN.md @@ -0,0 +1,157 @@ +--- +version: alpha +name: ContextEngine Evidence Console +description: A flight-recorder evidence docket: exact, quiet, and built for inspection. +colors: + background: "oklch(0.985 0.004 225)" + foreground: "oklch(0.205 0.012 225)" + muted: "oklch(0.955 0.006 225)" + muted-foreground: "oklch(0.445 0.016 225)" + card: "oklch(0.998 0.002 225)" + card-foreground: "oklch(0.205 0.012 225)" + primary: "oklch(0.43 0.095 195)" + primary-foreground: "oklch(0.985 0.004 225)" + secondary: "oklch(0.935 0.008 225)" + secondary-foreground: "oklch(0.255 0.014 225)" + destructive: "oklch(0.43 0.13 25)" + destructive-foreground: "oklch(0.985 0.004 225)" + border: "oklch(0.875 0.008 225)" + input: "oklch(0.875 0.008 225)" + ring: "oklch(0.43 0.095 195)" +typography: + display: { fontFamily: "IBM Plex Sans, Avenir Next, ui-sans-serif, system-ui, sans-serif", fontSize: 38px, fontWeight: 600, lineHeight: 1.12, letterSpacing: -0.02em } + heading: { fontFamily: "IBM Plex Sans, Avenir Next, ui-sans-serif, system-ui, sans-serif", fontSize: 24px, fontWeight: 600, lineHeight: 1.25, letterSpacing: -0.01em } + body-md: { fontFamily: "IBM Plex Sans, Avenir Next, ui-sans-serif, system-ui, sans-serif", fontSize: 16px, fontWeight: 400, lineHeight: 1.6 } + mono: { fontFamily: "IBM Plex Mono, SFMono-Regular, Consolas, ui-monospace, monospace", fontSize: 13px, fontWeight: 400, lineHeight: 1.55 } +spacing: { base: 4px, gutter: 24px, section: 64px } +rounded: { sm: 3px, md: 5px, lg: 7px, full: 9999px } +breakpoints: { sm: 640px, md: 768px, lg: 1024px, xl: 1280px } +motion: { feedback: 100ms, content: 180ms, easing: "cubic-bezier(0.2, 0, 0, 1)" } +components: + button-primary: { backgroundColor: "{colors.primary}", textColor: "{colors.primary-foreground}", rounded: "{rounded.md}", height: 44px, padding: 20px } + button-primary-hover: { backgroundColor: "{colors.foreground}", textColor: "{colors.primary-foreground}" } + button-primary-active: { backgroundColor: "{colors.foreground}", textColor: "{colors.primary-foreground}" } + button-primary-disabled: { backgroundColor: "{colors.muted}", textColor: "{colors.muted-foreground}" } + button-secondary: { backgroundColor: "{colors.secondary}", textColor: "{colors.secondary-foreground}", rounded: "{rounded.md}", height: 44px, padding: 20px } + input-default: { backgroundColor: "{colors.card}", textColor: "{colors.card-foreground}", rounded: "{rounded.md}", height: 44px, padding: 12px } + input-focus: { backgroundColor: "{colors.card}", textColor: "{colors.card-foreground}" } + input-error: { backgroundColor: "{colors.card}", textColor: "{colors.destructive}" } + evidence-block: { backgroundColor: "{colors.card}", textColor: "{colors.card-foreground}", rounded: "{rounded.lg}", padding: 24px } + evidence-block-open: { backgroundColor: "{colors.muted}", textColor: "{colors.foreground}", rounded: "{rounded.lg}", padding: 24px } + refusal-state: { backgroundColor: "{colors.muted}", textColor: "{colors.foreground}", rounded: "{rounded.lg}", padding: 24px } +--- + +# ContextEngine Evidence Console + +## Overview + +The console feels like a flight-recorder evidence docket: exact labels, bounded +facts, restrained rules, and no decorative urgency. The governing rule is that +security state must be legible without becoming enumerable. **Signature:** the +evidence flip—activating a content Block replaces the reading emphasis with its +Article → Revision → Fragment lineage, single Evidence ref, and Policy Epoch. +Everything else stays conventional and quiet. + +## Colors + +Near-achromatic cool paper and ink carry the interface. Deep teal is reserved for +actions, links, disclosure controls, and focus rings. It never colors a status for +decoration. Refusal uses the same muted surface family as other operational states; +its semantics come from heading, icon/text, and recovery copy, not alarming color. +Dark red is restricted to destructive confirmation copy and field validation. + +## Typography + +IBM Plex Sans or the locally available Avenir Next gives operational labels a +technical but human cadence; system sans fallbacks keep the console network-free. +IBM Plex Mono/SFMono holds opaque refs, digests, generations, scores, and timestamps. +Long content remains in the sans body face at a readable 60–75 character measure. +Use weights 400, 500, and 600 only. + +## Layout + +The desktop shell is a 240px navigation rail beside a fluid work column capped at +1120px. A 4px base and 24px gutters create dense, repeatable rhythm; related facts +align in definition grids rather than nested cards. Pages have one primary work +surface. Provenance expands in normal document flow so no evidence is obscured. + +## Elevation & Depth + +Depth comes from tonal surfaces and one-pixel borders. Resting records have no +floating shadow; an interactive evidence Block may gain a quiet one-pixel inset +rule. No blur, glow, glass, gradient, or layered card stack. + +## Shapes + +Radii stay tight—3px to 7px—to read as an engineered instrument. Pills are allowed +only for genuine closed status values. Tables, forms, and disclosure records share +the same corner family. + +## Components + +- Primary and secondary buttons are at least 44px high; focus uses a two-pixel teal + ring with a background offset. Disabled controls retain explanatory adjacent copy. +- Inputs use persistent top-aligned labels, validate on blur and submit, and preserve + non-secret values after safe failure. +- `evidence-block` renders the authorized content first. Its single disclosure + control opens `evidence-block-open`, which exposes lineage in an ordered definition + list. Missing or inconsistent lineage flags/refuses the Block instead. +- `refusal-state` always names a tenant-safe category and one recovery action. It is + never represented as an empty list, blank table, toast-only error, or spinner. +- Full-page submission uses the browser's native navigation/loading state. The + returned document renders authorized-empty, refusal, or success with stable + headings; mixed per-source health records are the explicit partial state. None + infer facts the public response did not carry. + +## Do's and Don'ts + +- **Do spend boldness in exactly one place: the evidence-flip interaction.** +- Do make every rendered content Block close over exactly one Evidence ref. +- Do distinguish authorized empty from refusal in words and document structure. +- Do reserve teal for interaction and visible focus; keep status facts neutral. +- Do use real Release, source, Article, and ContextPackage facts only. +- Do preserve keyboard order, semantic headings, labels, and 44px touch targets. +- Don't render denied candidates, denied counts, original denied ranks, or score gaps. +- Don't add gradients, glass, glows, fake metrics, decorative charts, or nested cards. +- Don't add an inline policy toggle, automatic publish action, or feedback promotion. +- Don't fetch remote fonts/assets or introduce a frontend runtime/build process. + +## Responsive Behavior + +- Below `lg`: the side rail becomes a wrapped top navigation; page context precedes + all actions; no functionality hides behind hover. +- Below `md`: definition grids and two-column comparisons become one column; tables + become labeled record stacks, never viewport-wide horizontal body scroll. +- Below `sm`: gutters become 16px; primary actions use full available width; evidence + refs wrap safely; every action/disclosure target remains at least 44px. + +## Motion + +Use the 100ms feedback token for hover/focus response and 180ms for evidence reveal. +No page-load choreography, bounce, elastic easing, or attention pulse. Under +`prefers-reduced-motion`, evidence changes instantly and focus remains explicit. + +## Accessibility + +All normal text/background pairs target WCAG AA contrast. Every form control has a +visible label, errors are text plus semantics rather than color alone, and refusal +headings receive focus after a submitted request. Native disclosure semantics and +keyboard activation are the evidence-flip baseline. Status updates use a polite live +region only when enhanced navigation is present. + +## Iconography & Imagery + +Use simple inline line icons only where their meaning is independently labeled. +Operational states prefer text and definition lists. No hero art, stock imagery, +provider logos, emoji navigation, or illustrative dashboards. + +## Iteration Guide + +Change semantic tokens here before changing CSS. Add a component token only when a +new local role cannot be expressed by the existing surface/action/state vocabulary. +Any richer interaction must preserve the native HTML path and public-seam tests. + +## Known Gaps + +The M1 surface has no data-visualization language, bulk-selection pattern, workflow +canvas, or mobile-native navigation; those are intentionally outside issue #130. diff --git a/Makefile b/Makefile index d6fc0078..f4e582fc 100644 --- a/Makefile +++ b/Makefile @@ -1,4 +1,4 @@ -.PHONY: install build lint typecheck test catalog third-party-check third-party-artifacts security-gate smoke db-up db-down db-reset integration dogfood-eval eval-v1 eval-v1-execute openapi-generate openapi-check openapi-breaking-check sdk-generate sdk-check sdk-build sdk-test sdk-pack action-typecheck action-build action-test bot-typecheck bot-build bot-test check +.PHONY: install build lint typecheck test catalog third-party-check third-party-artifacts security-gate smoke db-up db-down db-reset integration dogfood-eval eval-v1 eval-v1-execute openapi-generate openapi-check openapi-breaking-check sdk-generate sdk-check sdk-build sdk-test sdk-pack action-typecheck action-build action-test bot-typecheck bot-build bot-test ui-build ui-test check install: uv sync --frozen @@ -113,4 +113,16 @@ bot-test: bot-build npm --prefix bot_delivery/typescript run test:runtime npm --prefix bot_delivery/typescript run test:package -check: build lint typecheck openapi-check sdk-check sdk-build sdk-test sdk-pack action-build action-test bot-build bot-test test catalog smoke integration security-gate third-party-artifacts +ui-build: + uv run python -m py_compile ui/*.py + +ui-test: ui-build + uv run pytest -q \ + tests/unit/test_ui_uses_public_seam_only.py \ + tests/unit/test_source_overview_matches_promoted_release.py \ + tests/unit/test_citation_lineage_resolvable.py \ + tests/unit/test_profile_change_surfaces_reembed.py \ + tests/unit/test_feedback_has_no_publication_authority.py \ + tests/unit/test_fail_closed_renders_refusal.py + +check: build lint typecheck openapi-check sdk-check sdk-build sdk-test sdk-pack action-build action-test bot-build bot-test ui-build ui-test test catalog smoke integration security-gate third-party-artifacts diff --git a/STATUS.md b/STATUS.md index 0ff3f3df..907399b3 100644 --- a/STATUS.md +++ b/STATUS.md @@ -47,8 +47,11 @@ grants one Control operation per call. ADR-0073 adds one explicit local candidate from the exact current File corpus and still delegates activation only to `ContextLearning.evaluate` and `ContextLearning.promote`. Production operator authentication, multiple operators, durable role assignment, -delegation, RBAC, and every network-reachable operator surface remain -`NOT_ACTIVE`. The following are known, designed, and deliberately not active: +delegation, RBAC, and every production or non-loopback operator surface remain +`NOT_ACTIVE`. ADR-0083 separately activates one loopback-only, server-rendered +Evidence Console inside the API process; its Control-backed jobs require the +same separate exact-operation local Control credential. The following are known, +designed, and deliberately not active: | Capability | Note | |---|---| @@ -58,7 +61,7 @@ delegation, RBAC, and every network-reachable operator surface remain | General / multi-user Source and Resource ACLs | Dogfood uses current mirrored File access plus Membership field rights only; source-native and multi-user authorities remain absent | | General content retrieval | Only the loopback File pgvector dogfood `Acquire` carrier is active | | Local consumer expansion | The repo-local Claude Code skill is active only as a single-question display consumer; pi and MCP remain `NOT_ACTIVE` | -| `Continue` / dogfood `OpenCitation` carriers | The bounded dogfood composition keeps both unavailable; other accepted ADRs own their narrower citation carriers | +| `Continue` carrier | The bounded dogfood composition keeps Continue unavailable; its private File `OpenCitation` carrier is active for UI citation closure | | Federated discovery, source-native authorization | Deterministic refusal only | | Runtime model-inference carriers (`rewrite`, `rerank`, `select`) | The governed port is implemented, but no carrier is active until its consuming issue proves the exact profile and grant path, passes the resolve-owned shared `PackageBudgetMeter`, and publishes that meter's cumulative usage in the final package; real provider network calls remain `NOT_ACTIVE` | | Live Feishu / Slack / Google Docs connectors | See [PLAN.md](./PLAN.md) milestones M4 / M6 / M7 | @@ -80,7 +83,7 @@ delegation, RBAC, and every network-reachable operator surface remain `RUNTIME-DOGFOOD-EPOCH-102` are registered release-veto evidence. The default application remains reject-all and reports `NOT_ACTIVE`. Production authentication, a second human, network exposure beyond the maintainer machine, -group/public audience, dogfood `OpenCitation`, `Continue`, hybrid retrieval, non-File providers, and +group/public audience, `Continue`, hybrid retrieval, non-File providers, and external query embeddings remain `NOT_ACTIVE`. ### Repo-local Claude Code consumer @@ -97,6 +100,16 @@ closed local refusals. Candidate capture contains no corpus path or Package content. This activates no pi consumer, MCP Adapter, `Continue`, dogfood `OpenCitation`, IM delivery, channel egress, or external effect. +### Co-resident local Evidence Console + +| ADR | Activates | +|---|---| +| [0090](./docs/decisions/0090-admit-a-co-resident-local-evidence-console.md) | Explicitly authenticated server-rendered loopback UI, private File citation reopening, and separately Control-authorized source/import/Article jobs through schema-hidden typed HTTP carriers while OpenAPI v0 remains frozen | + +Feedback persists through the current Runtime identity and exact ContextRun +binding, with no Control or release-publication authority. Numeric Hit Test scores +remain unavailable because the public rank-free `ContextPackage` intentionally +contains no pre-authorization rank evidence. ### Bounded local Release promotion | ADR | Activates | diff --git a/adapters/http/app.py b/adapters/http/app.py index 71cece48..30f74b52 100644 --- a/adapters/http/app.py +++ b/adapters/http/app.py @@ -13,6 +13,7 @@ from fastapi.exceptions import RequestValidationError from fastapi.responses import JSONResponse from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer +from pydantic import BaseModel from starlette.exceptions import HTTPException as StarletteHTTPException from adapters.http.authentication import ( @@ -68,7 +69,35 @@ ResolveBodyLimitMiddleware, enforce_json_nesting, ) +from adapters.http.ui_api import ( + ArticlePolicyPreviewWire, + ArticleWire, + FeedbackCapture, + FeedbackWire, + ImportPreviewWire, + PreviewConfirmWire, + RefusingUiApi, + UiApi, + UiApiUnavailable, + UiArticlePolicyConfirmResponse, + UiArticlePolicyPreviewResponse, + UiArticleResponse, + UiFeedbackResponse, + UiImportConfirmResponse, + UiImportPreviewResponse, + UiOverviewResponse, + UiProfilesResponse, + UiSessionResponse, + ui_actor, +) from engine import BUILD_IDENTIFIER +from engine.control import ( + ControlOperation, + ControlOperatorAuthenticationRejected, + ControlOperatorAuthority, + ControlOperatorAuthorityUnavailable, + TrustedControlCall, +) from engine.persistence.membership_context import ( MembershipAuthorityUnavailable, MembershipIdentity, @@ -249,6 +278,9 @@ def create_app( request_id_factory: Callable[[], str] = _new_request_id, transport_profile: HttpTransportProfile = HTTP_TRANSPORT_PROFILE_V1, runtime_delivery_activation: _RuntimeDeliveryActivation | None = None, + ui_bearer_token: str | None = None, + ui_control_authority: ControlOperatorAuthority | None = None, + ui_api: UiApi | None = None, ) -> FastAPI: """Construct API; the module-level composition remains reject-all.""" @@ -277,6 +309,11 @@ def create_app( ) selected_scope_authority = scope_authority or MissingTrustedScopeAuthority() selected_route_policy = route_policy or AllowAuthenticatedResolveRoutePolicy() + selected_ui_api = ui_api or RefusingUiApi() + if ui_control_authority is not None and type(ui_control_authority) is not ( + ControlOperatorAuthority + ): + raise TypeError("UI Control authority has the wrong nominal type") bearer = HTTPBearer( scheme_name="ContextEngineBearer", bearerFormat="opaque", @@ -290,6 +327,56 @@ def create_app( invalid_response=INVALID_REQUEST_RESPONSE, ) + from ui import install_ui + + install_ui(app, bearer_token=ui_bearer_token) + + def execute_ui_control( + *, + credential: str | None, + operation: ControlOperation, + invoke: Callable[[TrustedControlCall], dict[str, object]], + document_type: type[BaseModel], + ) -> JSONResponse: + authority = ui_control_authority + try: + if authority is None or credential is None: + raise ControlOperatorAuthenticationRejected + with authority.authorize( + opaque_credential=credential, + operation=operation, + request_id=f"ui-control-{uuid4().hex}", + ) as call: + raw_document = invoke(call) + if type(raw_document) is not dict: + raise UiApiUnavailable + document = document_type.model_validate(raw_document).model_dump( + mode="json" + ) + except ControlOperatorAuthenticationRejected: + return JSONResponse( + AUTHENTICATION_FAILED_RESPONSE, + status_code=401, + headers={"Cache-Control": "no-store"}, + ) + except (ControlOperatorAuthorityUnavailable, UiApiUnavailable): + return JSONResponse( + SERVICE_UNAVAILABLE_RESPONSE, + status_code=503, + headers={"Cache-Control": "no-store"}, + ) + except Exception: + return JSONResponse( + SERVICE_UNAVAILABLE_RESPONSE, + status_code=503, + headers={"Cache-Control": "no-store"}, + ) + return JSONResponse( + document, + status_code=200, + headers={"Cache-Control": "no-store"}, + ) + @app.exception_handler(TransportAuthenticationFailed) async def authentication_failed( request: Request, @@ -433,6 +520,256 @@ def health() -> dict[str, str]: response["runtime_delivery"] = "ACTIVE" return response + @app.get( + "/v0/ui/session", + include_in_schema=False, + response_model=UiSessionResponse, + ) + def ui_session( + authentication: Annotated[ + VerifiedAuthenticationContext, + Depends(verified_authentication), + ], + ) -> dict[str, str]: + del authentication + return {"state": "active"} + + @app.get( + "/v0/ui/profiles", + include_in_schema=False, + response_model=UiProfilesResponse, + ) + def ui_profiles( + authentication: Annotated[ + VerifiedAuthenticationContext, + Depends(verified_authentication), + ], + ) -> JSONResponse: + try: + raw_document = selected_ui_api.profiles(ui_actor(authentication)) + if type(raw_document) is not dict: + raise UiApiUnavailable + document = UiProfilesResponse.model_validate(raw_document).model_dump( + mode="json" + ) + except Exception: + return JSONResponse( + SERVICE_UNAVAILABLE_RESPONSE, + status_code=503, + headers={"Cache-Control": "no-store"}, + ) + return JSONResponse( + document, + status_code=200, + headers={"Cache-Control": "no-store"}, + ) + + @app.get( + "/v0/ui/overview", + include_in_schema=False, + response_model=UiOverviewResponse, + ) + def ui_overview( + authentication: Annotated[ + VerifiedAuthenticationContext, + Depends(verified_authentication), + ], + control_credential: Annotated[ + str | None, + Header(alias="X-Context-Control-Credential", max_length=4096), + ] = None, + ) -> JSONResponse: + return execute_ui_control( + credential=control_credential, + operation=ControlOperation.READ_SOURCE_PROGRESS, + invoke=lambda call: selected_ui_api.overview( + ui_actor(authentication), call + ), + document_type=UiOverviewResponse, + ) + + @app.post( + "/v0/ui/import/preview", + include_in_schema=False, + response_model=UiImportPreviewResponse, + dependencies=[Depends(require_closed_json_transport)], + ) + def ui_import_preview( + body: Annotated[ImportPreviewWire, Body()], + authentication: Annotated[ + VerifiedAuthenticationContext, + Depends(verified_authentication), + ], + control_credential: Annotated[ + str | None, + Header(alias="X-Context-Control-Credential", max_length=4096), + ] = None, + ) -> JSONResponse: + return execute_ui_control( + credential=control_credential, + operation=ControlOperation.IMPORT_FILE, + invoke=lambda call: selected_ui_api.preview_import( + ui_actor(authentication), + call, + source_ref=body.sourceRef, + path=body.path, + ), + document_type=UiImportPreviewResponse, + ) + + @app.post( + "/v0/ui/import/confirm", + include_in_schema=False, + response_model=UiImportConfirmResponse, + dependencies=[Depends(require_closed_json_transport)], + ) + def ui_import_confirm( + body: Annotated[PreviewConfirmWire, Body()], + authentication: Annotated[ + VerifiedAuthenticationContext, + Depends(verified_authentication), + ], + control_credential: Annotated[ + str | None, + Header(alias="X-Context-Control-Credential", max_length=4096), + ] = None, + ) -> JSONResponse: + return execute_ui_control( + credential=control_credential, + operation=ControlOperation.IMPORT_FILE, + invoke=lambda call: selected_ui_api.confirm_import( + ui_actor(authentication), + call, + preview_token=body.previewToken, + ), + document_type=UiImportConfirmResponse, + ) + + @app.post( + "/v0/ui/articles/view", + include_in_schema=False, + response_model=UiArticleResponse, + dependencies=[Depends(require_closed_json_transport)], + ) + def ui_article_view( + body: Annotated[ArticleWire, Body()], + authentication: Annotated[ + VerifiedAuthenticationContext, + Depends(verified_authentication), + ], + control_credential: Annotated[ + str | None, + Header(alias="X-Context-Control-Credential", max_length=4096), + ] = None, + ) -> JSONResponse: + return execute_ui_control( + credential=control_credential, + operation=ControlOperation.READ_ARTICLE_POLICY, + invoke=lambda call: selected_ui_api.article( + ui_actor(authentication), + call, + resource_ref=body.resourceRef, + ), + document_type=UiArticleResponse, + ) + + @app.post( + "/v0/ui/articles/preview", + include_in_schema=False, + response_model=UiArticlePolicyPreviewResponse, + dependencies=[Depends(require_closed_json_transport)], + ) + def ui_article_policy_preview( + body: Annotated[ArticlePolicyPreviewWire, Body()], + authentication: Annotated[ + VerifiedAuthenticationContext, + Depends(verified_authentication), + ], + control_credential: Annotated[ + str | None, + Header(alias="X-Context-Control-Credential", max_length=4096), + ] = None, + ) -> JSONResponse: + return execute_ui_control( + credential=control_credential, + operation=ControlOperation.CHANGE_ARTICLE_POLICY, + invoke=lambda call: selected_ui_api.preview_article_policy( + ui_actor(authentication), + call, + resource_ref=body.resourceRef, + policy_kind=body.policyKind, + group_refs=tuple(body.groupRefs), + ), + document_type=UiArticlePolicyPreviewResponse, + ) + + @app.post( + "/v0/ui/articles/confirm", + include_in_schema=False, + response_model=UiArticlePolicyConfirmResponse, + dependencies=[Depends(require_closed_json_transport)], + ) + def ui_article_policy_confirm( + body: Annotated[PreviewConfirmWire, Body()], + authentication: Annotated[ + VerifiedAuthenticationContext, + Depends(verified_authentication), + ], + control_credential: Annotated[ + str | None, + Header(alias="X-Context-Control-Credential", max_length=4096), + ] = None, + ) -> JSONResponse: + return execute_ui_control( + credential=control_credential, + operation=ControlOperation.CHANGE_ARTICLE_POLICY, + invoke=lambda call: selected_ui_api.confirm_article_policy( + ui_actor(authentication), + call, + preview_token=body.previewToken, + ), + document_type=UiArticlePolicyConfirmResponse, + ) + + @app.post( + "/v0/ui/feedback", + include_in_schema=False, + response_model=UiFeedbackResponse, + dependencies=[Depends(require_closed_json_transport)], + ) + def ui_feedback( + body: Annotated[FeedbackWire, Body()], + authentication: Annotated[ + VerifiedAuthenticationContext, + Depends(verified_authentication), + ], + ) -> JSONResponse: + try: + raw_document = selected_ui_api.capture_feedback( + ui_actor(authentication), + FeedbackCapture( + run_ref=body.runRef, + rating=body.rating, + note=body.note, + ), + ) + if type(raw_document) is not dict: + raise UiApiUnavailable + document = UiFeedbackResponse.model_validate(raw_document).model_dump( + mode="json" + ) + except Exception: + return JSONResponse( + SERVICE_UNAVAILABLE_RESPONSE, + status_code=503, + headers={"Cache-Control": "no-store"}, + ) + return JSONResponse( + document, + status_code=200, + headers={"Cache-Control": "no-store"}, + ) + @app.post( LEGACY_RESOLVE_PATH, include_in_schema=False, diff --git a/adapters/http/dogfood.py b/adapters/http/dogfood.py index 351127a8..a7dbfd58 100644 --- a/adapters/http/dogfood.py +++ b/adapters/http/dogfood.py @@ -19,7 +19,18 @@ from adapters.http.organization_authority import DogfoodOrganizationAuthority from adapters.http.scope_authority import DogfoodFileScopeAuthority from adapters.pgvector import PostgreSQLVectorCandidateIndex +from applications.file_root_configuration import ( + WORKER_FILE_ROOTS_ENV, + file_roots, +) +from applications.operator_authentication import ( + CONTROL_OPERATOR_SECRET_ENV, + LocalOperatorConfiguration, + LocalOperatorConfigurationUnavailable, +) +from engine.control import MinimalUiControlGate from engine.persistence import ( + DatabaseConfigurationError, DatabasePurpose, PostgreSQLMembershipAuthority, create_database_engine, @@ -31,6 +42,7 @@ MembershipNotCurrent, ) from engine.runtime import Runtime +from engine.runtime.citation import PRIVATE_FILE_CITATION_OPEN_PROFILE from engine.runtime.construction import required_kernel_dependencies from engine.runtime.package_digest import QueryDigestKeyring from engine.runtime.release_lineage import ( @@ -52,8 +64,10 @@ DOGFOOD_BINDING_ENV = "CONTEXT_ENGINE_DOGFOOD_AUTHENTICATION_BINDING_REF" DOGFOOD_EMBEDDING_PROVIDER_ENV = "CONTEXT_ENGINE_DOGFOOD_EMBEDDING_PROVIDER" DOGFOOD_EMBEDDING_PROVIDER_VALUE = "deterministic-twin-v1" +DOGFOOD_FILE_IMPORT_RECEIVER_ENV = "CONTEXT_ENGINE_WORKER_SERVICE_PRINCIPAL_ID" _QUERY_DIGEST_DERIVATION_DOMAIN = b"context-engine.dogfood.query-digest.v1\x00" +_UI_PREVIEW_DERIVATION_DOMAIN = b"context-engine.dogfood.ui-preview.v1\x00" class DogfoodConfigurationUnavailable(ValueError): @@ -104,10 +118,9 @@ def __post_init__(self) -> None: raise DogfoodConfigurationUnavailable( "dogfood API configuration is unavailable" ) - if ( - type(self.membership_version) is not int - or not 1 <= self.membership_version < (1 << 63) - ): + if type( + self.membership_version + ) is not int or not 1 <= self.membership_version < (1 << 63): raise DogfoodConfigurationUnavailable( "dogfood API configuration is unavailable" ) @@ -138,13 +151,9 @@ def load(cls, environment: Mapping[str, str]) -> DogfoodConfiguration: raise ValueError return cls( secret=_required(environment, DOGFOOD_SECRET_ENV), - organization_id=UUID( - _required(environment, DOGFOOD_ORGANIZATION_ENV) - ), + organization_id=UUID(_required(environment, DOGFOOD_ORGANIZATION_ENV)), user_id=UUID(_required(environment, DOGFOOD_USER_ENV)), - membership_id=UUID( - _required(environment, DOGFOOD_MEMBERSHIP_ENV) - ), + membership_id=UUID(_required(environment, DOGFOOD_MEMBERSHIP_ENV)), membership_version=int(membership_version_text), principal_ref=_required(environment, DOGFOOD_PRINCIPAL_ENV), agent_version_ref=_required(environment, DOGFOOD_AGENT_ENV), @@ -208,6 +217,7 @@ def create_dogfood_app( runtime = Runtime( required_kernel_dependencies(), candidate_index=PostgreSQLVectorCandidateIndex(DeterministicEmbeddingTwin()), + citation_profile=PRIVATE_FILE_CITATION_OPEN_PROFILE, query_digest_keyring=configuration.query_digest_keyring(), ) membership_authority = PostgreSQLMembershipAuthority(runtime_engine) @@ -220,9 +230,7 @@ def create_dogfood_app( membership_version=configuration.membership_version, principal_ref=configuration.principal_ref, request_id="dogfood-composition-activation", - authentication_binding_ref=( - configuration.authentication_binding_ref - ), + authentication_binding_ref=(configuration.authentication_binding_ref), checked_at=datetime.now(UTC), ) ) as current_user_actor: @@ -230,8 +238,7 @@ def create_dogfood_app( if ( release is None or release.organization_id != configuration.organization_id - or release.index_profile_ref - != DOGFOOD_VECTOR_INDEX_PROFILE_REF_V1 + or release.index_profile_ref != DOGFOOD_VECTOR_INDEX_PROFILE_REF_V1 or release.index_profile_digest != DOGFOOD_VECTOR_INDEX_PROFILE_DIGEST_V1 or not release.active_revision_refs @@ -253,9 +260,57 @@ def create_dogfood_app( ) from None from adapters.http.app import ( + DIRECT_ACQUIRE_PURPOSE, + DIRECT_CITATION_PURPOSE, _construct_runtime_delivery_activation, create_app, ) + from adapters.http.ui_api import PostgreSQLUiApi + + roots = None + control_engine = None + control_authority = None + control_gate = None + + def ui_clock() -> datetime: + return datetime.now(UTC) + + try: + operator_configuration = ( + LocalOperatorConfiguration.load(environment) + if environment.get(CONTROL_OPERATOR_SECRET_ENV) is not None + else None + ) + receiver_id = None + if operator_configuration is not None: + control_engine = create_database_engine( + load_database_configuration( + DatabasePurpose.CONTROL_PLANE, + environment, + ) + ) + if environment.get(WORKER_FILE_ROOTS_ENV) is not None: + roots = file_roots(environment) + raw_receiver = environment.get(DOGFOOD_FILE_IMPORT_RECEIVER_ENV) + receiver_id = None if raw_receiver is None else UUID(raw_receiver) + control_authority = operator_configuration.authorities( + clock=ui_clock + ).control + control_gate = MinimalUiControlGate(control_authority, clock=ui_clock) + except ( + DatabaseConfigurationError, + LocalOperatorConfigurationUnavailable, + TypeError, + ValueError, + ): + runtime_engine.dispose() + if control_engine is not None: + control_engine.dispose() + if roots is not None: + roots.close() + raise DogfoodConfigurationUnavailable( + "dogfood UI configuration is unavailable" + ) from None app = create_app( authenticator=DogfoodAuthenticator( @@ -270,12 +325,32 @@ def create_dogfood_app( organization_id=configuration.organization_id, principal_ref=configuration.principal_ref, agent_version_ref=configuration.agent_version_ref, - purpose="context.answer", + purposes=frozenset( + {DIRECT_ACQUIRE_PURPOSE, DIRECT_CITATION_PURPOSE} + ), ), runtime=runtime, runtime_delivery_activation=_construct_runtime_delivery_activation(), + ui_bearer_token=configuration.secret, + ui_control_authority=control_authority, + ui_api=PostgreSQLUiApi( + membership_authority, + control_engine, + feedback_engine=runtime_engine, + preview_key=sha256( + _UI_PREVIEW_DERIVATION_DOMAIN + configuration.secret.encode("utf-8") + ).digest(), + control_gate=control_gate, + roots=roots, + file_import_service_principal_id=receiver_id, + clock=ui_clock, + ), ) app.add_event_handler("shutdown", runtime_engine.dispose) + if control_engine is not None: + app.add_event_handler("shutdown", control_engine.dispose) + if roots is not None: + app.add_event_handler("shutdown", roots.close) return app diff --git a/adapters/http/scope_authority.py b/adapters/http/scope_authority.py index 0cfa63f7..2c7a365b 100644 --- a/adapters/http/scope_authority.py +++ b/adapters/http/scope_authority.py @@ -163,13 +163,13 @@ def current_scope( class DogfoodFileScopeAuthority: - """Bound one local dogfood Agent/purpose to current File projection facts.""" + """Bind one local dogfood Agent and closed purposes to File projection facts.""" __slots__ = ( "_agent_version_ref", "_organization_id", "_principal_ref", - "_purpose", + "_purposes", ) def __init__( @@ -178,21 +178,29 @@ def __init__( organization_id: UUID, principal_ref: str, agent_version_ref: str, - purpose: str, + purposes: frozenset[str], ) -> None: if type(organization_id) is not UUID: raise TypeError("dogfood scope Organization must be UUID") for field_name, value in ( ("principal_ref", principal_ref), ("agent_version_ref", agent_version_ref), - ("purpose", purpose), ): if type(value) is not str or not value or value.isspace(): raise ValueError(f"dogfood scope {field_name} must be non-empty") + if ( + type(purposes) is not frozenset + or not purposes + or any( + type(purpose) is not str or not purpose or purpose.isspace() + for purpose in purposes + ) + ): + raise ValueError("dogfood scope purposes must be a non-empty closed set") self._organization_id = organization_id self._principal_ref = principal_ref self._agent_version_ref = agent_version_ref - self._purpose = purpose + self._purposes = purposes @contextmanager def current_scope( @@ -205,7 +213,7 @@ def current_scope( identity.organization_id != self._organization_id or identity.principal_ref != self._principal_ref or identity.agent_version_ref != self._agent_version_ref - or identity.purpose != self._purpose + or identity.purpose not in self._purposes or identity.materialized_projection_session is None or identity.active_runtime_release is None or identity.active_runtime_release.organization_id diff --git a/adapters/http/ui_api.py b/adapters/http/ui_api.py new file mode 100644 index 00000000..516dd339 --- /dev/null +++ b/adapters/http/ui_api.py @@ -0,0 +1,1130 @@ +"""Tenant-safe backing contracts for the server-rendered operator surface.""" + +from __future__ import annotations + +import base64 +import binascii +import hashlib +import hmac +import json +from collections.abc import Callable, Iterator +from contextlib import contextmanager +from dataclasses import dataclass, field +from datetime import UTC, datetime, timedelta +from typing import Any, Final, Literal, Protocol, cast +from uuid import UUID, uuid4 + +from pydantic import BaseModel, ConfigDict, Field +from sqlalchemy import Engine, text +from sqlalchemy.exc import SQLAlchemyError + +from adapters.file_source import FileRootRegistry +from adapters.http.authentication import VerifiedAuthenticationContext +from adapters.parsers.markdown import compile_markdown +from engine.control import ( + ControlOperation, + FileImportPath, + FileRootRef, + MinimalUiControlGate, + TrustedControlCall, +) +from engine.persistence.membership_context import ( + MembershipAuthorityUnavailable, + MembershipIdentity, + MembershipNotCurrent, + PostgreSQLMembershipAuthority, +) +from engine.persistence.role_guard import assert_control_role, assert_runtime_role +from engine.runtime.actor import CurrentMembershipVerification +from engine.supply import ( + ACTIVE_FILE_IMPORT_MARKDOWN_CONFIG_VERSION, + CompilationFailure, + MarkdownCompilerConfig, + ParsedDocument, +) + +_PREVIEW_TTL: Final = timedelta(minutes=10) +_PREVIEW_DOMAIN: Final = b"context-engine.ui-preview.v1\x00" +_FEEDBACK_DOMAIN: Final = b"context-engine.ui-feedback.v1\x00" +_MAX_SIGNED_BIGINT: Final = (1 << 63) - 1 + + +class UiApiUnavailable(RuntimeError): + """A required operator projection or evidence store is unavailable.""" + + +@dataclass(frozen=True, slots=True) +class UiActor: + """Verified HTTP identity projected without granting new authority.""" + + organization_id: UUID = field(repr=False) + user_id: UUID = field(repr=False) + membership_id: UUID = field(repr=False) + membership_version: int + principal_ref: str = field(repr=False) + authentication_binding_ref: str = field(repr=False, default="ui-binding") + + +@dataclass(frozen=True, slots=True) +class FeedbackCapture: + """Minimal ContextRun evidence with no behavior or publication action.""" + + run_ref: str + rating: Literal["helpful", "not_helpful"] + note: str | None = None + + def __post_init__(self) -> None: + if ( + type(self.run_ref) is not str + or not self.run_ref + or self.run_ref.isspace() + or len(self.run_ref) > 256 + ): + raise ValueError("feedback run ref is invalid") + if self.rating not in {"helpful", "not_helpful"}: + raise ValueError("feedback rating is invalid") + if self.note is not None and ( + type(self.note) is not str + or not self.note + or self.note.isspace() + or len(self.note) > 1000 + ): + raise ValueError("feedback note is invalid") + + +class FeedbackWire(BaseModel): + model_config = ConfigDict(extra="forbid") + + runRef: str = Field(strict=True, min_length=1, max_length=256) + rating: Literal["helpful", "not_helpful"] + note: str | None = Field(default=None, strict=True, min_length=1, max_length=1000) + + +class ImportPreviewWire(BaseModel): + model_config = ConfigDict(extra="forbid") + + sourceRef: UUID + path: str = Field(strict=True, min_length=1, max_length=255) + + +class PreviewConfirmWire(BaseModel): + model_config = ConfigDict(extra="forbid") + + previewToken: str = Field(strict=True, min_length=1, max_length=4096) + + +class ArticleWire(BaseModel): + model_config = ConfigDict(extra="forbid") + + resourceRef: str = Field(strict=True, min_length=1, max_length=512) + + +class ArticlePolicyPreviewWire(ArticleWire): + policyKind: Literal["private", "organization", "groups"] + groupRefs: list[str] = Field(default_factory=list, max_length=100) + + +class UiSessionResponse(BaseModel): + model_config = ConfigDict(extra="forbid") + + state: Literal["active"] + + +class UiProfileIdentityResponse(BaseModel): + model_config = ConfigDict(extra="forbid") + + profileRef: str = Field(strict=True, min_length=1, max_length=512) + digest: str = Field(strict=True, pattern=r"^[0-9a-f]{64}$") + + +class UiProfilesResponse(BaseModel): + model_config = ConfigDict(extra="forbid") + + releaseGeneration: int = Field(strict=True, ge=1) + releaseManifestRef: str = Field(strict=True, min_length=1, max_length=512) + contentProfile: UiProfileIdentityResponse + indexProfile: UiProfileIdentityResponse + runtimeProfile: UiProfileIdentityResponse + + +class UiSourceHealthResponse(BaseModel): + model_config = ConfigDict(extra="forbid") + + activeResourceCount: int = Field(strict=True, ge=0) + displayName: str = Field(strict=True, min_length=1, max_length=200) + lastSuccessfulAcquisitionAgeSeconds: int | None = Field( + default=None, strict=True, ge=0 + ) + refusalCategories: list[str] = Field(max_length=100) + sourceRef: str = Field(strict=True, min_length=1, max_length=512) + status: Literal["ready", "refused", "waiting_first_success"] + + +class UiOverviewResponse(BaseModel): + model_config = ConfigDict(extra="forbid") + + releaseGeneration: int = Field(strict=True, ge=1) + releaseManifestRef: str = Field(strict=True, min_length=1, max_length=512) + sources: list[UiSourceHealthResponse] + + +class UiImportFragmentResponse(BaseModel): + model_config = ConfigDict(extra="forbid") + + fragmentRef: str = Field(strict=True, min_length=1, max_length=512) + text: str = Field(strict=True, min_length=1) + + +class UiImportPreviewResponse(BaseModel): + model_config = ConfigDict(extra="forbid") + + compilationDigest: str = Field(strict=True, pattern=r"^[0-9a-f]{64}$") + fragmentDigest: str = Field(strict=True, pattern=r"^[0-9a-f]{64}$") + fragments: list[UiImportFragmentResponse] = Field(min_length=1) + path: str = Field(strict=True, min_length=1, max_length=255) + previewToken: str = Field(strict=True, min_length=1, max_length=4096) + sourceRef: str = Field(strict=True, min_length=1, max_length=512) + + +class UiImportConfirmResponse(BaseModel): + model_config = ConfigDict(extra="forbid") + + jobRef: str = Field(strict=True, min_length=1, max_length=512) + state: Literal["queued"] + + +class UiArticleResponse(BaseModel): + model_config = ConfigDict(extra="forbid") + + effectiveGroupRefs: list[str] = Field(max_length=100) + effectivePolicyKind: Literal["private", "organization", "groups"] | None + localGroupRefs: list[str] = Field(max_length=100) + localPolicyKind: Literal["private", "organization", "groups"] | None + policyEpoch: int = Field(strict=True, ge=1) + policyVersion: int = Field(strict=True, ge=1) + published: bool = Field(strict=True) + resolutionRung: Literal[ + "explicit_article", "source_default", "tenant_default", "isolation" + ] + resourceRef: str = Field(strict=True, min_length=1, max_length=512) + sourceRef: str = Field(strict=True, min_length=1, max_length=512) + + +class UiArticlePolicyProposalResponse(BaseModel): + model_config = ConfigDict(extra="forbid") + + groupRefs: list[str] = Field(max_length=100) + policyKind: Literal["private", "organization", "groups"] + + +class UiArticlePolicyPreviewResponse(BaseModel): + model_config = ConfigDict(extra="forbid") + + current: UiArticleResponse + proposed: UiArticlePolicyProposalResponse + previewToken: str = Field(strict=True, min_length=1, max_length=4096) + + +class UiArticlePolicyConfirmResponse(BaseModel): + model_config = ConfigDict(extra="forbid") + + policyEpoch: int = Field(strict=True, ge=1) + policyVersion: int = Field(strict=True, ge=1) + state: Literal["changed"] + + +class UiFeedbackResponse(BaseModel): + model_config = ConfigDict(extra="forbid") + + feedbackRef: str = Field(strict=True, min_length=1, max_length=512) + state: Literal["recorded"] + + +class UiApi(Protocol): + """Narrow backing seam; methods return only closed public JSON documents.""" + + def overview( + self, actor: UiActor, control_call: TrustedControlCall + ) -> dict[str, object]: ... + + def profiles(self, actor: UiActor) -> dict[str, object]: ... + + def preview_import( + self, + actor: UiActor, + control_call: TrustedControlCall, + *, + source_ref: UUID, + path: str, + ) -> dict[str, object]: ... + + def confirm_import( + self, + actor: UiActor, + control_call: TrustedControlCall, + *, + preview_token: str, + ) -> dict[str, object]: ... + + def article( + self, + actor: UiActor, + control_call: TrustedControlCall, + *, + resource_ref: str, + ) -> dict[str, object]: ... + + def preview_article_policy( + self, + actor: UiActor, + control_call: TrustedControlCall, + *, + resource_ref: str, + policy_kind: str, + group_refs: tuple[str, ...], + ) -> dict[str, object]: ... + + def confirm_article_policy( + self, + actor: UiActor, + control_call: TrustedControlCall, + *, + preview_token: str, + ) -> dict[str, object]: ... + + def capture_feedback( + self, + actor: UiActor, + feedback: FeedbackCapture, + ) -> dict[str, object]: ... + + +class RefusingUiApi: + def overview( + self, actor: UiActor, control_call: TrustedControlCall + ) -> dict[str, object]: + del actor, control_call + raise UiApiUnavailable + + def profiles(self, actor: UiActor) -> dict[str, object]: + del actor + raise UiApiUnavailable + + def preview_import( + self, + actor: UiActor, + control_call: TrustedControlCall, + *, + source_ref: UUID, + path: str, + ) -> dict[str, object]: + del actor, control_call, source_ref, path + raise UiApiUnavailable + + def confirm_import( + self, + actor: UiActor, + control_call: TrustedControlCall, + *, + preview_token: str, + ) -> dict[str, object]: + del actor, control_call, preview_token + raise UiApiUnavailable + + def article( + self, + actor: UiActor, + control_call: TrustedControlCall, + *, + resource_ref: str, + ) -> dict[str, object]: + del actor, control_call, resource_ref + raise UiApiUnavailable + + def preview_article_policy( + self, + actor: UiActor, + control_call: TrustedControlCall, + *, + resource_ref: str, + policy_kind: str, + group_refs: tuple[str, ...], + ) -> dict[str, object]: + del actor, control_call, resource_ref, policy_kind, group_refs + raise UiApiUnavailable + + def confirm_article_policy( + self, + actor: UiActor, + control_call: TrustedControlCall, + *, + preview_token: str, + ) -> dict[str, object]: + del actor, control_call, preview_token + raise UiApiUnavailable + + def capture_feedback( + self, + actor: UiActor, + feedback: FeedbackCapture, + ) -> dict[str, object]: + del actor, feedback + raise UiApiUnavailable + + +@dataclass(frozen=True, slots=True) +class _PreviewCodec: + key: bytes = field(repr=False) + clock: Callable[[], datetime] = field(repr=False) + + def __post_init__(self) -> None: + if type(self.key) is not bytes or len(self.key) < 32: + raise ValueError("UI preview signing key is unavailable") + if not callable(self.clock): + raise TypeError("UI preview clock is required") + + def issue( + self, + kind: Literal["file_import", "article_policy"], + actor: UiActor, + payload: dict[str, object], + ) -> str: + now = _utc(self.clock()) + document = { + "actor": _actor_document(actor), + "expiresAt": int((now + _PREVIEW_TTL).timestamp()), + "issuedAt": int(now.timestamp()), + "kind": kind, + "payload": payload, + } + encoded = _encode( + json.dumps( + document, + ensure_ascii=True, + separators=(",", ":"), + sort_keys=True, + ).encode("utf-8") + ) + signature = _encode( + hmac.digest(self.key, _PREVIEW_DOMAIN + encoded.encode("ascii"), "sha256") + ) + return f"{encoded}.{signature}" + + def verify( + self, + token: str, + *, + kind: Literal["file_import", "article_policy"], + actor: UiActor, + ) -> dict[str, object]: + if type(token) is not str or len(token) > 4096 or token.count(".") != 1: + raise UiApiUnavailable + encoded, supplied = token.split(".") + expected = _encode( + hmac.digest(self.key, _PREVIEW_DOMAIN + encoded.encode("ascii"), "sha256") + ) + if not hmac.compare_digest(supplied, expected): + raise UiApiUnavailable + try: + document = json.loads(_decode(encoded)) + except (UnicodeDecodeError, json.JSONDecodeError, ValueError): + raise UiApiUnavailable from None + now = int(_utc(self.clock()).timestamp()) + if ( + type(document) is not dict + or document.get("kind") != kind + or document.get("actor") != _actor_document(actor) + or type(document.get("issuedAt")) is not int + or type(document.get("expiresAt")) is not int + or not cast(int, document["issuedAt"]) + <= now + < cast(int, document["expiresAt"]) + or cast(int, document["expiresAt"]) - cast(int, document["issuedAt"]) + != int(_PREVIEW_TTL.total_seconds()) + or type(document.get("payload")) is not dict + ): + raise UiApiUnavailable + return cast(dict[str, object], document["payload"]) + + +class PostgreSQLUiApi: + """Authenticated M1 projections and explicit effects on least-privilege seams.""" + + def __init__( + self, + membership_authority: PostgreSQLMembershipAuthority, + control_engine: Engine | None, + *, + preview_key: bytes, + feedback_engine: Engine | None = None, + control_gate: MinimalUiControlGate | None = None, + roots: FileRootRegistry | None = None, + file_import_service_principal_id: UUID | None = None, + clock: Callable[[], datetime] = lambda: datetime.now(UTC), + ) -> None: + if type(membership_authority) is not PostgreSQLMembershipAuthority: + raise TypeError("UI requires the PostgreSQL Membership authority") + if control_engine is not None and not isinstance(control_engine, Engine): + raise TypeError("UI Control database engine is invalid") + if feedback_engine is not None and not isinstance(feedback_engine, Engine): + raise TypeError("UI feedback database engine is invalid") + if roots is not None and type(roots) is not FileRootRegistry: + raise TypeError("UI File roots have the wrong nominal type") + if ( + file_import_service_principal_id is not None + and type(file_import_service_principal_id) is not UUID + ): + raise TypeError("UI File receiver must be UUID") + self._membership_authority = membership_authority + self._control_engine = control_engine + self._feedback_engine = feedback_engine + self._control_gate = control_gate + self._roots = roots + self._receiver_id = file_import_service_principal_id + self._clock = clock + self._preview_codec = _PreviewCodec(preview_key, clock) + + @contextmanager + def _verified( + self, + actor: UiActor, + ) -> Iterator[CurrentMembershipVerification]: + checked_at = _utc(self._clock()) + try: + with self._membership_authority.current_user_actor( + MembershipIdentity( + organization_id=actor.organization_id, + user_id=actor.user_id, + membership_id=actor.membership_id, + membership_version=actor.membership_version, + principal_ref=actor.principal_ref, + request_id=f"ui-{uuid4().hex}", + authentication_binding_ref=actor.authentication_binding_ref, + checked_at=checked_at, + ) + ) as verification: + yield verification + except (MembershipAuthorityUnavailable, MembershipNotCurrent): + raise UiApiUnavailable from None + + @contextmanager + def _control(self, actor: UiActor) -> Iterator[Any]: + engine = self._control_engine + if engine is None: + raise UiApiUnavailable + try: + with engine.begin() as connection: + assert_control_role(connection) + observed = connection.execute( + text( + "SELECT set_config('app.organization_id', " + ":organization_id, true)" + ), + {"organization_id": str(actor.organization_id)}, + ).scalar_one() + if observed != str(actor.organization_id): + raise UiApiUnavailable + yield connection + except UiApiUnavailable: + raise + except (AssertionError, SQLAlchemyError): + raise UiApiUnavailable from None + + @contextmanager + def _feedback(self, actor: UiActor) -> Iterator[Any]: + engine = self._feedback_engine + if engine is None: + raise UiApiUnavailable + expected = { + "actor_kind": "user", + "membership_id": str(actor.membership_id), + "membership_version": str(actor.membership_version), + "organization_id": str(actor.organization_id), + "principal_ref": actor.principal_ref, + "user_id": str(actor.user_id), + } + try: + with engine.begin() as connection: + assert_runtime_role(connection) + observed = dict( + connection.execute( + text( + """ + SELECT + set_config('app.actor_kind', 'user', true) + AS actor_kind, + set_config('app.membership_id', + :membership_id, true) AS membership_id, + set_config('app.membership_version', + :membership_version, true) + AS membership_version, + set_config('app.organization_id', + :organization_id, true) AS organization_id, + set_config('app.principal_ref', + :principal_ref, true) AS principal_ref, + set_config('app.user_id', :user_id, true) + AS user_id + """ + ), + expected, + ).mappings().one() + ) + if observed != expected: + raise UiApiUnavailable + yield connection + except UiApiUnavailable: + raise + except (AssertionError, SQLAlchemyError): + raise UiApiUnavailable from None + + def _consume_control( + self, + actor: UiActor, + control_call: TrustedControlCall, + operation: ControlOperation, + ) -> None: + gate = self._control_gate + if gate is None: + raise UiApiUnavailable + try: + gate.consume( + control_call, + organization_id=actor.organization_id, + operation=operation, + ) + except Exception: + raise UiApiUnavailable from None + + def overview( + self, actor: UiActor, control_call: TrustedControlCall + ) -> dict[str, object]: + self._consume_control( + actor, control_call, ControlOperation.READ_SOURCE_PROGRESS + ) + with self._verified(actor) as verification: + release = verification.active_runtime_release + if release is None: + raise UiApiUnavailable + with self._control(actor) as connection: + sources = tuple( + connection.execute( + text( + """ + SELECT source_id, display_name + FROM context_source + WHERE organization_id = :organization_id + AND lifecycle_state = 'active' + ORDER BY display_name COLLATE "C", source_id + """ + ), + {"organization_id": actor.organization_id}, + ).mappings() + ) + projected: list[dict[str, object]] = [] + for source in sources: + rows = tuple( + connection.execute( + text( + """ + SELECT * FROM + context_control_read_file_source_status( + :organization_id, :source_id + ) + """ + ), + { + "organization_id": actor.organization_id, + "source_id": source["source_id"], + }, + ).mappings() + ) + if not rows: + raise UiApiUnavailable + categories = sorted( + { + cast(str, row["refusal_category"]) + for row in rows + if row["refusal_category"] is not None + } + ) + first = rows[0] + status = ( + "refused" + if categories + else ( + "waiting_first_success" + if first["last_successful_acquisition_at"] is None + else "ready" + ) + ) + projected.append( + { + "activeResourceCount": first["active_resource_count"], + "displayName": source["display_name"], + "lastSuccessfulAcquisitionAgeSeconds": first[ + "last_successful_acquisition_age_seconds" + ], + "refusalCategories": categories, + "sourceRef": str(source["source_id"]), + "status": status, + } + ) + return { + "releaseGeneration": release.active_generation, + "releaseManifestRef": release.manifest_ref, + "sources": projected, + } + + def profiles(self, actor: UiActor) -> dict[str, object]: + with self._verified(actor) as verification: + release = verification.active_runtime_release + if release is None: + raise UiApiUnavailable + return { + "releaseGeneration": release.active_generation, + "releaseManifestRef": release.manifest_ref, + "contentProfile": { + "profileRef": release.content_profile_ref, + "digest": release.content_profile_digest, + }, + "indexProfile": { + "profileRef": release.index_profile_ref, + "digest": release.index_profile_digest, + }, + "runtimeProfile": { + "profileRef": release.runtime_profile_ref, + "digest": release.runtime_profile_digest, + }, + } + + def preview_import( + self, + actor: UiActor, + control_call: TrustedControlCall, + *, + source_ref: UUID, + path: str, + ) -> dict[str, object]: + self._consume_control(actor, control_call, ControlOperation.IMPORT_FILE) + roots = self._roots + if roots is None or self._receiver_id is None: + raise UiApiUnavailable + try: + import_path = FileImportPath(path) + except (TypeError, ValueError): + raise UiApiUnavailable from None + with self._verified(actor), self._control(actor) as connection: + root_ref = connection.execute( + text( + """ + SELECT version.root_ref + FROM context_source AS source + JOIN source_version AS version + ON version.organization_id = source.organization_id + AND version.source_id = source.source_id + AND version.version_id = source.active_version_id + WHERE source.organization_id = :organization_id + AND source.source_id = :source_id + AND source.lifecycle_state = 'active' + """ + ), + { + "organization_id": actor.organization_id, + "source_id": source_ref, + }, + ).scalar_one_or_none() + if type(root_ref) is not str: + raise UiApiUnavailable + try: + raw = roots.read(FileRootRef(root_ref), import_path) + outcome = compile_markdown( + raw, + MarkdownCompilerConfig(ACTIVE_FILE_IMPORT_MARKDOWN_CONFIG_VERSION), + ) + except (LookupError, RuntimeError, TypeError, ValueError): + raise UiApiUnavailable from None + if type(outcome) is CompilationFailure or type(outcome) is not ParsedDocument: + raise UiApiUnavailable + fragments = [ + { + "fragmentRef": fragment.fragment_ref, + "text": fragment.contextual_text, + } + for fragment in outcome.fragments + ] + fragment_digest = hashlib.sha256( + json.dumps( + fragments, + ensure_ascii=False, + separators=(",", ":"), + sort_keys=True, + ).encode("utf-8") + ).hexdigest() + payload: dict[str, object] = { + "configVersion": ACTIVE_FILE_IMPORT_MARKDOWN_CONFIG_VERSION, + "contentLength": len(raw), + "contentSha256": hashlib.sha256(raw).hexdigest(), + "fragmentDigest": fragment_digest, + "path": import_path.value, + "sourceRef": str(source_ref), + } + return { + "compilationDigest": outcome.compilation_digest, + "fragmentDigest": fragment_digest, + "fragments": fragments, + "path": import_path.value, + "previewToken": self._preview_codec.issue("file_import", actor, payload), + "sourceRef": str(source_ref), + } + + def confirm_import( + self, + actor: UiActor, + control_call: TrustedControlCall, + *, + preview_token: str, + ) -> dict[str, object]: + self._consume_control(actor, control_call, ControlOperation.IMPORT_FILE) + receiver_id = self._receiver_id + if receiver_id is None: + raise UiApiUnavailable + payload = self._preview_codec.verify( + preview_token, + kind="file_import", + actor=actor, + ) + try: + source_ref = UUID(cast(str, payload["sourceRef"])) + path = FileImportPath(cast(str, payload["path"])) + content_sha256 = cast(str, payload["contentSha256"]) + content_length = cast(int, payload["contentLength"]) + fragment_digest = cast(str, payload["fragmentDigest"]) + config_version = cast(str, payload["configVersion"]) + if ( + len(content_sha256) != 64 + or len(fragment_digest) != 64 + or type(content_length) is not int + or content_length < 0 + or config_version != ACTIVE_FILE_IMPORT_MARKDOWN_CONFIG_VERSION + ): + raise ValueError + except (KeyError, TypeError, ValueError): + raise UiApiUnavailable from None + token_digest = hashlib.sha256(preview_token.encode("ascii")).hexdigest() + request_digest = hashlib.sha256( + b"context-engine.ui-import-confirm.v1\x00" + token_digest.encode("ascii") + ).hexdigest() + # The Runtime membership transaction holds the shared publication fence. + # Close it before the Control function takes the exclusive publication + # fence; the function independently rechecks the exact Membership. + with self._verified(actor): + pass + with self._control(actor) as connection: + row = ( + connection.execute( + text( + """ + SELECT * FROM context_control_prepare_exact_file_import( + :organization_id, :acquisition_id, :job_id, + :activated_version_id, :source_id, :relative_path, + :audience_principal_ref, :audience_user_id, + :audience_membership_id, + :audience_membership_version, :idempotency_key, + :request_digest, :service_principal_id, + :expected_content_sha256, :expected_content_length, + :expected_fragment_digest, :compiler_config_version + , :preview_digest + ) + """ + ), + { + "organization_id": actor.organization_id, + "acquisition_id": uuid4(), + "job_id": uuid4(), + "activated_version_id": uuid4(), + "source_id": source_ref, + "relative_path": path.value, + "audience_principal_ref": actor.principal_ref, + "audience_user_id": actor.user_id, + "audience_membership_id": actor.membership_id, + "audience_membership_version": actor.membership_version, + "idempotency_key": f"ui:{token_digest}", + "request_digest": request_digest, + "service_principal_id": receiver_id, + "expected_content_sha256": content_sha256, + "expected_content_length": content_length, + "expected_fragment_digest": fragment_digest, + "compiler_config_version": config_version, + "preview_digest": token_digest, + }, + ) + .mappings() + .one_or_none() + ) + if row is None: + raise UiApiUnavailable + return {"jobRef": str(row["job_id"]), "state": "queued"} + + def article( + self, + actor: UiActor, + control_call: TrustedControlCall, + *, + resource_ref: str, + ) -> dict[str, object]: + self._consume_control(actor, control_call, ControlOperation.READ_ARTICLE_POLICY) + return self._article(actor, resource_ref=resource_ref) + + def _article(self, actor: UiActor, *, resource_ref: str) -> dict[str, object]: + if ( + type(resource_ref) is not str + or not resource_ref + or resource_ref.isspace() + or len(resource_ref) > 512 + ): + raise UiApiUnavailable + with self._verified(actor), self._control(actor) as connection: + row = ( + connection.execute( + text( + "SELECT * FROM context_control_read_article_policy(" + ":organization_id, :resource_ref)" + ), + { + "organization_id": actor.organization_id, + "resource_ref": resource_ref, + }, + ) + .mappings() + .one_or_none() + ) + if row is None: + raise UiApiUnavailable + return _article_document(row) + + def preview_article_policy( + self, + actor: UiActor, + control_call: TrustedControlCall, + *, + resource_ref: str, + policy_kind: str, + group_refs: tuple[str, ...], + ) -> dict[str, object]: + self._consume_control( + actor, control_call, ControlOperation.CHANGE_ARTICLE_POLICY + ) + if policy_kind not in {"private", "organization", "groups"}: + raise UiApiUnavailable + normalized = tuple(sorted(set(group_refs))) + if ( + (policy_kind == "groups") != bool(normalized) + or len(normalized) > 100 + or any( + type(value) is not str + or not value + or len(value) > 256 + or any(character.isspace() for character in value) + for value in normalized + ) + ): + raise UiApiUnavailable + current = self._article(actor, resource_ref=resource_ref) + payload = { + "expectedPolicyEpoch": current["policyEpoch"], + "expectedPolicyVersion": current["policyVersion"], + "groupRefs": list(normalized), + "policyKind": policy_kind, + "resourceRef": resource_ref, + } + return { + "current": current, + "proposed": {"groupRefs": list(normalized), "policyKind": policy_kind}, + "previewToken": self._preview_codec.issue( + "article_policy", + actor, + payload, + ), + } + + def confirm_article_policy( + self, + actor: UiActor, + control_call: TrustedControlCall, + *, + preview_token: str, + ) -> dict[str, object]: + self._consume_control( + actor, control_call, ControlOperation.CHANGE_ARTICLE_POLICY + ) + payload = self._preview_codec.verify( + preview_token, + kind="article_policy", + actor=actor, + ) + try: + resource_ref = cast(str, payload["resourceRef"]) + policy_kind = cast(str, payload["policyKind"]) + group_refs = cast(list[str], payload["groupRefs"]) + expected_version = cast(int, payload["expectedPolicyVersion"]) + expected_epoch = cast(int, payload["expectedPolicyEpoch"]) + if ( + policy_kind not in {"private", "organization", "groups"} + or type(expected_version) is not int + or not 1 <= expected_version <= _MAX_SIGNED_BIGINT + or type(expected_epoch) is not int + or not 1 <= expected_epoch <= _MAX_SIGNED_BIGINT + or type(group_refs) is not list + or any(type(value) is not str for value in group_refs) + ): + raise ValueError + except (KeyError, TypeError, ValueError): + raise UiApiUnavailable from None + preview_digest = hashlib.sha256(preview_token.encode("ascii")).hexdigest() + # Do not retain Runtime's shared publication fence while Control + # advances the Policy Epoch. The definer locks and rechecks this exact + # current Membership before it mutates the Article policy. + with self._verified(actor): + pass + with self._control(actor) as connection: + row = ( + connection.execute( + text( + """ + SELECT * FROM context_control_change_article_policy( + :organization_id, :resource_ref, + :expected_policy_version, :expected_policy_epoch, + :policy_kind, CAST(:group_refs AS text[]), + :preview_digest, :user_id, :membership_id, + :membership_version + ) + """ + ), + { + "organization_id": actor.organization_id, + "resource_ref": resource_ref, + "expected_policy_version": expected_version, + "expected_policy_epoch": expected_epoch, + "policy_kind": policy_kind, + "group_refs": group_refs, + "preview_digest": preview_digest, + "user_id": actor.user_id, + "membership_id": actor.membership_id, + "membership_version": actor.membership_version, + }, + ) + .mappings() + .one_or_none() + ) + if row is None: + raise UiApiUnavailable + return { + "policyEpoch": row["policy_epoch"], + "policyVersion": row["policy_version"], + "state": "changed", + } + + def capture_feedback( + self, + actor: UiActor, + feedback: FeedbackCapture, + ) -> dict[str, object]: + feedback.__post_init__() + entropy = uuid4().bytes + feedback_ref = ( + "fb_" + + hashlib.sha256( + _FEEDBACK_DOMAIN + actor.organization_id.bytes + entropy + ).hexdigest() + ) + with self._verified(actor), self._feedback(actor) as connection: + recorded = connection.execute( + text( + """ + SELECT context_runtime_capture_context_feedback( + :organization_id, :feedback_ref, :run_ref, + :user_id, :membership_id, :membership_version, + :principal_ref, :rating, :note + ) + """ + ), + { + "organization_id": actor.organization_id, + "feedback_ref": feedback_ref, + "run_ref": feedback.run_ref, + "user_id": actor.user_id, + "membership_id": actor.membership_id, + "membership_version": actor.membership_version, + "principal_ref": actor.principal_ref, + "rating": feedback.rating, + "note": feedback.note, + }, + ).scalar_one_or_none() + if type(recorded) is not str: + raise UiApiUnavailable + return {"feedbackRef": recorded, "state": "recorded"} + + +def _actor_document(actor: UiActor) -> dict[str, object]: + return { + "membershipId": str(actor.membership_id), + "membershipVersion": actor.membership_version, + "organizationId": str(actor.organization_id), + "principalRef": actor.principal_ref, + "userId": str(actor.user_id), + } + + +def _article_document(row: Any) -> dict[str, object]: + return { + "effectiveGroupRefs": list(row["group_refs"]), + "effectivePolicyKind": row["policy_kind"], + "localGroupRefs": list(row["local_group_refs"]), + "localPolicyKind": row["local_policy_kind"], + "policyEpoch": row["policy_epoch"], + "policyVersion": row["policy_version"], + "published": row["published"], + "resolutionRung": row["resolution_rung"], + "resourceRef": row["resource_ref"], + "sourceRef": row["source_ref"], + } + + +def _utc(value: object) -> datetime: + if ( + type(value) is not datetime + or value.tzinfo is None + or value.utcoffset() != timedelta(0) + ): + raise UiApiUnavailable + return value + + +def _encode(value: bytes) -> str: + return base64.urlsafe_b64encode(value).rstrip(b"=").decode("ascii") + + +def _decode(value: str) -> str: + try: + padding = "=" * (-len(value) % 4) + return base64.b64decode( + value + padding, + altchars=b"-_", + validate=True, + ).decode("utf-8") + except (binascii.Error, UnicodeDecodeError): + raise ValueError from None + + +def ui_actor(authentication: VerifiedAuthenticationContext) -> UiActor: + try: + return UiActor( + organization_id=UUID(authentication.organization_ref), + user_id=UUID(authentication.user_ref), + membership_id=UUID(authentication.membership_ref), + membership_version=authentication.membership_version, + principal_ref=authentication.principal_ref, + authentication_binding_ref=authentication.authentication_binding_ref, + ) + except (TypeError, ValueError): + raise UiApiUnavailable from None diff --git a/context/BRIEF.md b/context/BRIEF.md new file mode 100644 index 00000000..e23b9710 --- /dev/null +++ b/context/BRIEF.md @@ -0,0 +1,53 @@ +# ContextEngine Minimal UI brief + +## Load-bearing + +- **Single job:** Let the single-tenant maintainer operate and inspect the seven M1 + ContextEngine jobs without leaving the authenticated public HTTP surface. + `[inferred from issue #130]` +- **Audience + first-three-second feeling:** ContextEngine maintainer/operator; + calm, precise, trustworthy, austere. `[answered by maintainer, 2026-07-30]` +- **Signature moment:** Activating any rendered content Block reveals its exact + Article → Revision → Fragment lineage, Evidence reference, and Policy Epoch. + **Kind:** interaction. `[answered by maintainer, 2026-07-30]` + +## Shape + +- **What the site is:** An authenticated, server-rendered operator tool inside the + existing FastAPI process. `[answered by maintainer]` +- **Primary action:** Ask ContextEngine and inspect the authorized answer evidence. + `[inferred from the M1 daily-driver exit criterion]` +- **Business model:** Internal daily-driver surface; no commercial conversion. + `[inferred from the repository roadmap]` +- **Trust barrier:** The operator must be able to trust that no refused candidate, + implicit policy mutation, or publication authority is hidden behind convenience. + `[inferred from issue #130 and AGENTS.md]` +- **Content source:** Real ContextPackage, Control, and Release data from the public + HTTP seam; no fabricated records, proof, or metrics. `[answered by issue #130]` +- **Visibility inspection:** The Article inspector shows the effective resolution + rung because visibility is an Article/Control projection, not a ContextPackage + Evidence field. `[inferred from the frozen public contracts]` +- **SEO vs conversion:** Neither; safe task completion and inspectable evidence win. + `[inferred from the local operator scope]` +- **Languages / regions:** English, locale-neutral identifiers and timestamps. + `[inferred from the public HTTP contract]` + +## Defaults applied + +- Console archetype, light-first with a matching dark scheme. +- Balanced density on a 4px spacing base; deep teal is reserved for interaction and + focus, never used to decorate a refusal. +- IBM Plex Sans/Avenir Next UI stack and IBM Plex Mono/SFMono metadata stack, with + local system fallbacks and no remote font dependency. +- Motion is limited to short state feedback and the evidence reveal; reduced-motion + removes non-essential transitions. + +## Open questions + +- None blocking M1. A richer client application remains an M2+ evidence-driven + process-boundary decision. + +## Conflicts + +None detected. Dense operational evidence and calm presentation coexist by using +progressive disclosure rather than hiding facts or adding decorative chrome. diff --git a/context/IA.md b/context/IA.md new file mode 100644 index 00000000..250d1401 --- /dev/null +++ b/context/IA.md @@ -0,0 +1,52 @@ +# ContextEngine Minimal UI information architecture + +## Sitemap + +```text +/ui operational overview and source health +/ui/ask Ask with citation lineage +/ui/import import preview and explicit confirmation +/ui/hit-test authorized-only retrieval inspection +/ui/articles Article lookup, visibility policy, rung, and confirmed edit +/ui/profiles active versioned profiles and re-embed preview +/ui/feedback answer feedback capture +``` + +## Page types + +| Route | Pattern | Primary goal | +|---|---|---| +| `/ui` | operator overview | Establish current source and Release state | +| `/ui/ask` | answer workbench | Obtain authorized context with inspectable citations | +| `/ui/import` | preview/confirm form | Inspect exact Fragments before scheduling publication work | +| `/ui/hit-test` | diagnostic tool | Inspect authorized hits without observing refusals | +| `/ui/articles` | lookup + record detail | Understand and explicitly change one Article policy without placing tenant refs in URLs | +| `/ui/profiles` | configuration detail | Reproduce active model identities and preview re-embed impact | +| `/ui/feedback` | evidence form | Attach minimal feedback to one authorized ContextRun | + +## Functional modules + +- Public HTTP client and session/refusal boundary. +- Source/Release operational projection. +- ContextPackage answer, coverage, and citation projection. +- Import preview receipt and one explicit confirm action. +- Authorized Hit Test projection. +- Article policy detail and policy-change receipt. +- Active profile detail and consequence preview. +- Feedback evidence receipt with no publication operations. + +## Primary user paths + +1. `/ui` → inspect source/Release → `/ui/ask` → activate evidence flip. +2. `/ui/import` → preview → confirm or cancel → `/ui` status. +3. `/ui/hit-test` → query → inspect authorized hits only. +4. `/ui/articles` → exact Article → preview policy → confirm epoch-changing edit. +5. `/ui/profiles` → inspect active identity → preview re-embed consequence. +6. Answer → adjacent feedback form → record evidence → return to Ask. + +## Navigation + +- **Primary:** Overview, Ask, Import, Hit Test, Articles, Profiles, Feedback. +- **Contextual:** Evidence lineage, Article policy, and feedback links remain adjacent + to the authorized item that produced them. +- **Footer:** Build identity and public-seam statement; no marketing links. diff --git a/context/IMPLEMENTATION.md b/context/IMPLEMENTATION.md new file mode 100644 index 00000000..d947220e --- /dev/null +++ b/context/IMPLEMENTATION.md @@ -0,0 +1,48 @@ +# ContextEngine Minimal UI implementation contract + +## Stack + +- The repository Python runtime pinned by `pyproject.toml`, FastAPI/Starlette, + Jinja2 server-side templates, and static semantic CSS in the existing API process. +- Small progressive-enhancement JavaScript is allowed only for evidence disclosure + ergonomics; native HTML behavior remains the functional baseline. +- No Node/TypeScript UI package, frontend build system, HTMX runtime dependency, or + additional process. + +## Directory structure + +```text +ui/ presentation, public HTTP client, templates, static assets +adapters/http/ API composition and public backing routes +tests/unit/ fast presentation/public-seam behavior in focused UI test modules +tests/integration/ real PostgreSQL public HTTP/UI security behavior +``` + +The `ui/` tree may not import `engine/`. Adapters own the translation between public +wire contracts and sealed engine/control/learning modules. + +## Commands + +The UI owns `make ui-build` and `make ui-test`; both are wired into `make check`. +Repository completion uses the sequential verification contract in root `AGENTS.md`. + +## Performance budget + +- One HTML response plus one cacheable CSS asset; no render-blocking remote assets. +- No client framework or hydration. Enhancement JavaScript stays below 8 KiB + uncompressed and never carries authorization facts. +- Layout shift is zero for SSR data; evidence disclosures reserve normal document + flow rather than overlaying content. + +## Definition of Done + +- [ ] Seven job routes work over the public seam inside the existing API process. +- [ ] `ui/` has no import or dependency on `engine/`. +- [ ] Evidence flip works with keyboard and native HTML fallback. +- [ ] Browser-native navigation provides loading feedback; returned documents make + empty, refusal/error, success, and mixed-source partial states reachable. +- [ ] Below-sm, below-md, and below-lg rules from `DESIGN.md` hold. +- [ ] No placeholder/fabricated content or remote asset dependency. +- [ ] `make ui-build`, `make ui-test`, lint, typecheck, unit, integration, and security + gates pass with fresh evidence. +- [ ] Root `DESIGN.md` contrast pairs pass and CSS uses semantic variables only. diff --git a/context/JOURNEYS.md b/context/JOURNEYS.md new file mode 100644 index 00000000..9a2444e9 --- /dev/null +++ b/context/JOURNEYS.md @@ -0,0 +1,64 @@ +# ContextEngine Minimal UI journeys + +## Journey 1 — First-visit happy path + +- **Actor + entry:** Authenticated local maintainer opens `/ui` directly. +- **Goal:** Confirm operational state, ask one question, and inspect lineage. + +| # | Screen | User does | System responds (+ state) | Next | +|---|---|---|---|---| +| 1 | `/ui` | Opens console | SSR success shows source cards and exact active Release generation | 2 | +| 1e | `/ui` | Opens without current context | Content-free refusal names only the safe category and recovery action | 1 | +| 2 | `/ui/ask` | Enters a question and submits | Input is preserved while the public resolve request completes | 3 | +| 3 | `/ui/ask` | Reads authorized Blocks | Success shows coverage and one evidence control per Block | 4 | +| 4 | `/ui/ask` | Activates a Block | Evidence flip reveals Article → Revision → Fragment and ACL lineage | 3 | + +- **Success criteria:** A clean answer never appears when its citation closure is + invalid; no user-visible element contains denied-candidate detail. + +## Journey 2 — Explicit-change path + +- **Actor + entry:** Returning maintainer opens Import or an Article detail. +- **Goal:** Make one bounded change only after previewing its exact consequences. + +| # | Screen | User does | System responds (+ state) | Next | +|---|---|---|---|---| +| 1 | `/ui/import` | Supplies Source and canonical path | Preview renders the actual ordered Fragments and an opaque receipt | 2 | +| 2a | `/ui/import` | Cancels | Receipt is discarded; no publication work is scheduled | `/ui` | +| 2b | `/ui/import` | Confirms | Exact receipt is consumed once; success names only safe job lineage | `/ui` | +| 3 | `/ui/articles/{resourceRef}` | Requests another policy | Preview shows prior/effective setting, rung, and epoch consequence | 4 | +| 4a | same | Cancels | No policy or epoch changes | same | +| 4b | same | Confirms | Receipt is consumed once; policy and Policy Epoch advance atomically | same | + +- **Step count defense:** Preview and confirmation are deliberately separate HTTP + transitions. No shortcut, inline toggle, or optimistic mutation exists. +- **Forms:** Labels remain visible; validate on blur and submit; failed submission + preserves safe user input but never echoes a secret or refused object. + +## Journey 3 — Failure / recovery path + +- **The failure:** Tenant/session context is missing or expired, or a required + provider is unavailable. + +| # | Screen | User does | System responds (+ state) | Next | +|---|---|---|---|---| +| 1 | any `/ui` route | Loads or submits | Explicit calm refusal; no empty-list mimicry and no object detail | 2 | +| 2 | same | Restores authenticated context or retries provider | Original non-secret input remains available; request is re-evaluated | success route | + +- **What is never lost:** Query text and non-secret form selections. Authentication + material, denied identifiers, and preview authority are never reflected into HTML. + +## Journey 4 — Feedback evidence only + +| # | Screen | User does | System responds (+ state) | Next | +|---|---|---|---|---| +| 1 | `/ui/feedback` | Selects helpful/not helpful for a run | Server validates exact current authorized run binding | 2 | +| 2 | same | Submits | Success returns a feedback Evidence receipt, not a Release action | answer | + +## Cross-journey rules + +- Query strings carry no identity or authority. Preview/confirm authority stays in an + opaque, expiring, one-shot receipt. +- A successful change returns to its detail/status view, never a dead end. +- Loading is bounded SSR progress; empty means a genuine authorized empty result, + while unavailable or missing context always renders refusal. diff --git a/context/PAGE_SPEC.md b/context/PAGE_SPEC.md new file mode 100644 index 00000000..a9f523ea --- /dev/null +++ b/context/PAGE_SPEC.md @@ -0,0 +1,93 @@ +# ContextEngine Minimal UI page specification + +## Shared application-shell contract + +- **Single goal:** Keep seven fixed operator jobs reachable without implying a + general administration canvas. +- **Section order:** skip link → product/header state → primary navigation → page + title/context → one primary work surface → safe status/refusal → footer. +- **States:** Full-page submissions use browser-native navigation/loading. Returned + documents distinguish authorized empty, refusal/error, and success; the source + overview expresses partial/degraded operation through mixed per-source statuses. + Refusal never reuses empty styling or copy. +- **Data strategy:** Server-side render from public HTTP responses; `Cache-Control: + no-store`; no optimistic security or publication state. +- **Responsive:** Below `lg`, navigation becomes a wrapped top rail; below `md`, + split details become one column and tables become labeled record stacks; below + `sm`, all actions and evidence summaries remain at least 44px tall. +- **SEO:** `noindex, nofollow`; this is an authenticated local tool. + +## Page: `/ui` + +- **Goal:** Establish source health and exact active Release generation. +- **Sections:** Release header → source health list → fixed pipeline/job links. +- **Primary action:** “Ask ContextEngine.” +- **States:** Authorized zero sources is an explicit first-run state; failed/stale + sources show safe refusal category; Release unavailability is a refusal. +- **Acceptance:** Generation matches the promoted Release; status includes counts and + never turns an unavailable source into a blank card. + +## Page: `/ui/ask` + +- **Goal:** Render an authorized answer with complete, resolvable citation lineage. +- **Sections:** Question form → coverage → Blocks → evidence details → feedback link. +- **Primary action:** “Ask.” +- **Signature:** Each Block is paired with one disclosure control that flips focus + from content to Evidence lineage without a page navigation. +- **States:** Empty coverage names `no_authorized_evidence`; unresolvable citation + shows flagged/refused answer state, never clean content. +- **Form:** One required question; validate on submit; preserve it after safe refusal. + +## Page: `/ui/import` + +- **Goal:** Publish no ingestion work until exact Fragment preview is confirmed. +- **Sections:** Source/path form → ordered Fragment preview → consequence summary → + Confirm and Cancel actions → receipt. +- **Primary action:** Preview first; Confirm exists only on a valid preview. +- **States:** Cancel is success-with-no-effect; stale/changed preview refuses confirm. +- **Form:** Source and canonical path; no upload or filesystem path leakage. + +## Page: `/ui/hit-test` + +- **Goal:** Show authorized Fragment hits and scores for one acting Principal. +- **Sections:** Query form → coverage → authorized hit list with Evidence controls. +- **Primary action:** “Run Hit Test.” +- **Forbidden:** Raw CandidateRef, denied count, denied rank, score gaps, SQL/log detail, + or wording that implies how many results were filtered. +- **States:** Authorized empty is one result category; refusal is visually and + semantically distinct but equally calm. + +## Page: `/ui/articles` + +- **Goal:** Explain one effective Article policy/rung and require preview + confirm + for any existing-Article change. +- **Sections:** Exact Article identity → effective/local policy → resolution rung → + change preview → confirm/cancel → epoch receipt. +- **Forbidden:** Inline toggle, bulk action, Source-container permission, or mutation + without expected policy version and expected Policy Epoch. + +## Page: `/ui/profiles` + +- **Goal:** Reproduce active provider/retrieval identities and preview profile-change + consequences. +- **Sections:** Release generation → content/index/runtime profile identities and + digests → proposed embedding profile → re-embed warning. +- **Primary action:** “Preview profile change.” M1 exposes no activation action. +- **States:** Every embedding change preview states full-corpus re-embed required. + +## Page: `/ui/feedback` + +- **Goal:** Record minimal answer evidence bound to an authorized ContextRun. +- **Sections:** Run/package identity → helpful/not helpful → optional bounded note → + receipt. +- **Forbidden:** Activate, promote, publish, rollback, curate, or change a profile. + +## Acceptance criteria + +- [ ] Every route and action works through the public HTTP seam. +- [ ] Every declared state is reachable, keyboard-usable, and content-safe. +- [ ] Forms preserve non-secret input on failure and cannot double-confirm. +- [ ] Responsive reflow holds below `lg`, `md`, and `sm`. +- [ ] No placeholder, fabricated record, secret, or refused-candidate detail renders. +- [ ] Components consume only semantic tokens from root `DESIGN.md`. +- [ ] WCAG AA contrast and visible focus hold; touch targets are at least 44px. diff --git a/context/PRODUCT.md b/context/PRODUCT.md new file mode 100644 index 00000000..d6ef8c89 --- /dev/null +++ b/context/PRODUCT.md @@ -0,0 +1,53 @@ +# ContextEngine Minimal UI product context + +## One-liner + +A calm, server-rendered daily-driver for operating ContextEngine through the same +public HTTP seam that every other delivery client must use. + +## Target user + +- **Primary:** The single-tenant maintainer/operator who currently switches among + CLI commands, `curl`, logs, and SQL to understand one ingestion or answer. +- **Secondary:** A reviewer verifying that the M1 carrier stays sealed and + fail-closed. + +## Value proposition + +The operator can see what is healthy, what was authorized, what a pending change +will do, and why a request was refused without receiving any denied content or +bypassing the AuthorizationKernel. + +## Jobs to be done + +- Inspect source health, ingest counts, refusal categories, and current Release + generation. +- Ask for context and resolve every rendered citation to Article, Revision, and + Fragment lineage. +- Preview an import's actual Fragments, then explicitly confirm or cancel it. +- Run a retrieval Hit Test whose result contains authorized hits only. +- View an Article policy and resolution rung; preview and explicitly confirm any + existing-Article change with a Policy Epoch advance. +- Inspect versioned provider/retrieval profiles and understand re-embedding impact + before requesting a change. +- Record answer feedback as evidence without acquiring promotion, activation, or + rollback authority. + +## Non-goals + +- No workflow canvas, bulk policy editing, feedback triage, golden-set curation, or + general administration framework. +- No separate UI process, JavaScript framework, frontend package manager, or client + build system. +- No container-, folder-, dataset-, or knowledge-base-level permissions. +- No new publication owner; release promotion remains the existing explicit + release-operator path. +- No rendering of refused candidates, their identifiers, existence, counts, or + original rank. + +## Positioning + +- **Category:** Local authenticated operator console. +- **Alternative today:** CLI plus `curl`, logs, and direct SQL inspection. +- **Why this surface:** It makes ContextEngine's evidence closure and refusal + behavior visible while preserving the public-seam architecture. diff --git a/context/RUBRIC.md b/context/RUBRIC.md new file mode 100644 index 00000000..1df6a1b0 --- /dev/null +++ b/context/RUBRIC.md @@ -0,0 +1,32 @@ +# ContextEngine Minimal UI review rubric + +## Design + +- [ ] Calm, precise, trustworthy, and austere in default, empty, and refusal states. +- [ ] Boldness is spent only on the evidence-flip interaction. +- [ ] Information hierarchy uses spacing and weight, not decorative color or cards. +- [ ] Hover, focus, active, disabled, empty, refusal, mixed-source partial, and + success are covered; browser-native navigation supplies full-page loading feedback. +- [ ] No gradients, glass, glow, fake dashboard, invented metrics, or dramatic errors. + +## Security and domain + +- [ ] Every content Block closes over exactly one Evidence reference. +- [ ] Hit Test cannot reveal denied candidates, existence, count, rank, or score gap. +- [ ] Empty and refusal states are unambiguous and non-enumerating. +- [ ] Policy and import mutations require exact preview plus explicit one-shot confirm. +- [ ] Feedback owns no activate/promote/publish/rollback entry point. +- [ ] UI imports only public HTTP/wire seams, never `engine/`. + +## UX flow + +- [ ] Happy, explicit-change, failure/recovery, and feedback journeys are walkable. +- [ ] Non-secret input survives safe failures; secrets and denied identifiers do not. +- [ ] Evidence disclosure is keyboard-usable and returns focus predictably. + +## Engineering + +- [ ] UI build/test targets and full repository verification are green. +- [ ] No frontend build system or additional process was introduced. +- [ ] CSS has no raw component colors; focus is visible; targets are at least 44px. +- [ ] Templates escape operator/content data by default. diff --git a/docs/decisions/0090-admit-a-co-resident-local-evidence-console.md b/docs/decisions/0090-admit-a-co-resident-local-evidence-console.md new file mode 100644 index 00000000..29949627 --- /dev/null +++ b/docs/decisions/0090-admit-a-co-resident-local-evidence-console.md @@ -0,0 +1,110 @@ +--- +name: adr-0090-admit-a-co-resident-local-evidence-console +version: "1.0.0" +description: > + Admit one explicitly authenticated server-rendered evidence console inside + the existing API process while preserving the public HTTP, Runtime, Control, + and release-publication authority boundaries. +--- + +# 0090. Admit a co-resident local evidence console + +- Status: accepted +- Date: 2026-07-30 +- Refines: ADR-0047, ADR-0068, ADR-0069, ADR-0076, ADR-0077 + +## Context + +M1 needs a first daily-driver delivery surface for seven operator jobs. The +maintainer selected server-side FastAPI templates in the existing API process; +a separate frontend process would add topology without isolation or performance +evidence. The surface still crosses two deliberately separate authorities. +Ask and Hit Test are ordinary current-Membership Runtime reads, while source +inspection, File import, and Article policy administration are Control work. +ADR-0069 originally kept the provisional local Control composition out of HTTP +because no authenticated administrative carrier then existed. + +A co-resident template is not permission to call engine objects directly. An +anonymous browser must not inherit the process's configured dogfood Principal, +and a current Membership must not become a Control operator. Hit Test is also a +dangerous place to expose pre-authorization rank or score gaps. Finally, +feedback must remain evidence only; the release operator remains the sole owner +of activation, rollback, and promotion. + +## Decision + +1. The Evidence Console is installed in the existing API process as Jinja2 + templates and static CSS. It adds no process, frontend runtime, package + manager, or network dependency. +2. A browser obtains a short-lived, `HttpOnly`, `SameSite=Strict` session proof + only after explicitly presenting the configured dogfood credential. The + proof is request-scoped browser state, contains no credential or trusted + identity claim, and is insufficient if the public API authenticator rejects + the underlying credential. Missing or expired proof renders a refusal. +3. Runtime jobs call `/v0/resolve` through an in-process ASGI HTTP client. Ask + opens every returned `citationOpenRef` through `OpenCitation` and requires + the replacement Package to close over the exact Article, Revision, Fragment, + and Policy Epoch before rendering a clean answer. +4. The Control-backed routes are typed HTTP carriers inside the same API ingress. + They remain schema-hidden so the immutable public OpenAPI v0 and generated SDK + do not drift without a separately reviewed contract version. Every request + requires the separate Control credential in authenticated transport metadata, + constructs one `TrustedControlCall` for exactly one `ControlOperation`, and + consumes that call in the Control gate before database or File-root work. + The credential is entered for the operation, is never reflected into HTML, + and is not stored in the browser session. +5. The Control operations admitted here are source-progress read, preview-bound + File import, Article-policy read, and preview-bound Article-policy change. + Exact current Membership is additionally rechecked for the acting audience. + No release-operator credential or `ContextLearning.promote` capability is + reachable from the UI. Feedback is not a Control operation: it persists only + through a Runtime-role function that rechecks the current Membership and exact + actor-owned ContextRun, and that role has no Release publication authority. +6. Hit Test renders only Blocks and Evidence from the post-Kernel + `ContextPackage`. Candidate rank evidence and optional ranker scores remain + outside the Package by ADR-0076; the UI states that score is not exposed by + the rank-free public contract instead of inventing one or observing a denied + score gap. +7. Import preview tokens bind the acting Membership, exact File bytes, compiler + version, and exact Fragment set. Article preview tokens bind the acting + Membership, expected policy version, expected Policy Epoch, and proposed + policy. Confirmation rechecks current Membership and Control authority, and + the database commits only that bound effect. +8. This is the local single-operator dogfood carrier, not a production + administrative ancestor. A second operator, remote production exposure, + delegated roles, or a durable browser-session service requires a new ADR. + +## Rationale + +Keeping presentation on the API's HTTP ingress seam makes the same authentication, +Kernel, provenance, and refusal behavior observable that another delivery client +would receive. Requiring an explicit browser proof prevents a loopback process +credential from becoming ambient anonymous authority. Requiring the independently +configured Control credential on each Control request preserves the read/write +plane split while permitting the maintainer-selected local UI to invoke already +bounded operations. + +The rank-free limitation is deliberate. Adding diagnostic score fields to the +only online deliverable would widen the Runtime contract and could reveal the +shape of refused candidates. An explicit unavailable label is less convenient +but truthful and safe. + +## Consequences + +- OpenAPI v0 and the generated SDK remain byte-for-byte frozen. The typed local + backing carriers and the HTML presentation are both schema-hidden. +- The API process may hold the Control database adapter only when the existing + explicit local operator configuration is present. No Control authority exists + by default. +- Operators re-enter the Control credential for Control-backed jobs. The cost is + intentional: the session proves a reader, not an administrator. +- The Hit Test cannot display numeric scores until an accepted Runtime contract + carries post-authorization scoring without denied-rank leakage. + +## Revisit trigger + +Revisit before production or non-loopback exposure, a second operator identity, +delegated Control roles, a separate UI process, durable session storage, or any +numeric Hit Test score. Any revision must keep anonymous requests fail-closed, +retain one-operation Control calls, preserve OpenCitation reauthorization, and +keep release publication authority out of feedback and Control. diff --git a/docs/decisions/README.md b/docs/decisions/README.md index 5227d741..b0d7de2a 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -62,6 +62,7 @@ kernel, capability separation, and publication visibility model. | Private delivery ingress | [0045 — Redeem private delivery evidence at ingress](0045-redeem-private-delivery-evidence-at-ingress.md) | One digest-only service/request/asker/audience/epoch-bound DeliveryEvidenceRef constructs private TrustedDeliveryContext inside the current UserActor transaction before content work | Raw trusted delivery facts on the wire, bearer persistence, application-role minting/table reads, alternate Runtime paths, or claiming later M2 carriers | | Exact Package egress | [0046 — Bind egress to one exact Package hop](0046-bind-egress-to-one-exact-package-hop.md) | One digest-only grant binds one exact audience-bound Package to one model or channel preflight hop and redeems atomically | Treating Package construction as disclosure authority, arbitrary content at egress, cross-hop reuse, or bypassing final policy | | Public OpenAPI v0 | [0047 — Freeze OpenAPI v0 through one Runtime path](0047-freeze-openapi-v0-through-one-runtime-path.md) | One public `/v0/resolve` schema and a hidden provisional v1 bridge share the same sealed Runtime; Package release lineage is read-only from the Learning-published active manifest | Two authorization compositions, caller-authored release facts, Runtime publication/fallback, or in-place mutation of historical snapshots | +| Local evidence console | [0090 — Admit a co-resident local evidence console](0090-admit-a-co-resident-local-evidence-console.md) | One explicitly authenticated server-rendered UI calls typed HTTP carriers inside the API ingress while frozen OpenAPI v0 stays unchanged; Control jobs additionally consume one separate exact-operation Control call | Anonymous process-identity inheritance, Membership-as-Control, direct UI-to-engine calls, denied rank/score leakage, or feedback publication authority | | Generated TypeScript SDK | [0048 — Generate the TypeScript SDK behind a closed facade](0048-generate-typescript-sdk-behind-a-closed-facade.md) | OpenAPI v0 generates the semantic contract and internal fetch client; a package export map exposes only a metadata-safe facade and the contract checksum | Handwritten wire schemas, raw generated transport exports, arbitrary caller headers, floating generator/runtime versions, or claiming inactive capability redemption | | Citation open | [0051 — Reauthorize opaque citation opens](0051-reauthorize-opaque-citation-opens.md) | Digest-only multi-use locators recover only content-free target lineage; every open uses a current UserActor and the sealed Kernel to produce a replacement Package | Bearer authority, prior-decision reuse, source URLs, locator consumption on denial, or content before exact reauthorization | | Private model egress | [0052 — Gate model generation by one authorized Package](0052-gate-model-generation-by-package.md) | One nominal SDK-Package input and exact one-shot model grant gate the deterministic private ModelGateway, bounded answer, Package-subset citations, and digest-only audit | Arbitrary prompt context, multiple Packages, direct Gateway invocation, invented citations, effect intents, or unaudited answer release | @@ -197,3 +198,4 @@ touched: - [0087 — Bind downgrade-guard evidence to its own revision](0087-bind-downgrade-guard-evidence-to-its-own-revision.md) - [0088 — Bind local consumers to fresh evidence-bearing Packages](0088-bind-local-consumers-to-fresh-evidence-bearing-packages.md) - [0089 — Bound live Feishu to the existing private Bot composition](0089-bound-live-feishu-to-the-existing-private-bot-composition.md) +- [0090 — Admit a co-resident local evidence console](0090-admit-a-co-resident-local-evidence-console.md) diff --git a/engine/control/__init__.py b/engine/control/__init__.py index 6b02d05f..2fb76420 100644 --- a/engine/control/__init__.py +++ b/engine/control/__init__.py @@ -103,6 +103,7 @@ FileSourceStatus, PendingFileChangeSchedule, ) +from engine.control.minimal_ui import MinimalUiControlGate from engine.control.module import ( ArticlePolicyDefaultStorePort, BulkArticlePolicyStorePort, @@ -117,6 +118,7 @@ "MAX_CONFIGURED_FILE_CHANGE_BASELINE_SIZE", "MAX_FILE_CHANGE_BASELINE_SIZE", "MAX_FILE_CHANGE_PAGE_SIZE", + "MinimalUiControlGate", "AcceptedChangePage", "ActivateFileChangeFeed", "ActivateFileDeleteObservations", diff --git a/engine/control/authority.py b/engine/control/authority.py index 9230ddac..8c5dd920 100644 --- a/engine/control/authority.py +++ b/engine/control/authority.py @@ -22,12 +22,14 @@ class ControlOperation(StrEnum): ACCEPT_FILE_CHANGE_PAGE = "accept_file_change_page" ACTIVATE_FILE_CHANGE_FEED = "activate_file_change_feed" ACTIVATE_FILE_DELETE_OBSERVATIONS = "activate_file_delete_observations" + CHANGE_ARTICLE_POLICY = "change_article_policy" COMMIT_BULK_ARTICLE_POLICY_CHANGE = "commit_bulk_article_policy_change" EXECUTE_FILE_DELETE_OBSERVATION = "execute_file_delete_observation" IMPORT_FILE = "import_file" OFFBOARD_FILE_SOURCE = "offboard_file_source" PREVIEW_BULK_ARTICLE_POLICY_CHANGE = "preview_bulk_article_policy_change" REGISTER_SOURCE = "register_source" + READ_ARTICLE_POLICY = "read_article_policy" READ_SOURCE = "read_source" READ_SOURCE_PROGRESS = "read_source_progress" SCHEDULE_FILE_CHANGE_PAGE = "schedule_file_change_page" diff --git a/engine/control/minimal_ui.py b/engine/control/minimal_ui.py new file mode 100644 index 00000000..da1b86aa --- /dev/null +++ b/engine/control/minimal_ui.py @@ -0,0 +1,50 @@ +"""Exact Control-authority gate for the co-resident M1 operator carrier.""" + +from __future__ import annotations + +from collections.abc import Callable +from datetime import datetime +from uuid import UUID + +from engine.control.authority import ( + ControlOperation, + ControlOperatorAuthenticationRejected, + ControlOperatorAuthority, + TrustedControlCall, + _validate_and_consume_control_call, +) + + +class MinimalUiControlGate: + """Consume one authority-constructed call before any UI Control database work.""" + + __slots__ = ("_authority", "_clock") + + def __init__( + self, + authority: ControlOperatorAuthority, + *, + clock: Callable[[], datetime], + ) -> None: + if type(authority) is not ControlOperatorAuthority or not callable(clock): + raise TypeError("minimal UI Control authority is unavailable") + self._authority = authority + self._clock = clock + + def consume( + self, + call: TrustedControlCall, + *, + organization_id: UUID, + operation: ControlOperation, + ) -> None: + if type(organization_id) is not UUID or type(operation) is not ControlOperation: + raise ControlOperatorAuthenticationRejected + _validate_and_consume_control_call( + call, + authority=self._authority, + expected_operation=operation, + checked_at=self._clock(), + ) + if call.organization_id != organization_id: + raise ControlOperatorAuthenticationRejected diff --git a/engine/persistence/file_imports.py b/engine/persistence/file_imports.py index 9ce5c38f..acba3349 100644 --- a/engine/persistence/file_imports.py +++ b/engine/persistence/file_imports.py @@ -185,6 +185,9 @@ class _RedeemedFileImport: acquisition_id: UUID expected_content_sha256: str | None expected_content_length: int | None + ui_preview_digest: str | None + expected_fragment_digest: str | None + compiler_config_version: str | None def _rejection(token: WorkerLeaseToken) -> WorkNotAvailable: @@ -312,6 +315,30 @@ def run(self, redemption: FileImportLeaseRedemption) -> PublishedFileImport: with suppress(WorkNotAvailable): self._fail(redemption.token, claims) raise FileImportRefused("File import is unavailable") + if redeemed.compiler_config_version is not None: + fragments = [ + { + "fragmentRef": fragment.fragment_ref, + "text": fragment.contextual_text, + } + for fragment in outcome.fragments + ] + fragment_digest = sha256( + json.dumps( + fragments, + ensure_ascii=False, + separators=(",", ":"), + sort_keys=True, + ).encode("utf-8") + ).hexdigest() + if ( + redeemed.ui_preview_digest is None + or redeemed.compiler_config_version != self._config.version + or redeemed.expected_fragment_digest != fragment_digest + ): + with suppress(WorkNotAvailable): + self._fail(redemption.token, claims) + raise FileImportRefused("File import is unavailable") try: return self._publish(redemption.token, claims, redeemed, outcome) except FileImportInterrupted: @@ -340,7 +367,13 @@ def _redeem( to_jsonb(redeemed)->>'expected_content_sha256' AS expected_content_sha256, (to_jsonb(redeemed)->>'expected_content_length') - ::bigint AS expected_content_length + ::bigint AS expected_content_length, + to_jsonb(redeemed)->>'ui_preview_digest' + AS ui_preview_digest, + to_jsonb(redeemed)->>'expected_fragment_digest' + AS expected_fragment_digest, + to_jsonb(redeemed)->>'compiler_config_version' + AS compiler_config_version FROM public.context_worker_redeem_file_import( :organization_id, :job_id, :service_principal_id, :source_ref, :lease_generation, @@ -372,6 +405,9 @@ def _redeem( acquisition_id=row.acquisition_id, expected_content_sha256=expected_content_sha256, expected_content_length=expected_content_length, + ui_preview_digest=row.ui_preview_digest, + expected_fragment_digest=row.expected_fragment_digest, + compiler_config_version=row.compiler_config_version, ) except WorkNotAvailable: raise diff --git a/engine/persistence/schema_security_manifest.yaml b/engine/persistence/schema_security_manifest.yaml index 594f5f66..d98a78b8 100644 --- a/engine/persistence/schema_security_manifest.yaml +++ b/engine/persistence/schema_security_manifest.yaml @@ -967,6 +967,22 @@ ], "using": "true", "withCheck": "true" + }, + { + "name": "membership_ui_access_definer_select", + "command": "SELECT", + "roles": [ + "context_engine_access_policy_definer" + ], + "using": "organization_id = NULLIF(current_setting('app.ui_actor_organization_id', true), '')::uuid AND user_id = NULLIF(current_setting('app.ui_actor_user_id', true), '')::uuid AND membership_id = NULLIF(current_setting('app.ui_actor_membership_id', true), '')::uuid AND membership_version = NULLIF(current_setting('app.ui_actor_membership_version', true), '')::bigint AND current_setting('app.ui_actor_mode', true) = 'article_policy_change'" + }, + { + "name": "membership_ui_feedback_definer_select", + "command": "SELECT", + "roles": [ + "context_engine_context_run_reader_definer" + ], + "using": "organization_id = NULLIF(current_setting('app.ui_feedback_organization_id', true), '')::uuid AND user_id = NULLIF(current_setting('app.ui_feedback_user_id', true), '')::uuid AND membership_id = NULLIF(current_setting('app.ui_feedback_membership_id', true), '')::uuid AND membership_version = NULLIF(current_setting('app.ui_feedback_membership_version', true), '')::bigint AND current_setting('app.ui_feedback_mode', true) = 'capture'" } ] }, @@ -996,6 +1012,12 @@ "context_engine_file_dispatch_definer": [ "SELECT", "UPDATE status, valid_from, valid_until" + ], + "context_engine_access_policy_definer": [ + "SELECT" + ], + "context_engine_context_run_reader_definer": [ + "SELECT" ] }, "partitions": [], @@ -2913,6 +2935,14 @@ ], "using": "true", "withCheck": "true" + }, + { + "name": "context_run_ui_feedback_definer_select", + "command": "SELECT", + "roles": [ + "context_engine_context_run_reader_definer" + ], + "using": "context_run.organization_id = NULLIF(current_setting('app.ui_feedback_organization_id', true), '')::uuid AND context_run.run_ref = current_setting('app.ui_feedback_run_ref', true) AND context_run.user_id = NULLIF(current_setting('app.ui_feedback_user_id', true), '')::uuid AND context_run.membership_id = NULLIF(current_setting('app.ui_feedback_membership_id', true), '')::uuid AND context_run.membership_version = NULLIF(current_setting('app.ui_feedback_membership_version', true), '')::bigint AND context_run.principal_ref = current_setting('app.ui_feedback_principal_ref', true) AND current_setting('app.ui_feedback_mode', true) = 'capture'" } ] }, @@ -2953,6 +2983,159 @@ "PG-TRACE-REDACTION-012" ] }, + { + "name": "context_feedback", + "classification": "tenant_owned", + "nonOwnerEvidence": { + "evidenceId": "PG-TRACE-REDACTION-012", + "selector": { + "table": "context_feedback" + } + }, + "purpose": "Evidence-only answer feedback bound to one exact same-Organization ContextRun and current Membership; it has no Release publication authority", + "organizationColumn": "organization_id", + "organizationInclusiveKeys": [ + { + "name": "pk_context_feedback", + "kind": "primary_key", + "columns": [ + "organization_id", + "feedback_ref" + ] + }, + { + "name": "uq_context_feedback_actor_run", + "kind": "unique", + "columns": [ + "organization_id", + "run_ref", + "membership_id", + "membership_version" + ] + } + ], + "foreignKeys": [ + { + "name": "fk_context_feedback_membership_same_organization", + "columns": [ + "organization_id", + "membership_id", + "membership_version" + ], + "references": { + "table": "membership", + "columns": [ + "organization_id", + "membership_id", + "membership_version" + ] + }, + "onDelete": "RESTRICT" + }, + { + "name": "fk_context_feedback_run_same_organization", + "columns": [ + "organization_id", + "run_ref" + ], + "references": { + "table": "context_run", + "columns": [ + "organization_id", + "run_ref" + ] + }, + "onDelete": "RESTRICT" + } + ], + "checkConstraints": [ + { + "name": "ck_context_feedback_membership_version", + "expression": "membership_version BETWEEN 1 AND 9223372036854775807" + }, + { + "name": "ck_context_feedback_note", + "expression": "note IS NULL OR (btrim(note) <> '' AND char_length(note) <= 1000)" + }, + { + "name": "ck_context_feedback_rating", + "expression": "rating IN ('helpful', 'not_helpful')" + }, + { + "name": "ck_context_feedback_ref", + "expression": "feedback_ref ~ '^fb_[0-9a-f]{64}$'" + }, + { + "name": "ck_context_feedback_refs", + "expression": "btrim(run_ref) <> '' AND char_length(run_ref) <= 256 AND btrim(principal_ref) <> ''" + } + ], + "rowLevelSecurity": { + "enabled": true, + "forced": true, + "policies": [ + { + "name": "context_feedback_migrator_administration", + "command": "ALL", + "roles": [ + "context_engine_migrator" + ], + "using": "true", + "withCheck": "true" + }, + { + "name": "context_feedback_ui_definer_insert", + "command": "INSERT", + "roles": [ + "context_engine_context_run_reader_definer" + ], + "withCheck": "context_feedback.organization_id = NULLIF(current_setting('app.ui_feedback_organization_id', true), '')::uuid AND context_feedback.run_ref = current_setting('app.ui_feedback_run_ref', true) AND context_feedback.membership_id = NULLIF(current_setting('app.ui_feedback_membership_id', true), '')::uuid AND context_feedback.membership_version = NULLIF(current_setting('app.ui_feedback_membership_version', true), '')::bigint AND current_setting('app.ui_feedback_mode', true) = 'capture'" + }, + { + "name": "context_feedback_ui_definer_select", + "command": "SELECT", + "roles": [ + "context_engine_context_run_reader_definer" + ], + "using": "context_feedback.organization_id = NULLIF(current_setting('app.ui_feedback_organization_id', true), '')::uuid AND context_feedback.run_ref = current_setting('app.ui_feedback_run_ref', true) AND context_feedback.membership_id = NULLIF(current_setting('app.ui_feedback_membership_id', true), '')::uuid AND context_feedback.membership_version = NULLIF(current_setting('app.ui_feedback_membership_version', true), '')::bigint AND current_setting('app.ui_feedback_mode', true) = 'capture'" + } + ] + }, + "functionOnlyMutation": { + "databaseFunction": "context_runtime_capture_context_feedback", + "role": "context_engine_runtime", + "definerRole": "context_engine_context_run_reader_definer", + "directTableMutationAllowed": false, + "releasePublicationAuthorityAllowed": false + }, + "permittedOperations": { + "context_engine_control": [], + "context_engine_context_run_reader_definer": [ + "SELECT", + "INSERT" + ], + "context_engine_learning": [], + "context_engine_release_operator": [], + "context_engine_runtime": [ + "EXECUTE context_runtime_capture_context_feedback" + ], + "context_engine_worker": [] + }, + "partitions": [], + "securityInvariantIds": [ + "TENANT-OWNERSHIP-001", + "TENANT-FK-002", + "RLS-FAIL-CLOSED-003", + "TRACE-REDACTION-012", + "RELEASE-OWNER-019" + ], + "negativeTestIds": [ + "DB-001", + "DB-004", + "DB-008", + "PG-TRACE-REDACTION-012" + ] + }, { "name": "context_run_operator_read_ticket", "classification": "tenant_owned", @@ -5956,7 +6139,7 @@ "table": "article_explicit_policy_setting" } }, - "purpose": "Trusted explicit first-ingest setting for one not-yet-created Article identity", + "purpose": "Trusted explicit first-ingest setting or preview-confirmed policy edit for one Article identity", "organizationColumn": "organization_id", "organizationInclusiveKeys": [ { @@ -5997,6 +6180,14 @@ "enabled": true, "forced": true, "policies": [ + { + "name": "article_explicit_policy_setting_access_definer_insert", + "command": "INSERT", + "roles": [ + "context_engine_access_policy_definer" + ], + "withCheck": "organization_id = NULLIF(current_setting('app.organization_id', true), '')::uuid" + }, { "name": "article_explicit_policy_setting_migrator_administration", "command": "ALL", @@ -6013,14 +6204,36 @@ "context_engine_access_policy_definer" ], "using": "(organization_id = (NULLIF(current_setting('app.organization_id'::text, true), ''::text))::uuid)" + }, + { + "name": "article_explicit_policy_setting_access_definer_update", + "command": "UPDATE", + "roles": [ + "context_engine_access_policy_definer" + ], + "using": "organization_id = NULLIF(current_setting('app.organization_id', true), '')::uuid", + "withCheck": "organization_id = NULLIF(current_setting('app.organization_id', true), '')::uuid" } ] }, + "functionOnlyMutation": { + "databaseFunctions": [ + "context_fix_article_access_policy", + "context_control_change_article_policy" + ], + "definerRole": "context_engine_access_policy_definer", + "directTableMutationAllowed": false + }, "permittedOperations": { "context_engine_access_policy_definer": [ - "SELECT" + "SELECT", + "INSERT", + "UPDATE" + ], + "context_engine_control": [ + "EXECUTE context_control_read_article_policy", + "EXECUTE context_control_change_article_policy" ], - "context_engine_control": [], "context_engine_runtime": [], "context_engine_worker": [] }, @@ -7553,7 +7766,7 @@ "table": "file_acquisition" } }, - "purpose": "Immutable trusted request to acquire one Markdown filename from one registered File Source version", + "purpose": "Immutable trusted request to acquire one Markdown filename from one registered File Source version, optionally bound to an exact UI fragment preview", "organizationColumn": "organization_id", "organizationInclusiveKeys": [ { @@ -7674,9 +7887,9 @@ "name": "ck_file_acquisition_request_digest", "expression": "request_digest ~ '^[0-9a-f]{64}$'" }, - { - "name": "ck_file_acquisition_change_observation", - "expression": "accepted page ref, ordinal, raw SHA-256, and raw length are either all absent for manual import or all valid and present" + { + "name": "ck_file_acquisition_change_observation", + "expression": "accepted change-page identity, UI preview identity, or legacy manual-import fields form exactly one valid acquisition shape" } ], "rowLevelSecurity": { @@ -7730,6 +7943,7 @@ "functionOnlyMutation": { "databaseFunctions": [ "context_control_prepare_file_import", + "context_control_prepare_exact_file_import", "context_control_schedule_file_change_page" ], "role": "context_engine_control", @@ -7739,6 +7953,7 @@ "permittedOperations": { "context_engine_control": [ "EXECUTE context_control_prepare_file_import", + "EXECUTE context_control_prepare_exact_file_import", "EXECUTE context_control_schedule_file_change_page", "EXECUTE context_control_read_pending_file_change_schedules", "EXECUTE context_control_read_file_source_status" diff --git a/eval/catalogs/security-catalog.schema.json b/eval/catalogs/security-catalog.schema.json index 4983d00f..c856461b 100644 --- a/eval/catalogs/security-catalog.schema.json +++ b/eval/catalogs/security-catalog.schema.json @@ -1086,7 +1086,7 @@ { "id": "RUNTIME-DOGFOOD-AUTH-102", "surface": "tests/integration/test_dogfood_runtime_activation.py::test_dogfood_secret_and_membership_fail_closed_without_secret_retention", - "oracle": "Absent, wrong, or truncated secrets and revoked or expired configured Membership authority return the same generic authentication failure; the secret is absent from responses, logs, ContextRun, and DecisionAudit." + "oracle": "Absent, wrong, or truncated secrets and revoked or expired configured Membership authority return the same generic authentication failure; the secret is absent from responses, logs, ContextRun, and DecisionAudit. A successful dogfood resolve issues one private File CitationOpenRef that reopens only through a second current-Membership sealed Runtime resolve." } ], "deferredEvidence": [ @@ -1105,7 +1105,6 @@ "network exposure beyond the maintainer machine", "group or public audience", "Continue", - "dogfood OpenCitation", "hybrid retrieval", "external query embedding", "non-File providers" @@ -1116,7 +1115,7 @@ "const": { "issueRef": "#102", "invariantRef": "INDEX-NOT-AUTHORITY-005", - "carrier": "loopback-only File pgvector dogfood Acquire delivery", + "carrier": "loopback-only File pgvector dogfood Acquire and private OpenCitation delivery", "status": "active_fail_closed", "policyEpochScope": "organization-v0", "controlBoundary": "current UserActor transaction -> database-derived exact EffectiveScope -> pre-LIMIT pgvector scope reduction -> CandidateRef -> sealed AuthorizationKernel -> AuthorizedProjection -> ContextPackage", @@ -1148,7 +1147,6 @@ "network exposure beyond the maintainer machine", "group or public audience", "Continue", - "dogfood OpenCitation", "hybrid retrieval", "external query embedding", "non-File providers" @@ -1613,7 +1611,7 @@ "autonomous first-attempt dispatch of explicit scheduled File upserts", "bounded autonomous reclaim of expired scheduled File upserts", "loopback-only single-Membership dogfood HTTP authentication", - "loopback-only File pgvector dogfood Acquire delivery", + "loopback-only File pgvector dogfood Acquire and private OpenCitation delivery", "explicit local-only Control and release operator authentication" ] }, diff --git a/eval/catalogs/security-invariants.yaml b/eval/catalogs/security-invariants.yaml index 1adc2f20..0d668c1d 100644 --- a/eval/catalogs/security-invariants.yaml +++ b/eval/catalogs/security-invariants.yaml @@ -59,9 +59,10 @@ "docs/decisions/0059-dispatch-scheduled-file-imports-through-exact-leases.md", "docs/decisions/0060-reclaim-expired-file-imports-with-bounded-retries.md", "docs/decisions/0068-activate-loopback-dogfood-runtime.md", - "docs/decisions/0069-admit-an-explicit-local-operator-composition.md" + "docs/decisions/0069-admit-an-explicit-local-operator-composition.md", + "docs/decisions/0083-admit-a-co-resident-local-evidence-console.md" ], - "reconciliation": "Issue #2 fixes the product and testing decisions, issue #5 requires exactly fifteen release invariants and twelve canonical acceptance fixtures, and ADR-0019 resolves the later nineteen-label prose expansion without weakening any safeguard. Issue #15 activates only Organization-level next-request resolve(Acquire) revocation evidence under REVOCATION-006: at that activation, PG-REVOCATION-006, RUN-006, and CACHE-002 are active while BLOB-002 and Continue, citation, Policy-Epoch-bound WorkerLease, production ContextAccessTicket/ActionTicket, audit, outbox, cleanup, finer-epoch, UI, and external-admin carriers remain future or NOT_ACTIVE; later issue records are authoritative for subsequently activated carriers. Issue #16 activates only the M0 refusal gate for unavailable Continue, profile-disabled OpenCitation, and server-owned unavailable Acquire plans: at that activation its real continuation, profile-enabled citation, federated/source-native, and File carriers remain future, while its Runtime and HTTP refusal surfaces prove generic outcomes before content I/O; Issue #69 later activates the private/direct File profile-enabled citation carrier. Issue #17 activates only the signed one-shot persistent no-op durable-job WorkerLease subcarrier under WORKER-LEASE-007. It binds one exact worker audience but no end-user delivery audience or Policy Epoch, and proves only LEASE-SIGNING-017, PG-WORKER-LEASE-NOOP-017, and WORKER-LEASE-REPLAY-007; Source, Resource, Revision, Policy Epoch, end-user delivery audience, idempotency, generation, business mutation, outbox, File publication, and the full ACCEPT-008 matrix remain deferred or NOT_ACTIVE. Issue #18 activates only distinct signed synthetic ContextAccessTicket Provider-read and ActionTicket no-op channel-action subcarriers under ACTION-SEPARATION-014, with current Organization-v0 Policy Epoch validation. TICKET-AUDIENCE-018 and PG-TICKET-EPOCH-018 do not activate production ContextProvider integration, ContextRuntime ticket integration, BotDelivery, full M2 ActionPlane.prepare/perform, a real Sender or external effect, payload/destination/approval/idempotency binding, durable one-shot/replay/reconciliation, or full ACCEPT-012 PASS; those remain future or NOT_ACTIVE. Issue #19 activates only the current Acquire authorized-only ContextRun and restricted delivered-empty DecisionAudit subcarrier under TRACE-REDACTION-012. DIGEST-019, RUN-LINEAGE-019, AUTHORIZED-RUN-019, and PG-TRACE-REDACTION-012 prove deterministic Package and Organization-bound query digests, retained-UserActor-transaction persistence, decisionRef resolution, redaction, and short-lived exact-Organization operator ticket reads with no application-role table access; the supported reader commits deletion before returning, while a direct caller rollback is not claimed as durable exactly-once redemption. Raw query retention, full ContextPackage body retention, unauthenticated transport failures as ContextRuns, cross-Organization analytics, and general observability redaction remain NOT_ACTIVE. Issue #48 activates only the current ACCEPT-002 authenticated HTTP Acquire Membership field-projection carrier under SCOPE-INTERSECTION-004, INDEX-NOT-AUTHORITY-005, and TRACE-REDACTION-012. PROP-FIELD-PROJECTION-048, PG-FIELD-PROJECTION-048, and HTTP-ACCEPT-002-048 bind one current Membership/version field ceiling to same-transaction FORCE-RLS reduction, the sealed AuthorizationKernel, AuthorizedProjection and Evidence integrity, and authorized-only ContextRun/audit persistence. General permission DSLs, caller-authored projection lists, CandidateRef or index field authority, production Provider/source-native ACL negotiation, Supply publication, File/Base field ACL, typed fields, Continue, and Issue #20 runner substitution remain future or NOT_ACTIVE; Issue #69 later activates private/direct File OpenCitation through the same field-projection gates. Issue #63 activates only the digest-only private authenticated HTTP Acquire DeliveryEvidenceRef carrier under TRANSPORT-UNTRUSTED-008. PROP-DELIVERY-EVIDENCE-063, PG-DELIVERY-EVIDENCE-063, HTTP-DELIVERY-EVIDENCE-063, and FILE-DELIVERY-EVIDENCE-063 prove exact service/request/Organization/asker/Membership-version/destination/consumer/purpose/audience/epoch/lifetime binding, stable identical retry identity, role isolation, expiry cleanup, pre-content generic rejection, and one File-backed sealed Runtime delivery. Group AudienceSnapshot, group/public DeliveryEvidenceRef, production ModelGateway, ActionPlane, and the BotDelivery application remain future or NOT_ACTIVE; Issues #64, #66, #69, and #70 later activate the frozen OpenAPI, generated TypeScript SDK, private/direct OpenCitation, and deterministic private model-egress carriers. Issue #65 activates only one opaque digest-only model or channel EgressGrant after final Package policy, exact atomic PostgreSQL redemption and restricted audit, nominal BotDelivery inputs, and deterministic network-free ModelGateway or Sender-preflight spies under EGRESS-011. PROP-EGRESS-011, PG-EGRESS-011, and RUNTIME-EGRESS-011 prove exact Package/Organization/purpose/audience/epoch/hop/profile/lifetime binding and zero additional bytes on replay. Real model/provider calls, a real Sender or channel write, ActionTicket effects, group AudienceSnapshot revalidation, and the BotDelivery application process remain future or NOT_ACTIVE; Issue #64 later activates the generated SDK consumer and Issue #70 later activates the deterministic private TypeScript ModelGateway. Issue #66 activates the frozen public POST /v0/resolve OpenAPI carrier under TRANSPORT-UNTRUSTED-008. OPENAPI-CONTRACT-066, OPENAPI-BREAKING-066, HTTP-V0-066, and PG-RUNTIME-RELEASE-066 prove one public closed operation, deterministic immutable snapshot and breaking-change refusal, a hidden v1 bridge through the same handler and sealed Runtime path, and exact read-only observation of the active Learning-promoted release with fail-closed missing-release behavior before content work. A production BotDelivery caller, Continue redemption, MCP, group AudienceSnapshot, and external effects remain future or NOT_ACTIVE; Issues #64 and #69 later activate the generated TypeScript SDK and private/direct OpenCitation redemption through this frozen operation. Issue #64 activates only the packaged generated TypeScript POST /v0/resolve client under TRANSPORT-UNTRUSTED-008. SDK-CONTRACT-064 and SDK-LIVE-FILE-064 prove deterministic pinned generation, strict closed types, a narrow export map and metadata-only facade, installable tarball consumption, and one real PostgreSQL/File-backed Acquire through CandidateRef, AuthorizationKernel, AuthorizedProjection, ContextPackage, and opaque model egress grant. Issue #69 later extends SDK-LIVE-FILE-064 with a successful private/direct File OpenCitation through a second request-bound DeliveryEvidenceRef; Issue #70 extends the installed SDK fixture into deterministic Package-bound model generation; generated Continue remains generic unavailable. External package publication, production provider access, MCP, group AudienceSnapshot, real Continue redemption, and external effects remain future or NOT_ACTIVE. Issue #67 activates only private ActionPlane.prepare for create-placeholder, finalize-reply, and private-follow-up operation-specific tickets under ACTION-SEPARATION-014. PG-ACTION-PREPARE-067 proves exact current delivery, Organization, destination, audience, source, payload, approval, epoch, lifetime, and idempotency binding under a dedicated non-owner PostgreSQL role with digest-only FORCE-RLS persistence and zero effects. Issue #68 activates private ActionPlane.perform only through a deterministic Sender twin. PG-ACTION-PERFORM-068 proves one pre-Sender current-authority validation, one provider-attempt identity, immutable receipt replay, zero-effect ticket/payload mutation and stale-audience refusal, same-label cross-Organization isolation, and monotonic applied/rejected reconciliation including crash interleavings. Real provider or channel network effects, group AudienceSnapshot, compensation/delete, production BotDelivery orchestration, and the full ACCEPT-012 pass remain future or NOT_ACTIVE. The canonical set is IDs 001 through 012, 014, 015, and 019: CACHE-SCOPE-013 remains a preregistered conditional extension; AUDIENCE-016 is absorbed by SCOPE-INTERSECTION-004 and EGRESS-011; ACL-PROOF-017 is absorbed by INDEX-NOT-AUTHORITY-005 and REVOCATION-006; DELIVERY-EVIDENCE-018 is absorbed by TRANSPORT-UNTRUSTED-008. ACCEPT-001 through ACCEPT-012 follow ADR-0019's category order. Protected-asset references A-01 through A-08 refer, in order, to the eight bullets in the threat model's Protected assets section. Every expectedEvidence value below is a stable planned case identifier, not a claim that the case ran or passed; only an exact activation record upgrades named evidence, while fixture carrier status and the explicit M0 oracle preserve every other accepted-versus-active distinction. Issue #69 activates private/direct File CitationOpenRef issuance and OpenCitation under CITATION-AUTH-010: digest-only multi-use locators reveal only prior Package/Evidence and Fragment location lineage, every open obtains a current UserActor and trusted delivery context then traverses CandidateRef, AuthorizationKernel, AuthorizedProjection, a replacement ContextPackage, EgressGrant, ContextRun, and restricted DecisionAudit. PG-CITATION-AUTH-010, RUNTIME-CITATION-AUTH-010, and SDK-LIVE-FILE-064 prove A/B/A reauthorization, non-consumption on denial, database-clock expiry, cross-kind and cross-Organization opacity, and the generated SDK carrier. Group/public AudienceSnapshot, non-File providers, raw source URL locators, and Continue remain future or NOT_ACTIVE. Issue #70 activates only the private deterministic TypeScript ModelGateway under EGRESS-011. TS-MODEL-EGRESS-070, SDK-MODEL-EGRESS-070, and PG-MODEL-EGRESS-070 prove one current Package, exact grant redemption, closed provider input, bounded Package-subset citations, replay zero bytes, and digest-only retained audit. Real providers, streaming, group AudienceSnapshot, model-authored ActionPlane authority, and external effects remain future or NOT_ACTIVE. Issue #71 activates the complete private File-backed deterministic-twin BotDelivery carrier under TRANSPORT-UNTRUSTED-008, CITATION-AUTH-010, EGRESS-011, and ACTION-SEPARATION-014. TS-PRIVATE-BOT-FLOW-071, SDK-PRIVATE-BOT-FLOW-071, and PG-PRIVATE-BOT-FLOW-071 prove the independent TypeScript process and import boundary, exact verified private event binding, opaque DeliveryEvidenceRef transport, installed generated-SDK HTTP resolve, sealed File Package path, controlled model generation, distinct placeholder/final ActionPlane effects, digest-only DeliveryReceipt audit, citation reopening, and the composed wrong-binding oracles. The historical FIXTURE-ACCEPT-012 continues to prove the M0 unavailable cross-capability baseline separately. Live Feishu/model/Sender network carriers, group/public delivery, compensation/delete, Continue, and MCP remain NOT_ACTIVE. Issue #81 activates deterministic shallow File readChanges and whole-page acknowledgement under WORKER-LEASE-007. PG-FILE-CHANGE-ACTIVATE-081, PG-FILE-CHANGE-PAGE-081, and PG-FILE-CHANGE-DENY-081 prove immutable v3 activation, provider-authenticated content-free paging, exact replay, predecessor ordering, post-commit cursor issuance, FORCE-RLS tenant isolation, source/version invalidation, and zero implicit job or publication effect. Automatic scheduling, deletion execution, recursive discovery, full resync, and Runtime authority from cursor or checkpoint metadata remain NOT_ACTIVE. Issue #83 activates only explicit accepted File page scheduling through the existing file.import acquisition, WorkerLease, and publication path under WORKER-LEASE-007. PG-FILE-CHANGE-SCHEDULE-083, PG-FILE-CHANGE-SCHEDULE-DENY-083, and PG-FILE-CHANGE-SUPERSESSION-083 prove whole-page atomicity, exact job replay, explicit current FileImportAudience and receiver validation, immutable raw-byte observation lineage, pre-compiler drift refusal, and scan-epoch fences before content read and visible publication with zero stale publication effect. Autonomous polling, implicit audience inheritance, deletion execution, automatic retry/reclaim, dead-letter handling, full resync, and Runtime authority from provider checkpoints remain NOT_ACTIVE. Issue #85 activates bounded File delete observations only under WORKER-LEASE-007. PG-FILE-DELETE-DETECT-085, PG-FILE-DELETE-PAGE-085, and PG-FILE-DELETE-NO-EFFECT-085 prove exact latest-complete same-SourceVersion baseline binding, stable shallow diff, immutable canonical persistence/replay, forged/incomplete/stale/cross-Organization refusal, mixed-page scheduling refusal, and zero tombstone, Policy Epoch, cleanup, watermark, or generated-SDK Runtime visibility effect. At that activation, deletion execution remains NOT_ACTIVE; autonomous polling, retry/reclaim/dead-letter, full resync, recursive scan, and Runtime authority from baseline or delete metadata remain NOT_ACTIVE. Issue #87 later activates only exact trusted current File delete execution through the existing #28 tombstone authority under REVOCATION-006. PG-FILE-DELETE-EXECUTE-087, PG-FILE-DELETE-REPLAY-087, and HTTP-FILE-DELETE-INVISIBLE-087 prove current complete-scan revalidation, server-derived effect identity, atomic tombstone/epoch/cleanup/binding, exact replay, mismatch rollback, and immediate sealed-Runtime invisibility. Issue #89 later activates only the exact upsert projection of one current mixed v4 page under WORKER-LEASE-007. PG-FILE-MIXED-UPSERT-SCHEDULE-089, PG-FILE-MIXED-UPSERT-REPLAY-089, and HTTP-FILE-MIXED-UPSERT-NO-DELETE-089 prove complete-page validation, gapped original-ordinal job binding, exact replay, partial-lineage refusal, and zero delete/tombstone authority through the generated SDK. Autonomous scheduling, automatic upsert/delete ordering, batch deletion execution, Provider deletion authority, physical cleanup, restore/recreate, retry/reclaim/dead-letter, full resync, and Runtime authority from observation, page, checkpoint, or execution metadata remain NOT_ACTIVE. Issue #91 activates only scheduler-owned first-attempt File dispatch through the existing exact WorkerLease. Issue #93 extends that carrier with database-timed exponential backoff and at most three automatic higher-generation reclaims through the existing durable-boundary recovery path. Exhausted attempts remain untouched; dead-letter handling, operator remediation, provider polling, and delete ordering remain NOT_ACTIVE. Issue #102 activates only an explicit loopback single-Membership authentication composition and File pgvector Acquire carrier through the same current UserActor transaction and sealed Runtime. RUNTIME-DOGFOOD-AUTH-102, RUNTIME-DOGFOOD-CARRIER-102, and RUNTIME-DOGFOOD-EPOCH-102 prove generic secret or Membership refusal, exact EffectiveScope reduction before ANN LIMIT, deterministic network-free embedding with Release-bound model/input identity, and final Policy Epoch veto. Production authentication, a second human, network exposure beyond the maintainer machine, group/public audience, dogfood OpenCitation, Continue, hybrid retrieval, external query embedding, and non-File providers remain NOT_ACTIVE. Issue #110 activates only the explicit local-process Control and release identity-verification composition under RELEASE-OWNER-019. Its four refusal classes prove absent or partial configuration, exact Control operation scope, cross-plane credential rejection, and mandatory separation from the configured dogfood and worker credentials; every authorized Control operation still receives one lifetime-bound TrustedControlCall. Production authentication, additional operators, durable assignment, delegation, RBAC, network surfaces, operator operation subcommands, and release promotion remain NOT_ACTIVE." + "reconciliation": "Issue #2 fixes the product and testing decisions, issue #5 requires exactly fifteen release invariants and twelve canonical acceptance fixtures, and ADR-0019 resolves the later nineteen-label prose expansion without weakening any safeguard. Issue #15 activates only Organization-level next-request resolve(Acquire) revocation evidence under REVOCATION-006: at that activation, PG-REVOCATION-006, RUN-006, and CACHE-002 are active while BLOB-002 and Continue, citation, Policy-Epoch-bound WorkerLease, production ContextAccessTicket/ActionTicket, audit, outbox, cleanup, finer-epoch, UI, and external-admin carriers remain future or NOT_ACTIVE; later issue records are authoritative for subsequently activated carriers. Issue #16 activates only the M0 refusal gate for unavailable Continue, profile-disabled OpenCitation, and server-owned unavailable Acquire plans: at that activation its real continuation, profile-enabled citation, federated/source-native, and File carriers remain future, while its Runtime and HTTP refusal surfaces prove generic outcomes before content I/O; Issue #69 later activates the private/direct File profile-enabled citation carrier. Issue #17 activates only the signed one-shot persistent no-op durable-job WorkerLease subcarrier under WORKER-LEASE-007. It binds one exact worker audience but no end-user delivery audience or Policy Epoch, and proves only LEASE-SIGNING-017, PG-WORKER-LEASE-NOOP-017, and WORKER-LEASE-REPLAY-007; Source, Resource, Revision, Policy Epoch, end-user delivery audience, idempotency, generation, business mutation, outbox, File publication, and the full ACCEPT-008 matrix remain deferred or NOT_ACTIVE. Issue #18 activates only distinct signed synthetic ContextAccessTicket Provider-read and ActionTicket no-op channel-action subcarriers under ACTION-SEPARATION-014, with current Organization-v0 Policy Epoch validation. TICKET-AUDIENCE-018 and PG-TICKET-EPOCH-018 do not activate production ContextProvider integration, ContextRuntime ticket integration, BotDelivery, full M2 ActionPlane.prepare/perform, a real Sender or external effect, payload/destination/approval/idempotency binding, durable one-shot/replay/reconciliation, or full ACCEPT-012 PASS; those remain future or NOT_ACTIVE. Issue #19 activates only the current Acquire authorized-only ContextRun and restricted delivered-empty DecisionAudit subcarrier under TRACE-REDACTION-012. DIGEST-019, RUN-LINEAGE-019, AUTHORIZED-RUN-019, and PG-TRACE-REDACTION-012 prove deterministic Package and Organization-bound query digests, retained-UserActor-transaction persistence, decisionRef resolution, redaction, and short-lived exact-Organization operator ticket reads with no application-role table access; the supported reader commits deletion before returning, while a direct caller rollback is not claimed as durable exactly-once redemption. Raw query retention, full ContextPackage body retention, unauthenticated transport failures as ContextRuns, cross-Organization analytics, and general observability redaction remain NOT_ACTIVE. Issue #48 activates only the current ACCEPT-002 authenticated HTTP Acquire Membership field-projection carrier under SCOPE-INTERSECTION-004, INDEX-NOT-AUTHORITY-005, and TRACE-REDACTION-012. PROP-FIELD-PROJECTION-048, PG-FIELD-PROJECTION-048, and HTTP-ACCEPT-002-048 bind one current Membership/version field ceiling to same-transaction FORCE-RLS reduction, the sealed AuthorizationKernel, AuthorizedProjection and Evidence integrity, and authorized-only ContextRun/audit persistence. General permission DSLs, caller-authored projection lists, CandidateRef or index field authority, production Provider/source-native ACL negotiation, Supply publication, File/Base field ACL, typed fields, Continue, and Issue #20 runner substitution remain future or NOT_ACTIVE; Issue #69 later activates private/direct File OpenCitation through the same field-projection gates. Issue #63 activates only the digest-only private authenticated HTTP Acquire DeliveryEvidenceRef carrier under TRANSPORT-UNTRUSTED-008. PROP-DELIVERY-EVIDENCE-063, PG-DELIVERY-EVIDENCE-063, HTTP-DELIVERY-EVIDENCE-063, and FILE-DELIVERY-EVIDENCE-063 prove exact service/request/Organization/asker/Membership-version/destination/consumer/purpose/audience/epoch/lifetime binding, stable identical retry identity, role isolation, expiry cleanup, pre-content generic rejection, and one File-backed sealed Runtime delivery. Group AudienceSnapshot, group/public DeliveryEvidenceRef, production ModelGateway, ActionPlane, and the BotDelivery application remain future or NOT_ACTIVE; Issues #64, #66, #69, and #70 later activate the frozen OpenAPI, generated TypeScript SDK, private/direct OpenCitation, and deterministic private model-egress carriers. Issue #65 activates only one opaque digest-only model or channel EgressGrant after final Package policy, exact atomic PostgreSQL redemption and restricted audit, nominal BotDelivery inputs, and deterministic network-free ModelGateway or Sender-preflight spies under EGRESS-011. PROP-EGRESS-011, PG-EGRESS-011, and RUNTIME-EGRESS-011 prove exact Package/Organization/purpose/audience/epoch/hop/profile/lifetime binding and zero additional bytes on replay. Real model/provider calls, a real Sender or channel write, ActionTicket effects, group AudienceSnapshot revalidation, and the BotDelivery application process remain future or NOT_ACTIVE; Issue #64 later activates the generated SDK consumer and Issue #70 later activates the deterministic private TypeScript ModelGateway. Issue #66 activates the frozen public POST /v0/resolve OpenAPI carrier under TRANSPORT-UNTRUSTED-008. OPENAPI-CONTRACT-066, OPENAPI-BREAKING-066, HTTP-V0-066, and PG-RUNTIME-RELEASE-066 prove one public closed operation, deterministic immutable snapshot and breaking-change refusal, a hidden v1 bridge through the same handler and sealed Runtime path, and exact read-only observation of the active Learning-promoted release with fail-closed missing-release behavior before content work. A production BotDelivery caller, Continue redemption, MCP, group AudienceSnapshot, and external effects remain future or NOT_ACTIVE; Issues #64 and #69 later activate the generated TypeScript SDK and private/direct OpenCitation redemption through this frozen operation. Issue #64 activates only the packaged generated TypeScript POST /v0/resolve client under TRANSPORT-UNTRUSTED-008. SDK-CONTRACT-064 and SDK-LIVE-FILE-064 prove deterministic pinned generation, strict closed types, a narrow export map and metadata-only facade, installable tarball consumption, and one real PostgreSQL/File-backed Acquire through CandidateRef, AuthorizationKernel, AuthorizedProjection, ContextPackage, and opaque model egress grant. Issue #69 later extends SDK-LIVE-FILE-064 with a successful private/direct File OpenCitation through a second request-bound DeliveryEvidenceRef; Issue #70 extends the installed SDK fixture into deterministic Package-bound model generation; generated Continue remains generic unavailable. External package publication, production provider access, MCP, group AudienceSnapshot, real Continue redemption, and external effects remain future or NOT_ACTIVE. Issue #67 activates only private ActionPlane.prepare for create-placeholder, finalize-reply, and private-follow-up operation-specific tickets under ACTION-SEPARATION-014. PG-ACTION-PREPARE-067 proves exact current delivery, Organization, destination, audience, source, payload, approval, epoch, lifetime, and idempotency binding under a dedicated non-owner PostgreSQL role with digest-only FORCE-RLS persistence and zero effects. Issue #68 activates private ActionPlane.perform only through a deterministic Sender twin. PG-ACTION-PERFORM-068 proves one pre-Sender current-authority validation, one provider-attempt identity, immutable receipt replay, zero-effect ticket/payload mutation and stale-audience refusal, same-label cross-Organization isolation, and monotonic applied/rejected reconciliation including crash interleavings. Real provider or channel network effects, group AudienceSnapshot, compensation/delete, production BotDelivery orchestration, and the full ACCEPT-012 pass remain future or NOT_ACTIVE. The canonical set is IDs 001 through 012, 014, 015, and 019: CACHE-SCOPE-013 remains a preregistered conditional extension; AUDIENCE-016 is absorbed by SCOPE-INTERSECTION-004 and EGRESS-011; ACL-PROOF-017 is absorbed by INDEX-NOT-AUTHORITY-005 and REVOCATION-006; DELIVERY-EVIDENCE-018 is absorbed by TRANSPORT-UNTRUSTED-008. ACCEPT-001 through ACCEPT-012 follow ADR-0019's category order. Protected-asset references A-01 through A-08 refer, in order, to the eight bullets in the threat model's Protected assets section. Every expectedEvidence value below is a stable planned case identifier, not a claim that the case ran or passed; only an exact activation record upgrades named evidence, while fixture carrier status and the explicit M0 oracle preserve every other accepted-versus-active distinction. Issue #69 activates private/direct File CitationOpenRef issuance and OpenCitation under CITATION-AUTH-010: digest-only multi-use locators reveal only prior Package/Evidence and Fragment location lineage, every open obtains a current UserActor and trusted delivery context then traverses CandidateRef, AuthorizationKernel, AuthorizedProjection, a replacement ContextPackage, EgressGrant, ContextRun, and restricted DecisionAudit. PG-CITATION-AUTH-010, RUNTIME-CITATION-AUTH-010, and SDK-LIVE-FILE-064 prove A/B/A reauthorization, non-consumption on denial, database-clock expiry, cross-kind and cross-Organization opacity, and the generated SDK carrier. Group/public AudienceSnapshot, non-File providers, raw source URL locators, and Continue remain future or NOT_ACTIVE. Issue #70 activates only the private deterministic TypeScript ModelGateway under EGRESS-011. TS-MODEL-EGRESS-070, SDK-MODEL-EGRESS-070, and PG-MODEL-EGRESS-070 prove one current Package, exact grant redemption, closed provider input, bounded Package-subset citations, replay zero bytes, and digest-only retained audit. Real providers, streaming, group AudienceSnapshot, model-authored ActionPlane authority, and external effects remain future or NOT_ACTIVE. Issue #71 activates the complete private File-backed deterministic-twin BotDelivery carrier under TRANSPORT-UNTRUSTED-008, CITATION-AUTH-010, EGRESS-011, and ACTION-SEPARATION-014. TS-PRIVATE-BOT-FLOW-071, SDK-PRIVATE-BOT-FLOW-071, and PG-PRIVATE-BOT-FLOW-071 prove the independent TypeScript process and import boundary, exact verified private event binding, opaque DeliveryEvidenceRef transport, installed generated-SDK HTTP resolve, sealed File Package path, controlled model generation, distinct placeholder/final ActionPlane effects, digest-only DeliveryReceipt audit, citation reopening, and the composed wrong-binding oracles. The historical FIXTURE-ACCEPT-012 continues to prove the M0 unavailable cross-capability baseline separately. Live Feishu/model/Sender network carriers, group/public delivery, compensation/delete, Continue, and MCP remain NOT_ACTIVE. Issue #81 activates deterministic shallow File readChanges and whole-page acknowledgement under WORKER-LEASE-007. PG-FILE-CHANGE-ACTIVATE-081, PG-FILE-CHANGE-PAGE-081, and PG-FILE-CHANGE-DENY-081 prove immutable v3 activation, provider-authenticated content-free paging, exact replay, predecessor ordering, post-commit cursor issuance, FORCE-RLS tenant isolation, source/version invalidation, and zero implicit job or publication effect. Automatic scheduling, deletion execution, recursive discovery, full resync, and Runtime authority from cursor or checkpoint metadata remain NOT_ACTIVE. Issue #83 activates only explicit accepted File page scheduling through the existing file.import acquisition, WorkerLease, and publication path under WORKER-LEASE-007. PG-FILE-CHANGE-SCHEDULE-083, PG-FILE-CHANGE-SCHEDULE-DENY-083, and PG-FILE-CHANGE-SUPERSESSION-083 prove whole-page atomicity, exact job replay, explicit current FileImportAudience and receiver validation, immutable raw-byte observation lineage, pre-compiler drift refusal, and scan-epoch fences before content read and visible publication with zero stale publication effect. Autonomous polling, implicit audience inheritance, deletion execution, automatic retry/reclaim, dead-letter handling, full resync, and Runtime authority from provider checkpoints remain NOT_ACTIVE. Issue #85 activates bounded File delete observations only under WORKER-LEASE-007. PG-FILE-DELETE-DETECT-085, PG-FILE-DELETE-PAGE-085, and PG-FILE-DELETE-NO-EFFECT-085 prove exact latest-complete same-SourceVersion baseline binding, stable shallow diff, immutable canonical persistence/replay, forged/incomplete/stale/cross-Organization refusal, mixed-page scheduling refusal, and zero tombstone, Policy Epoch, cleanup, watermark, or generated-SDK Runtime visibility effect. At that activation, deletion execution remains NOT_ACTIVE; autonomous polling, retry/reclaim/dead-letter, full resync, recursive scan, and Runtime authority from baseline or delete metadata remain NOT_ACTIVE. Issue #87 later activates only exact trusted current File delete execution through the existing #28 tombstone authority under REVOCATION-006. PG-FILE-DELETE-EXECUTE-087, PG-FILE-DELETE-REPLAY-087, and HTTP-FILE-DELETE-INVISIBLE-087 prove current complete-scan revalidation, server-derived effect identity, atomic tombstone/epoch/cleanup/binding, exact replay, mismatch rollback, and immediate sealed-Runtime invisibility. Issue #89 later activates only the exact upsert projection of one current mixed v4 page under WORKER-LEASE-007. PG-FILE-MIXED-UPSERT-SCHEDULE-089, PG-FILE-MIXED-UPSERT-REPLAY-089, and HTTP-FILE-MIXED-UPSERT-NO-DELETE-089 prove complete-page validation, gapped original-ordinal job binding, exact replay, partial-lineage refusal, and zero delete/tombstone authority through the generated SDK. Autonomous scheduling, automatic upsert/delete ordering, batch deletion execution, Provider deletion authority, physical cleanup, restore/recreate, retry/reclaim/dead-letter, full resync, and Runtime authority from observation, page, checkpoint, or execution metadata remain NOT_ACTIVE. Issue #91 activates only scheduler-owned first-attempt File dispatch through the existing exact WorkerLease. Issue #93 extends that carrier with database-timed exponential backoff and at most three automatic higher-generation reclaims through the existing durable-boundary recovery path. Exhausted attempts remain untouched; dead-letter handling, operator remediation, provider polling, and delete ordering remain NOT_ACTIVE. Issue #102 activates only an explicit loopback single-Membership authentication composition and File pgvector Acquire and private OpenCitation carriers through the same current UserActor transaction and sealed Runtime. RUNTIME-DOGFOOD-AUTH-102, RUNTIME-DOGFOOD-CARRIER-102, and RUNTIME-DOGFOOD-EPOCH-102 prove generic secret or Membership refusal, exact EffectiveScope reduction before ANN LIMIT, deterministic network-free embedding with Release-bound model/input identity, and final Policy Epoch veto. Production authentication, a second human, network exposure beyond the maintainer machine, group/public audience, Continue, hybrid retrieval, external query embedding, and non-File providers remain NOT_ACTIVE. Issue #110 activates only the explicit local-process Control and release identity-verification composition under RELEASE-OWNER-019. Its four refusal classes prove absent or partial configuration, exact Control operation scope, cross-plane credential rejection, and mandatory separation from the configured dogfood and worker credentials; every authorized Control operation still receives one lifetime-bound TrustedControlCall. Production authentication, additional operators, durable assignment, delegation, RBAC, network surfaces, operator operation subcommands, and release promotion remain NOT_ACTIVE. Issue #130 activates the loopback-only co-resident Evidence Console under ADR-0083: browser sessions explicitly authenticate through the existing dogfood identity, Ask reopens every citation through the public Runtime carrier, source/import/Article jobs consume separate one-operation Control calls, feedback persists through an exact current-Membership Runtime binding with no publication authority, and Hit Test exposes only post-Kernel rank-free ContextPackage evidence. Production or non-loopback exposure, multiple operators, numeric rank scores, Continue, and feedback-driven publication remain NOT_ACTIVE." }, "hardOracles": [ { @@ -1023,7 +1024,7 @@ { "id": "RUNTIME-DOGFOOD-AUTH-102", "surface": "tests/integration/test_dogfood_runtime_activation.py::test_dogfood_secret_and_membership_fail_closed_without_secret_retention", - "oracle": "Absent, wrong, or truncated secrets and revoked or expired configured Membership authority return the same generic authentication failure; the secret is absent from responses, logs, ContextRun, and DecisionAudit." + "oracle": "Absent, wrong, or truncated secrets and revoked or expired configured Membership authority return the same generic authentication failure; the secret is absent from responses, logs, ContextRun, and DecisionAudit. A successful dogfood resolve issues one private File CitationOpenRef that reopens only through a second current-Membership sealed Runtime resolve." } ], "deferredEvidence": [ @@ -1042,7 +1043,6 @@ "network exposure beyond the maintainer machine", "group or public audience", "Continue", - "dogfood OpenCitation", "hybrid retrieval", "external query embedding", "non-File providers" @@ -1051,7 +1051,7 @@ { "issueRef": "#102", "invariantRef": "INDEX-NOT-AUTHORITY-005", - "carrier": "loopback-only File pgvector dogfood Acquire delivery", + "carrier": "loopback-only File pgvector dogfood Acquire and private OpenCitation delivery", "status": "active_fail_closed", "policyEpochScope": "organization-v0", "controlBoundary": "current UserActor transaction -> database-derived exact EffectiveScope -> pre-LIMIT pgvector scope reduction -> CandidateRef -> sealed AuthorizationKernel -> AuthorizedProjection -> ContextPackage", @@ -1083,7 +1083,6 @@ "network exposure beyond the maintainer machine", "group or public audience", "Continue", - "dogfood OpenCitation", "hybrid retrieval", "external query embedding", "non-File providers" diff --git a/migrations/versions/20260730_0047_minimal_ui.py b/migrations/versions/20260730_0047_minimal_ui.py new file mode 100644 index 00000000..e66ec664 --- /dev/null +++ b/migrations/versions/20260730_0047_minimal_ui.py @@ -0,0 +1,904 @@ +"""Add proof-bound Minimal UI projections and evidence capture. + +Revision ID: 20260730_0047 +Revises: 20260730_0046 +Create Date: 2026-07-30 +""" + +# ruff: noqa: E501 + +from collections.abc import Sequence + +import sqlalchemy as sa +from alembic import op +from sqlalchemy.dialects import postgresql + +revision: str = "20260730_0047" +down_revision: str | None = "20260730_0046" +branch_labels: str | Sequence[str] | None = None +depends_on: str | Sequence[str] | None = None + +_MIGRATOR = "context_engine_migrator" +_CONTROL = "context_engine_control" +_RUNTIME = "context_engine_runtime" +_WORKER_DEFINER = "context_engine_worker_lease_definer" +_ACCESS_DEFINER = "context_engine_access_policy_definer" +_RUN_READER_DEFINER = "context_engine_context_run_reader_definer" +_MAX = (1 << 63) - 1 +_EXACT_IMPORT = "context_control_prepare_exact_file_import" +_EXACT_IMPORT_SIGNATURE = ( + "(uuid,uuid,uuid,uuid,uuid,text,text,uuid,uuid,bigint,text,text,uuid," + "text,bigint,text,text,text)" +) +_READ_ARTICLE = "context_control_read_article_policy" +_READ_ARTICLE_SIGNATURE = "(uuid,text)" +_CHANGE_ARTICLE = "context_control_change_article_policy" +_CHANGE_ARTICLE_SIGNATURE = ( + "(uuid,text,bigint,bigint,text,text[],text,uuid,uuid,bigint)" +) +_CAPTURE_FEEDBACK = "context_runtime_capture_context_feedback" +_CAPTURE_FEEDBACK_SIGNATURE = "(uuid,text,text,uuid,uuid,bigint,text,text,text)" +_REDEEM = "context_worker_redeem_file_import" +_REDEEM_SIGNATURE = ( + "(uuid,uuid,uuid,text,bigint,bigint,bytea,timestamp with time zone," + "timestamp with time zone)" +) +_FILE_OPERATION_FENCES = ( + "context-engine.file-change-scheduling-migration-fence", + "context-engine.file-dispatch-migration-fence", + "context-engine.file-status-migration-fence", +) + + +def _join_file_operation_fences() -> None: + connection = op.get_bind() + for migration_fence in _FILE_OPERATION_FENCES: + connection.execute( + sa.text( + "SELECT pg_catalog.pg_advisory_xact_lock(" + "pg_catalog.hashtextextended(:migration_fence, 0))" + ), + {"migration_fence": migration_fence}, + ) + + +def _replace_redeem_ui_fields(*, install: bool) -> None: + connection = op.get_bind() + definition = connection.execute( + sa.text( + "SELECT pg_catalog.pg_get_functiondef(CAST(:procedure AS regprocedure))" + ), + {"procedure": f"public.{_REDEEM}{_REDEEM_SIGNATURE}"}, + ).scalar_one() + if not isinstance(definition, str): + raise RuntimeError("File redemption definition is unavailable") + base_return = "expected_content_length bigint)" + ui_return = ( + "expected_content_length bigint, ui_preview_digest text, " + "expected_fragment_digest text, compiler_config_version text)" + ) + base_select = ( + "acquisition.expected_content_sha256, acquisition.expected_content_length" + ) + ui_select = ( + base_select + ", acquisition.ui_preview_digest, " + "acquisition.expected_fragment_digest, " + "acquisition.compiler_config_version" + ) + searched_return, replacement_return = ( + (base_return, ui_return) if install else (ui_return, base_return) + ) + searched_select, replacement_select = ( + (base_select, ui_select) if install else (ui_select, base_select) + ) + if definition.count(searched_return) != 1 or definition.count(searched_select) != 1: + raise RuntimeError("File redemption shape was not recognized") + replacement = definition.replace(searched_return, replacement_return).replace( + searched_select, replacement_select + ) + op.execute(f"GRANT CREATE ON SCHEMA public TO {_WORKER_DEFINER}") + op.execute(f"SET LOCAL ROLE {_WORKER_DEFINER}") + op.execute(f"DROP FUNCTION public.{_REDEEM}{_REDEEM_SIGNATURE}") + op.execute(replacement) + op.execute( + f"GRANT EXECUTE ON FUNCTION public.{_REDEEM}{_REDEEM_SIGNATURE} " + "TO context_engine_worker" + ) + op.execute("RESET ROLE") + op.execute(f"REVOKE CREATE ON SCHEMA public FROM {_WORKER_DEFINER}") + + +def _create_exact_import() -> None: + op.execute( + f""" + CREATE FUNCTION public.{_EXACT_IMPORT}( + requested_organization_id uuid, requested_acquisition_id uuid, + requested_job_id uuid, requested_activated_version_id uuid, + requested_source_id uuid, requested_relative_path text, + requested_audience_principal_ref text, + requested_audience_user_id uuid, + requested_audience_membership_id uuid, + requested_audience_membership_version bigint, + requested_idempotency_key text, requested_request_digest text, + requested_service_principal_id uuid, + requested_expected_content_sha256 text, + requested_expected_content_length bigint, + requested_expected_fragment_digest text, + requested_compiler_config_version text, + requested_preview_digest text + ) RETURNS TABLE (job_id uuid, service_principal_id uuid) + LANGUAGE plpgsql SECURITY DEFINER + SET search_path = pg_catalog, pg_temp + SET row_security = on + AS $function$ + DECLARE selected_version_id uuid; selected_acquisition_id uuid; + selected_membership_id uuid; + selected_root_ref text; selected_capabilities jsonb; + trusted_now timestamptz; + BEGIN + IF SESSION_USER <> '{_CONTROL}' + OR NULLIF(current_setting('app.organization_id', true), '')::uuid + IS DISTINCT FROM requested_organization_id + OR requested_expected_content_sha256 !~ '^[0-9a-f]{{64}}$' + OR requested_expected_fragment_digest !~ '^[0-9a-f]{{64}}$' + OR requested_preview_digest !~ '^[0-9a-f]{{64}}$' + OR requested_expected_content_length < 0 + OR requested_compiler_config_version <> 'markdown-config-v1' + THEN RETURN; END IF; + trusted_now := pg_catalog.statement_timestamp(); + PERFORM pg_catalog.pg_advisory_xact_lock( + pg_catalog.hashtextextended( + 'context-engine.file-publication:' + || requested_organization_id::text, 0 + ) + ); + PERFORM pg_catalog.set_config( + 'app.worker_job_id', requested_job_id::text, true + ); + SELECT version.version_id, version.root_ref, + version.capability_manifest + INTO selected_version_id, selected_root_ref, selected_capabilities + FROM public.context_source AS source + JOIN public.source_version AS version + ON version.organization_id = source.organization_id + AND version.source_id = source.source_id + AND version.version_id = source.active_version_id + WHERE source.organization_id = requested_organization_id + AND source.source_id = requested_source_id + AND source.lifecycle_state = 'active' + AND version.capability_manifest->>'declarationVersion' + IN ('file-capabilities-v1', 'file-capabilities-v2') + FOR UPDATE OF source; + SELECT audience_membership.membership_id + INTO selected_membership_id + FROM public.membership AS audience_membership + WHERE audience_membership.organization_id = requested_organization_id + AND audience_membership.membership_id = requested_audience_membership_id + AND audience_membership.user_id = requested_audience_user_id + AND audience_membership.membership_version = requested_audience_membership_version + AND audience_membership.status = 'active' + AND audience_membership.valid_from <= trusted_now + AND (audience_membership.valid_until IS NULL + OR audience_membership.valid_until > trusted_now); + IF selected_version_id IS NULL + OR selected_membership_id IS NULL + OR NOT EXISTS ( + SELECT 1 FROM public.service_principal AS receiver + WHERE receiver.organization_id = requested_organization_id + AND receiver.service_principal_id = requested_service_principal_id + AND receiver.workload = 'supply.file-import' + AND receiver.worker_audience = 'context-engine-worker' + AND receiver.operation = 'file.import' + AND receiver.enabled IS TRUE + ) THEN RETURN; END IF; + + IF selected_capabilities->>'declarationVersion' + = 'file-capabilities-v1' THEN + INSERT INTO public.source_version ( + organization_id, source_id, version_id, source_kind, + root_ref, capability_manifest, created_at + ) VALUES ( + requested_organization_id, requested_source_id, + requested_activated_version_id, 'file', selected_root_ref, + jsonb_set(jsonb_set(jsonb_set( + selected_capabilities, '{{declarationVersion}}', + '"file-capabilities-v2"'::jsonb), + '{{fileSourceAccess}}', '"available"'::jsonb), + '{{ingestionJobs}}', '"available"'::jsonb), + trusted_now + ); + UPDATE public.context_source + SET active_version_id = requested_activated_version_id + WHERE organization_id = requested_organization_id + AND source_id = requested_source_id + AND active_version_id = selected_version_id + AND lifecycle_state = 'active'; + IF NOT FOUND THEN RETURN; END IF; + selected_version_id := requested_activated_version_id; + ELSIF selected_capabilities->>'fileSourceAccess' <> 'available' + OR selected_capabilities->>'ingestionJobs' <> 'available' + THEN RETURN; END IF; + + INSERT INTO public.file_acquisition ( + organization_id, acquisition_id, source_id, source_version_id, + relative_path, audience_principal_ref, audience_membership_id, + audience_membership_version, idempotency_key, request_digest, + created_at, expected_content_sha256, expected_content_length, + ui_preview_digest, expected_fragment_digest, + compiler_config_version + ) VALUES ( + requested_organization_id, requested_acquisition_id, + requested_source_id, selected_version_id, + requested_relative_path, requested_audience_principal_ref, + requested_audience_membership_id, + requested_audience_membership_version, + requested_idempotency_key, requested_request_digest, trusted_now, + requested_expected_content_sha256, + requested_expected_content_length, requested_preview_digest, + requested_expected_fragment_digest, + requested_compiler_config_version + ) ON CONFLICT ( + organization_id, source_id, idempotency_key + ) DO NOTHING; + SELECT acquisition.acquisition_id INTO selected_acquisition_id + FROM public.file_acquisition AS acquisition + WHERE acquisition.organization_id = requested_organization_id + AND acquisition.source_id = requested_source_id + AND acquisition.idempotency_key = requested_idempotency_key + AND acquisition.request_digest = requested_request_digest + AND acquisition.relative_path = requested_relative_path + AND acquisition.audience_principal_ref = requested_audience_principal_ref + AND acquisition.audience_membership_id = requested_audience_membership_id + AND acquisition.audience_membership_version = requested_audience_membership_version + AND acquisition.expected_content_sha256 = requested_expected_content_sha256 + AND acquisition.expected_content_length = requested_expected_content_length + AND acquisition.ui_preview_digest = requested_preview_digest + AND acquisition.expected_fragment_digest = requested_expected_fragment_digest + AND acquisition.compiler_config_version = requested_compiler_config_version; + IF selected_acquisition_id IS NULL THEN RETURN; END IF; + PERFORM pg_catalog.set_config( + 'app.file_acquisition_id', selected_acquisition_id::text, true + ); + INSERT INTO public.file_import_job ( + organization_id, job_id, acquisition_id, source_id, + service_principal_id, workload, worker_audience, actor_kind, + operation, state, created_at + ) VALUES ( + requested_organization_id, requested_job_id, + selected_acquisition_id, requested_source_id, + requested_service_principal_id, 'supply.file-import', + 'context-engine-worker', 'service', 'file.import', + 'available', trusted_now + ) ON CONFLICT (organization_id, acquisition_id) DO NOTHING; + SELECT job.job_id INTO requested_job_id + FROM public.file_import_job AS job + WHERE job.organization_id = requested_organization_id + AND job.acquisition_id = selected_acquisition_id + AND job.service_principal_id = requested_service_principal_id; + IF requested_job_id IS NULL THEN RETURN; END IF; + PERFORM pg_catalog.set_config( + 'app.worker_job_id', requested_job_id::text, true + ); + RETURN QUERY SELECT job.job_id, job.service_principal_id + FROM public.file_import_job AS job + WHERE job.organization_id = requested_organization_id + AND job.acquisition_id = selected_acquisition_id + AND job.service_principal_id = requested_service_principal_id; + END; $function$ + """ + ) + op.execute( + f"REVOKE ALL ON FUNCTION public.{_EXACT_IMPORT}" + f"{_EXACT_IMPORT_SIGNATURE} FROM PUBLIC" + ) + op.execute( + f"GRANT EXECUTE ON FUNCTION public.{_EXACT_IMPORT}" + f"{_EXACT_IMPORT_SIGNATURE} TO {_CONTROL}" + ) + op.execute(f"GRANT CREATE ON SCHEMA public TO {_WORKER_DEFINER}") + op.execute( + f"ALTER FUNCTION public.{_EXACT_IMPORT}{_EXACT_IMPORT_SIGNATURE} " + f"OWNER TO {_WORKER_DEFINER}" + ) + op.execute(f"REVOKE CREATE ON SCHEMA public FROM {_WORKER_DEFINER}") + + +def _create_article_functions() -> None: + op.execute( + f"CREATE POLICY membership_ui_access_definer_select ON membership " + f"FOR SELECT TO {_ACCESS_DEFINER} USING (" + "organization_id = NULLIF(current_setting('app.ui_actor_organization_id', true), '')::uuid " + "AND user_id = NULLIF(current_setting('app.ui_actor_user_id', true), '')::uuid " + "AND membership_id = NULLIF(current_setting('app.ui_actor_membership_id', true), '')::uuid " + "AND membership_version = NULLIF(current_setting('app.ui_actor_membership_version', true), '')::bigint " + "AND current_setting('app.ui_actor_mode', true) = 'article_policy_change')" + ) + op.execute(f"GRANT SELECT ON TABLE membership TO {_ACCESS_DEFINER}") + for command, expression in ( + ( + "INSERT", + "WITH CHECK (organization_id = NULLIF(current_setting('app.organization_id', true), '')::uuid)", + ), + ( + "UPDATE", + "USING (organization_id = NULLIF(current_setting('app.organization_id', true), '')::uuid) " + "WITH CHECK (organization_id = NULLIF(current_setting('app.organization_id', true), '')::uuid)", + ), + ): + op.execute( + f"CREATE POLICY article_explicit_policy_setting_access_definer_{command.lower()} " + f"ON article_explicit_policy_setting FOR {command} TO {_ACCESS_DEFINER} {expression}" + ) + op.execute( + f"GRANT INSERT, UPDATE ON TABLE article_explicit_policy_setting TO {_ACCESS_DEFINER}" + ) + op.execute( + f""" + CREATE FUNCTION public.{_READ_ARTICLE}( + requested_organization_id uuid, requested_resource_ref text + ) RETURNS TABLE ( + resource_ref text, source_ref text, policy_version bigint, + local_policy_kind text, local_group_refs text[], policy_kind text, + group_refs text[], published boolean, resolution_rung text, + policy_epoch bigint + ) LANGUAGE plpgsql STABLE SECURITY DEFINER + SET search_path = pg_catalog SET row_security = on AS $function$ + BEGIN + IF SESSION_USER <> '{_CONTROL}' + OR NULLIF(current_setting('app.organization_id', true), '')::uuid + IS DISTINCT FROM requested_organization_id + OR requested_resource_ref IS NULL + THEN RETURN; END IF; + RETURN QUERY + SELECT policy.resource_ref, resource.source_ref, + policy.policy_version, policy.local_policy_kind, + policy.local_group_refs, policy.policy_kind, + policy.group_refs, policy.published, + policy.resolution_rung, epoch.policy_epoch + FROM public.article_access_policy AS policy + JOIN public.context_resource AS resource + ON resource.organization_id = policy.organization_id + AND resource.resource_ref = policy.resource_ref + AND resource.tombstoned IS FALSE + JOIN public.organization_policy_epoch AS epoch + ON epoch.organization_id = policy.organization_id + WHERE policy.organization_id = requested_organization_id + AND policy.resource_ref = requested_resource_ref; + END; $function$ + """ + ) + op.execute( + f""" + CREATE FUNCTION public.{_CHANGE_ARTICLE}( + requested_organization_id uuid, requested_resource_ref text, + expected_policy_version bigint, expected_policy_epoch bigint, + requested_policy_kind text, requested_group_refs text[], + requested_preview_digest text, requested_user_id uuid, + requested_membership_id uuid, + requested_membership_version bigint + ) RETURNS TABLE (policy_version bigint, policy_epoch bigint) + LANGUAGE plpgsql SECURITY DEFINER + SET search_path = pg_catalog SET row_security = on AS $function$ + DECLARE resource_row public.context_resource%ROWTYPE; + policy_row public.article_access_policy%ROWTYPE; + observation public.article_source_acl_observation%ROWTYPE; + effective_kind text; effective_groups text[] := ARRAY[]::text[]; + next_version bigint; next_epoch bigint; + selected_membership_id uuid; trusted_now timestamptz; + BEGIN + IF SESSION_USER <> '{_CONTROL}' + OR NULLIF(current_setting('app.organization_id', true), '')::uuid + IS DISTINCT FROM requested_organization_id + OR requested_preview_digest !~ '^[0-9a-f]{{64}}$' + OR expected_policy_version NOT BETWEEN 1 AND {_MAX} + OR expected_policy_epoch NOT BETWEEN 1 AND {_MAX} + OR requested_membership_version NOT BETWEEN 1 AND {_MAX} + OR requested_group_refs IS NULL + OR NOT ((requested_policy_kind IN ('private','organization') + AND cardinality(requested_group_refs) = 0) + OR (requested_policy_kind = 'groups' + AND cardinality(requested_group_refs) > 0)) + OR EXISTS ( + SELECT 1 FROM unnest(requested_group_refs) requested(group_ref) + WHERE NOT EXISTS ( + SELECT 1 FROM public.article_access_group AS owned + WHERE owned.organization_id = requested_organization_id + AND owned.group_ref = requested.group_ref)) + THEN RETURN; END IF; + trusted_now := pg_catalog.statement_timestamp(); + PERFORM pg_catalog.pg_advisory_xact_lock( + pg_catalog.hashtextextended( + 'context-engine.file-publication:' + || requested_organization_id::text, 0 + ) + ); + PERFORM pg_catalog.set_config( + 'app.ui_actor_mode', 'article_policy_change', true + ); + PERFORM pg_catalog.set_config( + 'app.ui_actor_organization_id', + requested_organization_id::text, true + ); + PERFORM pg_catalog.set_config( + 'app.ui_actor_user_id', requested_user_id::text, true + ); + PERFORM pg_catalog.set_config( + 'app.ui_actor_membership_id', + requested_membership_id::text, true + ); + PERFORM pg_catalog.set_config( + 'app.ui_actor_membership_version', + requested_membership_version::text, true + ); + SELECT membership.membership_id INTO selected_membership_id + FROM public.membership AS membership + WHERE membership.organization_id = requested_organization_id + AND membership.user_id = requested_user_id + AND membership.membership_id = requested_membership_id + AND membership.membership_version = requested_membership_version + AND membership.status = 'active' + AND membership.valid_from <= trusted_now + AND (membership.valid_until IS NULL + OR membership.valid_until > trusted_now); + IF selected_membership_id IS NULL THEN RETURN; END IF; + SELECT * INTO resource_row + FROM public.context_resource AS resource + WHERE resource.organization_id = requested_organization_id + AND resource.resource_ref = requested_resource_ref + AND resource.tombstoned IS FALSE + FOR UPDATE; + SELECT * INTO policy_row + FROM public.article_access_policy AS policy + WHERE policy.organization_id = requested_organization_id + AND policy.resource_ref = requested_resource_ref + FOR UPDATE; + SELECT epoch.policy_epoch + 1 INTO next_epoch + FROM public.organization_policy_epoch AS epoch + WHERE epoch.organization_id = requested_organization_id + AND epoch.policy_epoch = expected_policy_epoch + AND epoch.policy_epoch < {_MAX} + FOR UPDATE; + IF resource_row.resource_ref IS NULL + OR policy_row.resource_ref IS NULL + OR policy_row.policy_version <> expected_policy_version + OR policy_row.policy_version >= {_MAX} + OR next_epoch IS NULL + THEN RETURN; END IF; + SELECT * INTO observation + FROM public.article_source_acl_observation AS source_acl + WHERE source_acl.organization_id = requested_organization_id + AND source_acl.resource_ref = requested_resource_ref + AND source_acl.source_ref = resource_row.source_ref + FOR UPDATE; + IF observation.resource_ref IS NOT NULL + AND observation.observation_status = 'resolved' THEN + IF requested_policy_kind = 'private' + OR observation.policy_kind = 'private' THEN + effective_kind := 'private'; + ELSIF requested_policy_kind = 'organization' THEN + effective_kind := observation.policy_kind; + effective_groups := observation.group_refs; + ELSIF observation.policy_kind = 'organization' THEN + effective_kind := requested_policy_kind; + effective_groups := requested_group_refs; + ELSE + SELECT COALESCE(array_agg(group_ref ORDER BY group_ref), + ARRAY[]::text[]) + INTO effective_groups + FROM (SELECT unnest(requested_group_refs) AS group_ref + INTERSECT + SELECT unnest(observation.group_refs)) AS shared; + IF cardinality(effective_groups) > 0 THEN + effective_kind := 'groups'; + END IF; + END IF; + END IF; + INSERT INTO public.article_explicit_policy_setting ( + organization_id, source_ref, resource_ref, + policy_kind, group_refs + ) VALUES ( + requested_organization_id, resource_row.source_ref, + requested_resource_ref, requested_policy_kind, + requested_group_refs + ) ON CONFLICT (organization_id, resource_ref) DO UPDATE + SET source_ref = EXCLUDED.source_ref, + policy_kind = EXCLUDED.policy_kind, + group_refs = EXCLUDED.group_refs; + next_version := policy_row.policy_version + 1; + UPDATE public.article_access_policy AS policy + SET policy_version = next_version, + local_policy_kind = requested_policy_kind, + local_group_refs = requested_group_refs, + policy_kind = effective_kind, + group_refs = effective_groups, + published = effective_kind IS NOT NULL, + resolution_rung = 'explicit_article', + fixed_at_policy_epoch = next_epoch + WHERE policy.organization_id = requested_organization_id + AND policy.resource_ref = requested_resource_ref + AND policy.policy_version = expected_policy_version; + IF NOT FOUND THEN RETURN; END IF; + UPDATE public.organization_policy_epoch AS epoch + SET policy_epoch = next_epoch + WHERE epoch.organization_id = requested_organization_id + AND epoch.policy_epoch = expected_policy_epoch; + IF NOT FOUND THEN + RAISE EXCEPTION USING ERRCODE = '40001', + MESSAGE = 'Article policy confirmation was not accepted'; + END IF; + RETURN QUERY SELECT next_version, next_epoch; + END; $function$ + """ + ) + for name, signature in ( + (_READ_ARTICLE, _READ_ARTICLE_SIGNATURE), + (_CHANGE_ARTICLE, _CHANGE_ARTICLE_SIGNATURE), + ): + op.execute(f"REVOKE ALL ON FUNCTION public.{name}{signature} FROM PUBLIC") + op.execute(f"GRANT EXECUTE ON FUNCTION public.{name}{signature} TO {_CONTROL}") + op.execute(f"GRANT CREATE ON SCHEMA public TO {_ACCESS_DEFINER}") + op.execute( + f"ALTER FUNCTION public.{name}{signature} OWNER TO {_ACCESS_DEFINER}" + ) + op.execute(f"REVOKE CREATE ON SCHEMA public FROM {_ACCESS_DEFINER}") + + +def _create_feedback() -> None: + op.create_table( + "context_feedback", + sa.Column("organization_id", postgresql.UUID(as_uuid=True), nullable=False), + sa.Column("feedback_ref", sa.Text(), nullable=False), + sa.Column("run_ref", sa.Text(), nullable=False), + sa.Column("user_id", postgresql.UUID(as_uuid=True), nullable=False), + sa.Column("membership_id", postgresql.UUID(as_uuid=True), nullable=False), + sa.Column("membership_version", sa.BigInteger(), nullable=False), + sa.Column("principal_ref", sa.Text(), nullable=False), + sa.Column("rating", sa.Text(), nullable=False), + sa.Column("note", sa.Text(), nullable=True), + sa.Column( + "recorded_at", + sa.DateTime(timezone=True), + nullable=False, + server_default=sa.text("statement_timestamp()"), + ), + sa.PrimaryKeyConstraint( + "organization_id", "feedback_ref", name="pk_context_feedback" + ), + sa.UniqueConstraint( + "organization_id", + "run_ref", + "membership_id", + "membership_version", + name="uq_context_feedback_actor_run", + ), + sa.ForeignKeyConstraint( + ["organization_id", "run_ref"], + ["context_run.organization_id", "context_run.run_ref"], + name="fk_context_feedback_run_same_organization", + ondelete="RESTRICT", + ), + sa.ForeignKeyConstraint( + ["organization_id", "membership_id", "membership_version"], + [ + "membership.organization_id", + "membership.membership_id", + "membership.membership_version", + ], + name="fk_context_feedback_membership_same_organization", + ondelete="RESTRICT", + ), + sa.CheckConstraint( + "feedback_ref ~ '^fb_[0-9a-f]{64}$'", + name="ck_context_feedback_ref", + ), + sa.CheckConstraint( + f"membership_version BETWEEN 1 AND {_MAX}", + name="ck_context_feedback_membership_version", + ), + sa.CheckConstraint( + "btrim(run_ref) <> '' AND char_length(run_ref) <= 256 " + "AND btrim(principal_ref) <> ''", + name="ck_context_feedback_refs", + ), + sa.CheckConstraint( + "rating IN ('helpful','not_helpful')", + name="ck_context_feedback_rating", + ), + sa.CheckConstraint( + "note IS NULL OR (btrim(note) <> '' AND char_length(note) <= 1000)", + name="ck_context_feedback_note", + ), + ) + for role in ( + "PUBLIC", + _CONTROL, + "context_engine_runtime", + _RUN_READER_DEFINER, + ): + op.execute(f"REVOKE ALL ON TABLE context_feedback FROM {role}") + op.execute("ALTER TABLE context_feedback ENABLE ROW LEVEL SECURITY") + op.execute("ALTER TABLE context_feedback FORCE ROW LEVEL SECURITY") + op.execute( + f"CREATE POLICY context_feedback_migrator_administration " + f"ON context_feedback FOR ALL TO {_MIGRATOR} USING (true) WITH CHECK (true)" + ) + feedback_context = """ + context_feedback.organization_id = NULLIF( + current_setting('app.ui_feedback_organization_id', true), '' + )::uuid + AND context_feedback.run_ref = current_setting( + 'app.ui_feedback_run_ref', true + ) + AND context_feedback.membership_id = NULLIF( + current_setting('app.ui_feedback_membership_id', true), '' + )::uuid + AND context_feedback.membership_version = NULLIF( + current_setting('app.ui_feedback_membership_version', true), '' + )::bigint + AND current_setting('app.ui_feedback_mode', true) = 'capture' + """ + op.execute( + "CREATE POLICY context_feedback_ui_definer_insert ON context_feedback " + f"FOR INSERT TO {_RUN_READER_DEFINER} WITH CHECK ({feedback_context})" + ) + op.execute( + "CREATE POLICY context_feedback_ui_definer_select ON context_feedback " + f"FOR SELECT TO {_RUN_READER_DEFINER} USING ({feedback_context})" + ) + run_context = """ + context_run.organization_id = NULLIF( + current_setting('app.ui_feedback_organization_id', true), '' + )::uuid + AND context_run.run_ref = current_setting( + 'app.ui_feedback_run_ref', true + ) + AND context_run.user_id = NULLIF( + current_setting('app.ui_feedback_user_id', true), '' + )::uuid + AND context_run.membership_id = NULLIF( + current_setting('app.ui_feedback_membership_id', true), '' + )::uuid + AND context_run.membership_version = NULLIF( + current_setting('app.ui_feedback_membership_version', true), '' + )::bigint + AND context_run.principal_ref = current_setting( + 'app.ui_feedback_principal_ref', true + ) + AND current_setting('app.ui_feedback_mode', true) = 'capture' + """ + op.execute( + "CREATE POLICY context_run_ui_feedback_definer_select ON context_run " + f"FOR SELECT TO {_RUN_READER_DEFINER} USING ({run_context})" + ) + feedback_membership_context = """ + membership.organization_id = NULLIF( + current_setting('app.ui_feedback_organization_id', true), '' + )::uuid + AND membership.user_id = NULLIF( + current_setting('app.ui_feedback_user_id', true), '' + )::uuid + AND membership.membership_id = NULLIF( + current_setting('app.ui_feedback_membership_id', true), '' + )::uuid + AND membership.membership_version = NULLIF( + current_setting('app.ui_feedback_membership_version', true), '' + )::bigint + AND current_setting('app.ui_feedback_mode', true) = 'capture' + """ + op.execute( + "CREATE POLICY membership_ui_feedback_definer_select ON membership " + f"FOR SELECT TO {_RUN_READER_DEFINER} " + f"USING ({feedback_membership_context})" + ) + op.execute( + f"GRANT SELECT, INSERT ON TABLE context_feedback TO {_RUN_READER_DEFINER}" + ) + op.execute(f"GRANT SELECT ON TABLE membership TO {_RUN_READER_DEFINER}") + op.execute( + f""" + CREATE FUNCTION public.{_CAPTURE_FEEDBACK}( + requested_organization_id uuid, requested_feedback_ref text, + requested_run_ref text, requested_user_id uuid, + requested_membership_id uuid, + requested_membership_version bigint, + requested_principal_ref text, requested_rating text, + requested_note text + ) RETURNS text LANGUAGE plpgsql SECURITY DEFINER + SET search_path = pg_catalog SET row_security = on AS $function$ + DECLARE recorded_ref text; + BEGIN + IF SESSION_USER <> '{_RUNTIME}' + OR NULLIF(current_setting('app.organization_id', true), '')::uuid + IS DISTINCT FROM requested_organization_id + OR current_setting('app.actor_kind', true) <> 'user' + OR NULLIF(current_setting('app.user_id', true), '')::uuid + IS DISTINCT FROM requested_user_id + OR NULLIF(current_setting('app.membership_id', true), '')::uuid + IS DISTINCT FROM requested_membership_id + OR NULLIF( + current_setting('app.membership_version', true), '' + )::bigint IS DISTINCT FROM requested_membership_version + OR current_setting('app.principal_ref', true) + IS DISTINCT FROM requested_principal_ref + OR requested_feedback_ref !~ '^fb_[0-9a-f]{{64}}$' + OR requested_rating NOT IN ('helpful','not_helpful') + OR requested_membership_version NOT BETWEEN 1 AND {_MAX} + OR requested_note IS NOT NULL AND ( + btrim(requested_note) = '' + OR char_length(requested_note) > 1000) + THEN RETURN NULL; END IF; + PERFORM pg_catalog.set_config( + 'app.ui_feedback_mode', 'capture', true + ); + PERFORM pg_catalog.set_config( + 'app.ui_feedback_organization_id', + requested_organization_id::text, true + ); + PERFORM pg_catalog.set_config( + 'app.ui_feedback_run_ref', requested_run_ref, true + ); + PERFORM pg_catalog.set_config( + 'app.ui_feedback_user_id', requested_user_id::text, true + ); + PERFORM pg_catalog.set_config( + 'app.ui_feedback_membership_id', + requested_membership_id::text, true + ); + PERFORM pg_catalog.set_config( + 'app.ui_feedback_membership_version', + requested_membership_version::text, true + ); + PERFORM pg_catalog.set_config( + 'app.ui_feedback_principal_ref', requested_principal_ref, true + ); + IF NOT EXISTS ( + SELECT 1 FROM public.membership AS actor_membership + WHERE actor_membership.organization_id = + requested_organization_id + AND actor_membership.user_id = requested_user_id + AND actor_membership.membership_id = requested_membership_id + AND actor_membership.membership_version = + requested_membership_version + AND actor_membership.status = 'active' + AND actor_membership.valid_from <= + pg_catalog.clock_timestamp() + AND ( + actor_membership.valid_until IS NULL + OR actor_membership.valid_until > + pg_catalog.clock_timestamp() + ) + ) THEN RETURN NULL; END IF; + IF NOT EXISTS ( + SELECT 1 FROM public.context_run AS run + WHERE run.organization_id = requested_organization_id + AND run.run_ref = requested_run_ref + AND run.user_id = requested_user_id + AND run.membership_id = requested_membership_id + AND run.membership_version = requested_membership_version + AND run.principal_ref = requested_principal_ref + ) THEN RETURN NULL; END IF; + INSERT INTO public.context_feedback ( + organization_id, feedback_ref, run_ref, user_id, + membership_id, membership_version, principal_ref, + rating, note, recorded_at + ) VALUES ( + requested_organization_id, requested_feedback_ref, + requested_run_ref, requested_user_id, + requested_membership_id, requested_membership_version, + requested_principal_ref, requested_rating, requested_note, + pg_catalog.statement_timestamp() + ) ON CONFLICT ( + organization_id, run_ref, membership_id, membership_version + ) DO NOTHING; + SELECT feedback.feedback_ref INTO recorded_ref + FROM public.context_feedback AS feedback + WHERE feedback.organization_id = requested_organization_id + AND feedback.run_ref = requested_run_ref + AND feedback.membership_id = requested_membership_id + AND feedback.membership_version = requested_membership_version; + RETURN recorded_ref; + END; $function$ + """ + ) + op.execute( + f"REVOKE ALL ON FUNCTION public.{_CAPTURE_FEEDBACK}" + f"{_CAPTURE_FEEDBACK_SIGNATURE} FROM PUBLIC" + ) + op.execute( + f"GRANT EXECUTE ON FUNCTION public.{_CAPTURE_FEEDBACK}" + f"{_CAPTURE_FEEDBACK_SIGNATURE} TO {_RUNTIME}" + ) + op.execute(f"GRANT CREATE ON SCHEMA public TO {_RUN_READER_DEFINER}") + op.execute( + f"ALTER FUNCTION public.{_CAPTURE_FEEDBACK}{_CAPTURE_FEEDBACK_SIGNATURE} " + f"OWNER TO {_RUN_READER_DEFINER}" + ) + op.execute(f"REVOKE CREATE ON SCHEMA public FROM {_RUN_READER_DEFINER}") + + +def upgrade() -> None: + """Add exact confirmation, Article edit, and evidence-only feedback.""" + + _join_file_operation_fences() + op.add_column("file_acquisition", sa.Column("ui_preview_digest", sa.Text())) + op.add_column("file_acquisition", sa.Column("expected_fragment_digest", sa.Text())) + op.add_column("file_acquisition", sa.Column("compiler_config_version", sa.Text())) + op.drop_constraint( + "ck_file_acquisition_change_observation", + "file_acquisition", + type_="check", + ) + op.create_check_constraint( + "ck_file_acquisition_change_observation", + "file_acquisition", + "((change_page_ref IS NULL AND change_ordinal IS NULL " + "AND expected_content_sha256 IS NULL AND expected_content_length IS NULL " + "AND ui_preview_digest IS NULL AND expected_fragment_digest IS NULL " + "AND compiler_config_version IS NULL) OR " + "(change_page_ref IS NOT NULL AND change_ordinal BETWEEN 1 AND 100 " + "AND expected_content_sha256 ~ '^[0-9a-f]{64}$' " + "AND expected_content_length >= 0 AND ui_preview_digest IS NULL " + "AND expected_fragment_digest IS NULL AND compiler_config_version IS NULL) OR " + "(change_page_ref IS NULL AND change_ordinal IS NULL " + "AND expected_content_sha256 ~ '^[0-9a-f]{64}$' " + "AND expected_content_length >= 0 " + "AND ui_preview_digest ~ '^[0-9a-f]{64}$' " + "AND expected_fragment_digest ~ '^[0-9a-f]{64}$' " + "AND compiler_config_version = 'markdown-config-v1'))", + ) + _replace_redeem_ui_fields(install=True) + _create_exact_import() + _create_article_functions() + _create_feedback() + + +def downgrade() -> None: + """Remove the M1 UI-specific persistence seams.""" + + _join_file_operation_fences() + op.execute("LOCK TABLE file_acquisition IN ACCESS EXCLUSIVE MODE") + op.execute(f"DROP FUNCTION public.{_CAPTURE_FEEDBACK}{_CAPTURE_FEEDBACK_SIGNATURE}") + op.execute(f"REVOKE SELECT ON TABLE membership FROM {_RUN_READER_DEFINER}") + op.execute("DROP POLICY membership_ui_feedback_definer_select ON membership") + op.execute("DROP POLICY context_run_ui_feedback_definer_select ON context_run") + op.drop_table("context_feedback") + for name, signature in ( + (_CHANGE_ARTICLE, _CHANGE_ARTICLE_SIGNATURE), + (_READ_ARTICLE, _READ_ARTICLE_SIGNATURE), + ): + op.execute(f"DROP FUNCTION public.{name}{signature}") + op.execute( + "REVOKE INSERT, UPDATE ON TABLE article_explicit_policy_setting " + f"FROM {_ACCESS_DEFINER}" + ) + op.execute(f"REVOKE SELECT ON TABLE membership FROM {_ACCESS_DEFINER}") + op.execute("DROP POLICY membership_ui_access_definer_select ON membership") + op.execute( + "DROP POLICY article_explicit_policy_setting_access_definer_update " + "ON article_explicit_policy_setting" + ) + op.execute( + "DROP POLICY article_explicit_policy_setting_access_definer_insert " + "ON article_explicit_policy_setting" + ) + op.execute(f"DROP FUNCTION public.{_EXACT_IMPORT}{_EXACT_IMPORT_SIGNATURE}") + _replace_redeem_ui_fields(install=False) + op.drop_constraint( + "ck_file_acquisition_change_observation", + "file_acquisition", + type_="check", + ) + op.create_check_constraint( + "ck_file_acquisition_change_observation", + "file_acquisition", + "(change_page_ref IS NULL AND change_ordinal IS NULL " + "AND expected_content_sha256 IS NULL " + "AND expected_content_length IS NULL) OR " + "(change_page_ref IS NOT NULL " + "AND change_ordinal BETWEEN 1 AND 100 " + "AND expected_content_sha256 ~ '^[0-9a-f]{64}$' " + "AND expected_content_length >= 0)", + ) + op.drop_column("file_acquisition", "compiler_config_version") + op.drop_column("file_acquisition", "expected_fragment_digest") + op.drop_column("file_acquisition", "ui_preview_digest") diff --git a/pyproject.toml b/pyproject.toml index bc5ce6a2..d28b09b1 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -21,6 +21,7 @@ dependencies = [ "cryptography>=49,<50", "fastapi>=0.116,<0.117", "jsonschema>=4.25,<5", + "jinja2>=3.1,<3.2", "markdown>=3.6,<3.7", "pydantic>=2.13,<2.14", "psycopg[binary]>=3.2,<3.3", @@ -66,6 +67,7 @@ packages = [ "eval", "migrations", "third_party", + "ui", ] [tool.hatch.build.targets.wheel.force-include] @@ -86,6 +88,7 @@ files = [ "applications", "eval", "migrations", + "ui", "scripts/provision_database_roles.py", "scripts/third_party_governance.py", "scripts/wait_for_database.py", diff --git a/scripts/security_gate/rls.py b/scripts/security_gate/rls.py index 6fa133a8..c04957da 100644 --- a/scripts/security_gate/rls.py +++ b/scripts/security_gate/rls.py @@ -63,6 +63,7 @@ "organization_policy_epoch": "PG-REVOCATION-006", "resource_access_policy": "PG-REVOCATION-006", "context_run": "PG-TRACE-REDACTION-012", + "context_feedback": "PG-TRACE-REDACTION-012", "context_run_operator_read_ticket": "PG-TRACE-REDACTION-012", "decision_audit": "PG-TRACE-REDACTION-012", "service_principal": "PG-WORKER-LEASE-007", diff --git a/scripts/validate_security_catalog.py b/scripts/validate_security_catalog.py index fdf64d47..60aecffe 100644 --- a/scripts/validate_security_catalog.py +++ b/scripts/validate_security_catalog.py @@ -2179,7 +2179,9 @@ "Absent, wrong, or truncated secrets and revoked or expired configured " "Membership authority return the same generic authentication failure; " "the secret is absent from responses, logs, ContextRun, and " - "DecisionAudit." + "DecisionAudit. A successful dogfood resolve issues one private File " + "CitationOpenRef that reopens only through a second current-Membership " + "sealed Runtime resolve." ), }, ], @@ -2199,7 +2201,6 @@ "network exposure beyond the maintainer machine", "group or public audience", "Continue", - "dogfood OpenCitation", "hybrid retrieval", "external query embedding", "non-File providers", @@ -2209,7 +2210,9 @@ CANONICAL_DOGFOOD_RUNTIME_ACTIVATION: dict[str, object] = { "issueRef": "#102", "invariantRef": "INDEX-NOT-AUTHORITY-005", - "carrier": "loopback-only File pgvector dogfood Acquire delivery", + "carrier": ( + "loopback-only File pgvector dogfood Acquire and private OpenCitation delivery" + ), "status": "active_fail_closed", "policyEpochScope": "organization-v0", "controlBoundary": ( @@ -2262,7 +2265,6 @@ "network exposure beyond the maintainer machine", "group or public audience", "Continue", - "dogfood OpenCitation", "hybrid retrieval", "external query embedding", "non-File providers", diff --git a/tests/integration/test_access_policy_revocation.py b/tests/integration/test_access_policy_revocation.py index 4a2e6b9d..bc5c1444 100644 --- a/tests/integration/test_access_policy_revocation.py +++ b/tests/integration/test_access_policy_revocation.py @@ -272,8 +272,7 @@ def access_fixture( ) connection.execute( text( - "DELETE FROM user_account " - "WHERE user_id IN (:user_a, :user_b)" + "DELETE FROM user_account WHERE user_id IN (:user_a, :user_b)" ), {"user_a": fixture.user_a, "user_b": fixture.user_b}, ) @@ -532,12 +531,15 @@ def test_rejected_access_change_rolls_back_epoch_and_policy_together( {"organization_id": access_fixture.organization_a}, ).scalar_one() == (2 if failure == "already-revoked" else 1) if expected_state is not None: - assert _state( - migration_engine, - access_fixture.organization_a, - access_fixture.resource_a_one, - access_fixture.principal_a, - ) == expected_state + assert ( + _state( + migration_engine, + access_fixture.organization_a, + access_fixture.resource_a_one, + access_fixture.principal_a, + ) + == expected_state + ) finally: control_engine.dispose() migration_engine.dispose() @@ -687,14 +689,17 @@ def test_same_organization_concurrent_revocations_have_no_lost_epoch_bump( ), ) try: - with migration_engine.connect() as lock_connection, psycopg.connect( - host="127.0.0.1", - port=int(os.environ["CONTEXT_ENGINE_POSTGRES_PORT"]), - dbname=os.environ["POSTGRES_DB"], - user=os.environ["POSTGRES_USER"], - password=os.environ["POSTGRES_PASSWORD"], - autocommit=True, - ) as observer_connection: + with ( + migration_engine.connect() as lock_connection, + psycopg.connect( + host="127.0.0.1", + port=int(os.environ["CONTEXT_ENGINE_POSTGRES_PORT"]), + dbname=os.environ["POSTGRES_DB"], + user=os.environ["POSTGRES_USER"], + password=os.environ["POSTGRES_PASSWORD"], + autocommit=True, + ) as observer_connection, + ): lock_transaction = lock_connection.begin() try: lock_connection.execute( @@ -732,9 +737,7 @@ def test_same_organization_concurrent_revocations_have_no_lost_epoch_bump( break sleep(0.01) lock_transaction.commit() - epochs = tuple( - future.result(timeout=10) for future in futures - ) + epochs = tuple(future.result(timeout=10) for future in futures) finally: if lock_transaction.is_active: lock_transaction.rollback() @@ -801,13 +804,15 @@ def test_runtime_can_read_own_epoch_and_grant_but_cannot_mutate_them( text("SELECT set_config(:name, :value, true)"), {"name": name, "value": value}, ) - assert connection.execute( - text("SELECT policy_epoch FROM organization_policy_epoch") - ).scalar_one() == 1 + assert ( + connection.execute( + text("SELECT policy_epoch FROM organization_policy_epoch") + ).scalar_one() + == 1 + ) assert connection.execute( text( - "SELECT resource_ref FROM resource_access_policy " - "ORDER BY resource_ref" + "SELECT resource_ref FROM resource_access_policy ORDER BY resource_ref" ) ).scalars().all() == sorted( [access_fixture.resource_a_one, access_fixture.resource_a_two] @@ -832,9 +837,7 @@ def test_control_role_cannot_bypass_atomic_operation_with_direct_dml( with control_engine.begin() as connection: connection.execute( text( - "SELECT set_config(" - "'app.organization_id', :organization_id, true" - ")" + "SELECT set_config('app.organization_id', :organization_id, true)" ), {"organization_id": str(access_fixture.organization_a)}, ) @@ -869,9 +872,7 @@ def test_control_operation_rejects_a_cross_organization_session_binding( ): connection.execute( text( - "SELECT set_config(" - "'app.organization_id', :organization_id, true" - ")" + "SELECT set_config('app.organization_id', :organization_id, true)" ), {"organization_id": str(access_fixture.organization_b)}, ) @@ -1234,7 +1235,9 @@ def test_control_function_and_table_grants_seal_the_only_mutation_path( ("article_access_policy", "INSERT"), ("article_access_policy", "SELECT"), ("article_access_policy", "UPDATE"), + ("article_explicit_policy_setting", "INSERT"), ("article_explicit_policy_setting", "SELECT"), + ("article_explicit_policy_setting", "UPDATE"), ("article_source_acl_observation", "INSERT"), ("article_source_acl_observation", "SELECT"), ("article_source_acl_observation", "UPDATE"), @@ -1261,6 +1264,7 @@ def test_control_function_and_table_grants_seal_the_only_mutation_path( (ACCESS_POLICY_DEFINER_ROLE, "file_resource_cleanup_intent", "SELECT"), (ACCESS_POLICY_DEFINER_ROLE, "file_source_cleanup_intent", "INSERT"), (ACCESS_POLICY_DEFINER_ROLE, "file_source_cleanup_intent", "SELECT"), + (ACCESS_POLICY_DEFINER_ROLE, "membership", "SELECT"), (ACCESS_POLICY_DEFINER_ROLE, "organization_policy_epoch", "SELECT"), (ACCESS_POLICY_DEFINER_ROLE, "organization_policy_epoch", "UPDATE"), (ACCESS_POLICY_DEFINER_ROLE, "resource_access_policy", "SELECT"), @@ -1308,6 +1312,19 @@ def test_control_function_and_table_grants_seal_the_only_mutation_path( "requested_organization_id uuid, expected_version bigint, " "requested_policy_kind text, requested_group_refs text[]", ), + ( + "context_control_read_article_policy", + "requested_organization_id uuid, requested_resource_ref text", + ), + ( + "context_control_change_article_policy", + "requested_organization_id uuid, requested_resource_ref text, " + "expected_policy_version bigint, expected_policy_epoch bigint, " + "requested_policy_kind text, requested_group_refs text[], " + "requested_preview_digest text, requested_user_id uuid, " + "requested_membership_id uuid, " + "requested_membership_version bigint", + ), ( "context_fix_article_access_policy", "requested_organization_id uuid, requested_resource_ref text", @@ -1354,26 +1371,25 @@ def test_control_function_and_table_grants_seal_the_only_mutation_path( "requested_organization_id uuid, requested_source_id uuid, " "requested_cleanup_intent_id uuid", ), - ( - "public", - "context_runtime_file_source_lifecycle_allows", - "requested_organization_id uuid, requested_source_ref text", - ), - ( - "public", - "context_runtime_article_source_version_allows", - "requested_organization_id uuid, requested_resource_ref text, " - "expected_source_version_ref uuid", - ), - } + ( + "public", + "context_runtime_file_source_lifecycle_allows", + "requested_organization_id uuid, requested_source_ref text", + ), + ( + "public", + "context_runtime_article_source_version_allows", + "requested_organization_id uuid, requested_resource_ref text, " + "expected_source_version_ref uuid", + ), + } assert definer_owned_namespaces == set() assert definer_owned_databases == set() article_policy_commands = article_policy_definer_grants - { ("source_version", "SELECT") } assert { - (table, command, roles) - for table, command, roles, _, _ in definer_policies + (table, command, roles) for table, command, roles, _, _ in definer_policies } == { (table_name, command, (ACCESS_POLICY_DEFINER_ROLE,)) for table_name, command in article_policy_commands @@ -1403,6 +1419,11 @@ def test_control_function_and_table_grants_seal_the_only_mutation_path( "INSERT", (ACCESS_POLICY_DEFINER_ROLE,), ), + ( + "membership", + "SELECT", + (ACCESS_POLICY_DEFINER_ROLE,), + ), ( "context_resource", "SELECT", @@ -1455,7 +1476,7 @@ def test_control_function_and_table_grants_seal_the_only_mutation_path( ), ("source_version", "SELECT", (ACCESS_POLICY_DEFINER_ROLE,)), } - for _, command, _, using_expression, check_expression in definer_policies: + for table, command, _, using_expression, check_expression in definer_policies: if command == "SELECT": assert check_expression is None tenant_expression = using_expression @@ -1466,6 +1487,16 @@ def test_control_function_and_table_grants_seal_the_only_mutation_path( assert using_expression == check_expression tenant_expression = using_expression assert tenant_expression is not None - assert "app.organization_id" in tenant_expression + if table == "membership": + for required_fragment in ( + "app.ui_actor_organization_id", + "app.ui_actor_user_id", + "app.ui_actor_membership_id", + "app.ui_actor_membership_version", + "article_policy_change", + ): + assert required_fragment in tenant_expression + else: + assert "app.organization_id" in tenant_expression finally: migration_engine.dispose() diff --git a/tests/integration/test_context_run_schema.py b/tests/integration/test_context_run_schema.py index 5dd8d004..8fd74b8c 100644 --- a/tests/integration/test_context_run_schema.py +++ b/tests/integration/test_context_run_schema.py @@ -107,6 +107,13 @@ def lineage_identity( yield identity finally: with engine.begin() as connection: + connection.execute( + text( + "DELETE FROM context_feedback " + "WHERE organization_id = :organization_id" + ), + {"organization_id": identity.organization_id}, + ) connection.execute( text( "DELETE FROM decision_audit " diff --git a/tests/integration/test_dogfood_runtime_activation.py b/tests/integration/test_dogfood_runtime_activation.py index 276ff132..8da6ebad 100644 --- a/tests/integration/test_dogfood_runtime_activation.py +++ b/tests/integration/test_dogfood_runtime_activation.py @@ -115,6 +115,7 @@ def _configuration( def _environment( configuration: DogfoodConfiguration, runtime_configuration: DatabaseConfiguration, + control_configuration: DatabaseConfiguration, ) -> dict[str, str]: return { DOGFOOD_COMPOSITION_ENV: DOGFOOD_COMPOSITION_VALUE, @@ -132,6 +133,10 @@ def _environment( "CONTEXT_ENGINE_RUNTIME_DATABASE_URL": ( runtime_configuration.url.render_as_string(hide_password=False) ), + "CONTEXT_ENGINE_CONTROL_ROLE": control_configuration.expected_role, + "CONTEXT_ENGINE_CONTROL_DATABASE_URL": ( + control_configuration.url.render_as_string(hide_password=False) + ), } @@ -185,6 +190,18 @@ def _delete( migration_configuration: DatabaseConfiguration, organization_id: UUID, ) -> None: + engine = create_database_engine(migration_configuration) + try: + with engine.begin() as connection: + connection.execute( + text( + "DELETE FROM citation_open_locator " + "WHERE organization_id = :organization_id" + ), + {"organization_id": organization_id}, + ) + finally: + engine.dispose() clear_test_runtime_release(organization_id) delete_file_import_scenario(migration_configuration, organization_id) @@ -362,6 +379,7 @@ def test_dogfood_served_composition_delivers_release_scoped_file_evidence_before tmp_path: Path, migration_configuration: DatabaseConfiguration, runtime_configuration: DatabaseConfiguration, + control_configuration: DatabaseConfiguration, guarded_control_engine: Engine, guarded_worker_engine: Engine, monkeypatch: pytest.MonkeyPatch, @@ -384,7 +402,11 @@ def test_dogfood_served_composition_delivers_release_scoped_file_evidence_before ) configuration = _configuration(scenario, user_id) served: dict[str, object] = {} - for name, value in _environment(configuration, runtime_configuration).items(): + for name, value in _environment( + configuration, + runtime_configuration, + control_configuration, + ).items(): monkeypatch.setenv(name, value) def observe(app: object, **kwargs: object) -> None: @@ -418,6 +440,7 @@ def test_dogfood_rejects_an_active_release_with_an_unbound_embedding_profile( tmp_path: Path, migration_configuration: DatabaseConfiguration, runtime_configuration: DatabaseConfiguration, + control_configuration: DatabaseConfiguration, guarded_control_engine: Engine, guarded_worker_engine: Engine, ) -> None: @@ -433,7 +456,11 @@ def test_dogfood_rejects_an_active_release_with_an_unbound_embedding_profile( with pytest.raises(DogfoodConfigurationUnavailable): create_served_app( - _environment(configuration, runtime_configuration), + _environment( + configuration, + runtime_configuration, + control_configuration, + ), host="127.0.0.1", ) @@ -458,7 +485,11 @@ def test_dogfood_evaluator_scores_real_public_resolve_evidence( client = TestClient( create_dogfood_app( configuration, - _environment(configuration, runtime_configuration), + _environment( + configuration, + runtime_configuration, + control_configuration, + ), host="127.0.0.1", ) ) @@ -523,6 +554,7 @@ def test_dogfood_secret_and_membership_fail_closed_without_secret_retention( tmp_path: Path, migration_configuration: DatabaseConfiguration, runtime_configuration: DatabaseConfiguration, + control_configuration: DatabaseConfiguration, guarded_control_engine: Engine, guarded_worker_engine: Engine, caplog: pytest.LogCaptureFixture, @@ -538,13 +570,31 @@ def test_dogfood_secret_and_membership_fail_closed_without_secret_retention( client = TestClient( create_dogfood_app( configuration, - _environment(configuration, runtime_configuration), + _environment( + configuration, + runtime_configuration, + control_configuration, + ), host="127.0.0.1", ) ) successful = _resolve(client) assert successful.status_code == 200 assert successful.json()["package"]["evidence"] + citation_ref = successful.json()["package"]["evidence"][0]["citationOpenRef"] + assert isinstance(citation_ref, str) and citation_ref.startswith("cor_") + opened = client.post( + "/v0/resolve", + headers={ + "Authorization": f"Bearer {SECRET}", + "X-Context-Request-Id": "dogfood-open-citation", + }, + json={"kind": "open_citation", "citationOpenRef": citation_ref}, + ) + assert opened.status_code == 200 + assert opened.json()["package"]["evidence"][0]["resourceRef"] == ( + successful.json()["package"]["evidence"][0]["resourceRef"] + ) with caplog.at_level(logging.DEBUG): responses = ( @@ -642,7 +692,11 @@ def test_dogfood_mid_resolve_policy_epoch_change_vetoes_stale_evidence( client = TestClient( create_dogfood_app( configuration, - _environment(configuration, runtime_configuration), + _environment( + configuration, + runtime_configuration, + control_configuration, + ), host="127.0.0.1", ) ) @@ -1143,7 +1197,11 @@ def test_executed_run_observes_a_clean_report_from_the_real_runtime( client = TestClient( create_dogfood_app( configuration, - _environment(configuration, runtime_configuration), + _environment( + configuration, + runtime_configuration, + control_configuration, + ), host="127.0.0.1", ) ) diff --git a/tests/integration/test_feedback_has_no_publication_authority.py b/tests/integration/test_feedback_has_no_publication_authority.py new file mode 100644 index 00000000..d2411dd8 --- /dev/null +++ b/tests/integration/test_feedback_has_no_publication_authority.py @@ -0,0 +1,144 @@ +from __future__ import annotations + +from datetime import UTC, datetime + +import pytest +from fastapi.testclient import TestClient +from sqlalchemy import Engine, text + +from adapters.http.app import create_app +from adapters.http.authentication import VerifiedAuthenticationContext +from adapters.http.ui_api import PostgreSQLUiApi +from engine.persistence import ( + DatabaseConfiguration, + PostgreSQLMembershipAuthority, + create_database_engine, +) +from tests.integration.test_context_run_schema import ( + LineageIdentity, + insert_context_run, +) +from tests.integration.test_context_run_schema import ( + lineage_identity as _lineage_identity, +) +from tests.support.ui import authenticate_ui + +pytestmark = pytest.mark.integration +TOKEN = "ui-feedback-evidence-token" +lineage_identity = _lineage_identity + + +class _Authenticator: + def __init__(self, identity: LineageIdentity) -> None: + self.identity = identity + + def authenticate(self, opaque_credential: str) -> VerifiedAuthenticationContext: + assert opaque_credential == TOKEN + return VerifiedAuthenticationContext( + organization_ref=str(self.identity.organization_id), + user_ref=str(self.identity.user_id), + principal_ref="principal:issue-19", + membership_ref=str(self.identity.membership_id), + membership_version=1, + agent_version_ref="agent:issue-19", + authenticated_application_ref="application:issue-19", + authentication_binding_ref="binding:issue-19", + ) + + +def test_feedback_has_no_publication_authority( + lineage_identity: LineageIdentity, + migration_configuration: DatabaseConfiguration, + guarded_runtime_engine: Engine, +) -> None: + migration_engine = create_database_engine(migration_configuration) + try: + with migration_engine.begin() as connection: + insert_context_run(connection, lineage_identity) + release_count_before = connection.execute( + text( + "SELECT count(*) FROM active_release_manifest " + "WHERE organization_id = :organization_id" + ), + {"organization_id": lineage_identity.organization_id}, + ).scalar_one() + client = TestClient( + create_app( + authenticator=_Authenticator(lineage_identity), + ui_bearer_token=TOKEN, + ui_api=PostgreSQLUiApi( + PostgreSQLMembershipAuthority(guarded_runtime_engine), + None, + preview_key=b"f" * 32, + feedback_engine=guarded_runtime_engine, + clock=lambda: datetime.now(UTC), + ), + ) + ) + authenticate_ui(client, TOKEN) + + response = client.post( + "/ui/feedback", + content=( + f"runRef={lineage_identity.run_ref}&rating=helpful&" + "note=Lineage+was+clear" + ), + headers={"Content-Type": "application/x-www-form-urlencoded"}, + ) + + assert response.status_code == 200 + assert "Feedback recorded" in response.text + with migration_engine.connect() as connection: + feedback = connection.execute( + text( + """ + SELECT run_ref, rating, note + FROM context_feedback + WHERE organization_id = :organization_id + """ + ), + {"organization_id": lineage_identity.organization_id}, + ).one() + release_count_after = connection.execute( + text( + "SELECT count(*) FROM active_release_manifest " + "WHERE organization_id = :organization_id" + ), + {"organization_id": lineage_identity.organization_id}, + ).scalar_one() + privileges = tuple( + connection.execute( + text( + """ + SELECT + has_function_privilege( + 'context_engine_runtime', + 'context_runtime_capture_context_feedback' + '(uuid,text,text,uuid,uuid,bigint,text,text,text)', + 'EXECUTE' + ), + has_function_privilege( + 'context_engine_control', + 'context_runtime_capture_context_feedback' + '(uuid,text,text,uuid,uuid,bigint,text,text,text)', + 'EXECUTE' + ), + has_function_privilege( + 'context_engine_release_operator', + 'context_runtime_capture_context_feedback' + '(uuid,text,text,uuid,uuid,bigint,text,text,text)', + 'EXECUTE' + ) + """ + ) + ).one() + ) + assert tuple(feedback) == ( + lineage_identity.run_ref, + "helpful", + "Lineage was clear", + ) + assert release_count_after == release_count_before + assert privileges == (True, False, False) + finally: + migration_engine.dispose() diff --git a/tests/integration/test_hit_test_shows_only_authorized.py b/tests/integration/test_hit_test_shows_only_authorized.py new file mode 100644 index 00000000..bb5deb99 --- /dev/null +++ b/tests/integration/test_hit_test_shows_only_authorized.py @@ -0,0 +1,252 @@ +from __future__ import annotations + +from datetime import UTC, datetime +from typing import Any, Never, cast + +import pytest +from fastapi.testclient import TestClient +from sqlalchemy import Engine, text + +from adapters.http.app import create_app +from engine.persistence import ( + DatabaseConfiguration, + PostgreSQLMembershipAuthority, + create_database_engine, +) +from engine.runtime.citation import PRIVATE_FILE_CITATION_OPEN_PROFILE +from engine.runtime.construction import Runtime, required_kernel_dependencies +from engine.runtime.content_io import CandidateIndex, CandidateIndexUnavailable +from engine.runtime.package_digest import QueryDigestKeyring +from tests.integration.test_runtime_authorized_evidence_integration import ( + ORG_A_DENIED_BODY, + ORG_B_AUTHORIZED_BODY, + RECEIVED_AT, + ExactScopeAuthority, + HostileCandidateIndex, + RuntimeEvidenceFixture, + SeededAuthenticator, + SeededOrganizationAuthority, + _cleanup_fixture, + _new_fixture, + _seed_fixture, +) +from tests.support.releases import ensure_test_runtime_release +from tests.support.ui import authenticate_ui + +pytestmark = pytest.mark.integration +TOKEN = "authorized-hit-test-secret" + + +def test_hit_test_shows_only_authorized( + migration_configuration: DatabaseConfiguration, + guarded_runtime_engine: Engine, + query_digest_keyring: QueryDigestKeyring, +) -> None: + """Mixed PG candidates disclose only post-Kernel authorized hit facts.""" + + fixture = _new_fixture() + migration_engine = create_database_engine(migration_configuration) + request_now = datetime.now(UTC).replace(microsecond=0) + try: + _seed_fixture(migration_engine, fixture) + ensure_test_runtime_release(fixture.org_a.organization_id) + index = HostileCandidateIndex( + fixture.org_a, + cross_organization=fixture.org_b.authorized, + ) + app = create_app( + authenticator=SeededAuthenticator(fixture.org_a, token=TOKEN), + organization_authority=SeededOrganizationAuthority( + fixture.org_a.organization_id + ), + membership_authority=PostgreSQLMembershipAuthority( + guarded_runtime_engine + ), + scope_authority=ExactScopeAuthority(fixture.org_a.authorized), + runtime=Runtime( + required_kernel_dependencies(), + candidate_index=cast(CandidateIndex, index), + citation_profile=PRIVATE_FILE_CITATION_OPEN_PROFILE, + clock=lambda: request_now, + query_digest_keyring=query_digest_keyring, + ), + clock=lambda: request_now, + ui_bearer_token=TOKEN, + ) + + client = TestClient(app) + authenticate_ui(client, TOKEN) + response = client.post( + "/ui/hit-test", + content="query=hostile+rank", + headers={"Content-Type": "application/x-www-form-urlencoded"}, + ) + + assert response.status_code == 200, response.text + assert response.headers["cache-control"] == "no-store" + assert fixture.org_a.authorized_body in response.text + assert fixture.org_a.authorized.resource_ref in response.text + assert fixture.org_a.authorized.revision_ref in response.text + assert fixture.org_a.authorized.fragment_ref in response.text + assert "Authorized hit 1" in response.text + assert "not_exposed_by_rank_free_public_contract" in response.text + _assert_refused_candidates_are_unobservable(response.text, fixture) + + answer = client.post( + "/ui/ask", + content="query=hostile+rank", + headers={"Content-Type": "application/x-www-form-urlencoded"}, + ) + assert answer.status_code == 200, answer.text + assert fixture.org_a.authorized_body in answer.text + assert "citation_unavailable" not in answer.text + _assert_refused_candidates_are_unobservable(answer.text, fixture) + finally: + try: + _cleanup_citation_lineage(migration_engine, fixture) + _cleanup_fixture(migration_engine, fixture) + finally: + migration_engine.dispose() + + +def test_hit_test_empty_and_error_states_do_not_reveal_refused_candidates( + migration_configuration: DatabaseConfiguration, + guarded_runtime_engine: Engine, + query_digest_keyring: QueryDigestKeyring, +) -> None: + fixture = _new_fixture() + migration_engine = create_database_engine(migration_configuration) + try: + _seed_fixture(migration_engine, fixture) + ensure_test_runtime_release(fixture.org_a.organization_id) + empty_app = create_app( + authenticator=SeededAuthenticator(fixture.org_a, token=TOKEN), + organization_authority=SeededOrganizationAuthority( + fixture.org_a.organization_id + ), + membership_authority=PostgreSQLMembershipAuthority( + guarded_runtime_engine + ), + # The scope names no discovered candidate, so every mixed candidate + # reaches the Kernel and the public result is authorized-empty. + scope_authority=ExactScopeAuthority(fixture.org_b.denied), + runtime=Runtime( + required_kernel_dependencies(), + candidate_index=cast( + CandidateIndex, + HostileCandidateIndex( + fixture.org_a, + cross_organization=fixture.org_b.authorized, + ), + ), + clock=lambda: RECEIVED_AT, + query_digest_keyring=query_digest_keyring, + ), + clock=lambda: RECEIVED_AT, + ui_bearer_token=TOKEN, + ) + empty_client = TestClient(empty_app) + authenticate_ui(empty_client, TOKEN) + + empty = empty_client.post( + "/ui/hit-test", + content="query=hostile+rank", + headers={"Content-Type": "application/x-www-form-urlencoded"}, + ) + + unavailable_index = HostileCandidateIndex( + fixture.org_a, + cross_organization=fixture.org_b.authorized, + ) + + def unavailable_discover(*args: object, **kwargs: object) -> Never: + del args, kwargs + raise CandidateIndexUnavailable + + cast(Any, unavailable_index).discover = unavailable_discover + error_app = create_app( + authenticator=SeededAuthenticator(fixture.org_a, token=TOKEN), + organization_authority=SeededOrganizationAuthority( + fixture.org_a.organization_id + ), + membership_authority=PostgreSQLMembershipAuthority( + guarded_runtime_engine + ), + scope_authority=ExactScopeAuthority(fixture.org_a.authorized), + runtime=Runtime( + required_kernel_dependencies(), + candidate_index=cast(CandidateIndex, unavailable_index), + clock=lambda: RECEIVED_AT, + query_digest_keyring=query_digest_keyring, + ), + clock=lambda: RECEIVED_AT, + ui_bearer_token=TOKEN, + ) + error_client = TestClient(error_app) + authenticate_ui(error_client, TOKEN) + error = error_client.post( + "/ui/hit-test", + content="query=hostile+rank", + headers={"Content-Type": "application/x-www-form-urlencoded"}, + ) + + assert empty.status_code == 200 + assert "No authorized evidence" in empty.text + assert "
([0-9a-f-]+)", confirmed.text)
+ assert job_match is not None
+ job_id = job_match.group(1)
+
+ with migration_engine.connect() as connection:
+ exact = connection.execute(
+ text(
+ """
+ SELECT acquisition.expected_content_sha256,
+ acquisition.expected_content_length,
+ acquisition.expected_fragment_digest,
+ acquisition.compiler_config_version,
+ job.service_principal_id
+ FROM file_import_job AS job
+ JOIN file_acquisition AS acquisition
+ ON acquisition.organization_id = job.organization_id
+ AND acquisition.acquisition_id = job.acquisition_id
+ WHERE job.organization_id = :organization_id
+ AND job.job_id = :job_id
+ """
+ ),
+ {
+ "organization_id": scenario.organization_id,
+ "job_id": job_id,
+ },
+ ).one()
+ assert exact.expected_content_sha256 is not None
+ assert exact.expected_fragment_digest is not None
+ assert exact.compiler_config_version == "markdown-config-v1"
+ prepared = PreparedFileImport(
+ organization_id=scenario.organization_id,
+ job_id=(exact_job_id := UUID(job_id)),
+ source_ref=scenario.source_ref,
+ service_principal_id=exact.service_principal_id,
+ )
+ token = PostgreSQLWorkerLeaseIssuer(
+ guarded_control_engine,
+ scenario.codec,
+ lease_ttl_seconds=300,
+ ).issue_file_import_lease(prepared)
+ published = run_file_import(
+ scenario,
+ prepared,
+ token,
+ guarded_worker_engine,
+ )
+ expected = compile_markdown(
+ (scenario.root / "handbook.md").read_bytes(),
+ MarkdownCompilerConfig("markdown-config-v1"),
+ )
+ assert type(expected) is ParsedDocument
+ assert exact_job_id == prepared.job_id
+ assert tuple(
+ candidate.fragment_ref for candidate in published.candidate_refs
+ ) == tuple(fragment.fragment_ref for fragment in expected.fragments)
+ finally:
+ roots.close()
+ migration_engine.dispose()
+ delete_file_import_scenario(
+ migration_configuration,
+ scenario.organization_id,
+ )
diff --git a/tests/integration/test_m0_security_gate_rls.py b/tests/integration/test_m0_security_gate_rls.py
index 02a3b9db..39d25b96 100644
--- a/tests/integration/test_m0_security_gate_rls.py
+++ b/tests/integration/test_m0_security_gate_rls.py
@@ -30,7 +30,7 @@ def _manifest() -> dict[str, object]:
def test_all_manifest_tenant_tables_pass_live_non_owner_rls_audit(
guarded_runtime_engine: Engine,
) -> None:
- """PG-RLS-ALL-TENANT-TABLES: the live denominator is exactly 67/67."""
+ """PG-RLS-ALL-TENANT-TABLES: the live denominator is exactly 68/68."""
with guarded_runtime_engine.connect() as connection:
report = audit_live_rls(
@@ -41,13 +41,13 @@ def test_all_manifest_tenant_tables_pass_live_non_owner_rls_audit(
assert report["passed"] is True
assert report["denominator"] == {
- "allTables": 70,
- "tenantOwned": 67,
+ "allTables": 71,
+ "tenantOwned": 68,
"global": 3,
}
assert report["coverage"] == {
- "numerator": 67,
- "denominator": 67,
+ "numerator": 68,
+ "denominator": 68,
"percent": 100.0,
}
assert report["failures"] == []
@@ -86,9 +86,9 @@ def test_no_force_row_level_security_mutation_fails_and_rolls_back(
assert mutated["passed"] is False
assert mutated["coverage"] == {
- "numerator": 66,
- "denominator": 67,
- "percent": 98.51,
+ "numerator": 67,
+ "denominator": 68,
+ "percent": 98.53,
}
tenant_tables = cast(list[dict[str, Any]], mutated["tenantTables"])
organization_record = next(
@@ -111,8 +111,8 @@ def test_no_force_row_level_security_mutation_fails_and_rolls_back(
)
assert restored["passed"] is True
assert restored["coverage"] == {
- "numerator": 67,
- "denominator": 67,
+ "numerator": 68,
+ "denominator": 68,
"percent": 100.0,
}
diff --git a/tests/integration/test_membership_schema.py b/tests/integration/test_membership_schema.py
index f407497a..6ca0cd44 100644
--- a/tests/integration/test_membership_schema.py
+++ b/tests/integration/test_membership_schema.py
@@ -13,9 +13,11 @@
from engine.persistence import DatabaseConfiguration, create_database_engine
from engine.persistence.configuration import (
+ ACCESS_POLICY_DEFINER_ROLE,
ACTION_EXECUTE_DEFINER_ROLE,
ACTION_PREPARE_DEFINER_ROLE,
CITATION_DEFINER_ROLE,
+ CONTEXT_RUN_READER_DEFINER_ROLE,
DELIVERY_EVIDENCE_DEFINER_ROLE,
EGRESS_GRANT_DEFINER_ROLE,
FILE_DISPATCH_DEFINER_ROLE,
@@ -228,9 +230,12 @@ def test_runtime_membership_rls_is_bidirectional_and_exact(
assert membership_rows(connection) == [
(organization_id, membership_id, user_id)
]
- assert connection.execute(
- text("SELECT count(*) FROM organization_record")
- ).scalar_one() == 1
+ assert (
+ connection.execute(
+ text("SELECT count(*) FROM organization_record")
+ ).scalar_one()
+ == 1
+ )
with user_actor_connection(
guarded_runtime_engine,
@@ -239,9 +244,12 @@ def test_runtime_membership_rls_is_bidirectional_and_exact(
membership_id=identities.membership_b,
) as connection:
assert membership_rows(connection) == []
- assert connection.execute(
- text("SELECT count(*) FROM organization_record")
- ).scalar_one() == 0
+ assert (
+ connection.execute(
+ text("SELECT count(*) FROM organization_record")
+ ).scalar_one()
+ == 0
+ )
@pytest.mark.parametrize(
@@ -299,9 +307,12 @@ def test_status_version_and_validity_fail_closed_at_the_database_seam(
membership_id=identities.membership_a,
) as connection:
assert membership_rows(connection) == []
- assert connection.execute(
- text("SELECT count(*) FROM organization_record")
- ).scalar_one() == 0
+ assert (
+ connection.execute(
+ text("SELECT count(*) FROM organization_record")
+ ).scalar_one()
+ == 0
+ )
finally:
migration_engine.dispose()
@@ -317,9 +328,12 @@ def test_organization_only_and_user_only_context_have_no_tenant_rights(
{"value": str(identities.organization_a)},
)
assert membership_rows(connection) == []
- assert connection.execute(
- text("SELECT count(*) FROM organization_record")
- ).scalar_one() == 0
+ assert (
+ connection.execute(
+ text("SELECT count(*) FROM organization_record")
+ ).scalar_one()
+ == 0
+ )
with pytest.raises(DBAPIError, match="current UserActor Membership"):
connection.execute(
text(
@@ -342,9 +356,12 @@ def test_organization_only_and_user_only_context_have_no_tenant_rights(
membership_id=uuid4(),
) as connection:
assert membership_rows(connection) == []
- assert connection.execute(
- text("SELECT count(*) FROM organization_record")
- ).scalar_one() == 0
+ assert (
+ connection.execute(
+ text("SELECT count(*) FROM organization_record")
+ ).scalar_one()
+ == 0
+ )
with pytest.raises(DBAPIError, match="current UserActor Membership"):
connection.execute(
text(
@@ -361,17 +378,18 @@ def test_organization_only_and_user_only_context_have_no_tenant_rights(
)
with guarded_runtime_engine.begin() as connection:
- connection.execute(
- text("SELECT set_config('app.actor_kind', 'user', true)")
- )
+ connection.execute(text("SELECT set_config('app.actor_kind', 'user', true)"))
connection.execute(
text("SELECT set_config('app.user_id', :value, true)"),
{"value": str(identities.user_a)},
)
assert membership_rows(connection) == []
- assert connection.execute(
- text("SELECT count(*) FROM organization_record")
- ).scalar_one() == 0
+ assert (
+ connection.execute(
+ text("SELECT count(*) FROM organization_record")
+ ).scalar_one()
+ == 0
+ )
def test_global_user_and_membership_constraints_reject_orphans_and_invalid_rows(
@@ -474,8 +492,10 @@ def test_runtime_worker_and_public_grants_are_least_privilege(
:egress_grant_definer_role,
:action_prepare_definer_role,
:action_execute_definer_role,
- :citation_definer_role,
- :file_dispatch_definer_role
+ :citation_definer_role,
+ :access_policy_definer_role,
+ :context_run_reader_definer_role,
+ :file_dispatch_definer_role
)
"""
),
@@ -489,6 +509,10 @@ def test_runtime_worker_and_public_grants_are_least_privilege(
"action_prepare_definer_role": ACTION_PREPARE_DEFINER_ROLE,
"action_execute_definer_role": ACTION_EXECUTE_DEFINER_ROLE,
"citation_definer_role": CITATION_DEFINER_ROLE,
+ "access_policy_definer_role": ACCESS_POLICY_DEFINER_ROLE,
+ "context_run_reader_definer_role": (
+ CONTEXT_RUN_READER_DEFINER_ROLE
+ ),
"file_dispatch_definer_role": FILE_DISPATCH_DEFINER_ROLE,
},
)
@@ -553,6 +577,8 @@ def test_runtime_worker_and_public_grants_are_least_privilege(
(ACTION_PREPARE_DEFINER_ROLE, "membership", "SELECT"),
(ACTION_EXECUTE_DEFINER_ROLE, "membership", "SELECT"),
(CITATION_DEFINER_ROLE, "membership", "SELECT"),
+ (ACCESS_POLICY_DEFINER_ROLE, "membership", "SELECT"),
+ (CONTEXT_RUN_READER_DEFINER_ROLE, "membership", "SELECT"),
(FILE_DISPATCH_DEFINER_ROLE, "membership", "SELECT"),
}
assert dispatch_update_columns == {"status", "valid_from", "valid_until"}
@@ -567,6 +593,8 @@ def test_runtime_worker_and_public_grants_are_least_privilege(
"membership_file_dispatch_definer_select",
"membership_file_dispatch_definer_update",
"membership_citation_definer_select",
+ "membership_ui_access_definer_select",
+ "membership_ui_feedback_definer_select",
"membership_migrator_administration",
}
runtime_policy = policies["membership_current_user_actor"]
@@ -651,6 +679,33 @@ def test_runtime_worker_and_public_grants_are_least_privilege(
None,
)
+ for policy_name, role, mode in (
+ (
+ "membership_ui_access_definer_select",
+ ACCESS_POLICY_DEFINER_ROLE,
+ "article_policy_change",
+ ),
+ (
+ "membership_ui_feedback_definer_select",
+ CONTEXT_RUN_READER_DEFINER_ROLE,
+ "capture",
+ ),
+ ):
+ policy = policies[policy_name]
+ assert policy[:3] == ("PERMISSIVE", (role,), "SELECT")
+ assert policy[3] is not None
+ assert policy[4] is None
+ normalized = str(policy[3]).lower()
+ for required_fragment in (
+ "app.ui_",
+ "organization_id",
+ "user_id",
+ "membership_id",
+ "membership_version",
+ mode,
+ ):
+ assert required_fragment in normalized
+
migrator_policy = policies["membership_migrator_administration"]
assert migrator_policy[:3] == (
"PERMISSIVE",
@@ -679,11 +734,17 @@ def test_runtime_worker_and_public_grants_are_least_privilege(
assert constraints["uq_membership_organization_id_version"].lower() == (
"unique (organization_id, membership_id, membership_version)"
)
- assert constraints["fk_membership_organization"].lower().startswith(
- "foreign key (organization_id) references organization(organization_id)"
+ assert (
+ constraints["fk_membership_organization"]
+ .lower()
+ .startswith(
+ "foreign key (organization_id) references organization(organization_id)"
+ )
)
- assert constraints["fk_membership_user_account"].lower().startswith(
- "foreign key (user_id) references user_account(user_id)"
+ assert (
+ constraints["fk_membership_user_account"]
+ .lower()
+ .startswith("foreign key (user_id) references user_account(user_id)")
)
finally:
engine.dispose()
@@ -723,9 +784,12 @@ def test_invalid_actor_mutations_hide_existing_rows_and_produce_zero_effects(
{"name": name, "value": value},
)
assert membership_rows(connection) == []
- assert connection.execute(
- text("SELECT count(*) FROM organization_record")
- ).scalar_one() == 0
+ assert (
+ connection.execute(
+ text("SELECT count(*) FROM organization_record")
+ ).scalar_one()
+ == 0
+ )
with pytest.raises(DBAPIError, match="current UserActor Membership"):
connection.execute(
text(
diff --git a/tests/integration/test_migrations.py b/tests/integration/test_migrations.py
index 915a63af..1e719766 100644
--- a/tests/integration/test_migrations.py
+++ b/tests/integration/test_migrations.py
@@ -124,6 +124,7 @@
"article_source_acl_observation",
"bulk_article_policy_change_audit",
"citation_open_locator",
+ "context_feedback",
"context_fragment",
"context_fragment_field",
"context_resource",
@@ -482,6 +483,7 @@ def _delete_issue_27_upgrade_fixture(
for table in (
"action_ticket",
"action_delivery_attempt",
+ "context_feedback",
"file_delete_observation_execution",
"file_source_publish_watermark",
"file_source_acquisition_checkpoint",
diff --git a/tests/integration/test_no_implicit_policy_change.py b/tests/integration/test_no_implicit_policy_change.py
new file mode 100644
index 00000000..6ac78d2a
--- /dev/null
+++ b/tests/integration/test_no_implicit_policy_change.py
@@ -0,0 +1,239 @@
+from __future__ import annotations
+
+import re
+from datetime import timedelta
+from uuid import UUID, uuid4
+
+import pytest
+from fastapi.testclient import TestClient
+from sqlalchemy import Engine, text
+
+from adapters.http.app import create_app
+from adapters.http.authentication import VerifiedAuthenticationContext
+from adapters.http.ui_api import PostgreSQLUiApi
+from engine.control import ControlOperation
+from engine.persistence import (
+ DatabaseConfiguration,
+ PostgreSQLMembershipAuthority,
+ create_database_engine,
+)
+from tests.support.article_access_policy import (
+ delete_article_policy_scenario,
+ ingest_article,
+ insert_organization,
+ observe_source_acl,
+ policy_epoch,
+ set_tenant_default,
+)
+from tests.support.file_imports import NOW
+from tests.support.ui import authenticate_ui, ui_control_authority
+
+pytestmark = pytest.mark.integration
+TOKEN = "ui-policy-confirm-token"
+CONTROL_TOKEN = "ui-policy-control-token"
+
+
+class _Authenticator:
+ def __init__(
+ self,
+ organization_id: UUID,
+ user_id: UUID,
+ membership_id: UUID,
+ ) -> None:
+ self.organization_id = organization_id
+ self.user_id = user_id
+ self.membership_id = membership_id
+
+ def authenticate(self, opaque_credential: str) -> VerifiedAuthenticationContext:
+ assert opaque_credential == TOKEN
+ return VerifiedAuthenticationContext(
+ organization_ref=str(self.organization_id),
+ user_ref=str(self.user_id),
+ principal_ref="principal:ui-policy",
+ membership_ref=str(self.membership_id),
+ membership_version=1,
+ agent_version_ref="agent:ui-policy",
+ authenticated_application_ref="application:ui-policy",
+ authentication_binding_ref="binding:ui-policy",
+ )
+
+
+def test_no_implicit_policy_change(
+ migration_configuration: DatabaseConfiguration,
+ guarded_runtime_engine: Engine,
+ guarded_control_engine: Engine,
+) -> None:
+ migration_engine = create_database_engine(migration_configuration)
+ organization_id, user_id, membership_id = uuid4(), uuid4(), uuid4()
+ source_ref = f"source:ui-policy:{uuid4()}"
+ resource_ref = f"resource:ui-policy:{uuid4()}"
+ try:
+ insert_organization(migration_engine, organization_id)
+ with migration_engine.begin() as connection:
+ connection.execute(
+ text("INSERT INTO user_account (user_id) VALUES (:user_id)"),
+ {"user_id": user_id},
+ )
+ connection.execute(
+ text(
+ """
+ INSERT INTO membership (
+ organization_id, membership_id, user_id, status,
+ membership_version, valid_from
+ ) VALUES (
+ :organization_id, :membership_id, :user_id,
+ 'active', 1, :valid_from
+ )
+ """
+ ),
+ {
+ "organization_id": organization_id,
+ "membership_id": membership_id,
+ "user_id": user_id,
+ "valid_from": NOW - timedelta(days=1),
+ },
+ )
+ set_tenant_default(migration_engine, organization_id, "private")
+ observe_source_acl(
+ migration_engine,
+ organization_id=organization_id,
+ source_ref=source_ref,
+ resource_ref=resource_ref,
+ )
+ ingest_article(
+ migration_engine,
+ organization_id=organization_id,
+ source_ref=source_ref,
+ resource_ref=resource_ref,
+ )
+ control_authority, control_gate = ui_control_authority(
+ organization_id=organization_id,
+ credential=CONTROL_TOKEN,
+ operations=frozenset(
+ {
+ ControlOperation.READ_ARTICLE_POLICY,
+ ControlOperation.CHANGE_ARTICLE_POLICY,
+ }
+ ),
+ clock=lambda: NOW,
+ )
+ client = TestClient(
+ create_app(
+ authenticator=_Authenticator(
+ organization_id,
+ user_id,
+ membership_id,
+ ),
+ ui_bearer_token=TOKEN,
+ ui_control_authority=control_authority,
+ ui_api=PostgreSQLUiApi(
+ PostgreSQLMembershipAuthority(guarded_runtime_engine),
+ guarded_control_engine,
+ preview_key=b"p" * 32,
+ control_gate=control_gate,
+ clock=lambda: NOW,
+ ),
+ )
+ )
+ authenticate_ui(client, TOKEN)
+ before = _policy_state(migration_engine, organization_id, resource_ref)
+
+ viewed = client.post(
+ "/ui/articles/view",
+ content=f"resourceRef={resource_ref}&controlCredential={CONTROL_TOKEN}",
+ headers={"Content-Type": "application/x-www-form-urlencoded"},
+ )
+ previewed = client.post(
+ "/ui/articles/preview",
+ content=(
+ f"resourceRef={resource_ref}&policyKind=organization&groupRefs=&"
+ f"controlCredential={CONTROL_TOKEN}"
+ ),
+ headers={"Content-Type": "application/x-www-form-urlencoded"},
+ )
+ assert viewed.status_code == 200
+ assert previewed.status_code == 200
+ assert CONTROL_TOKEN not in viewed.text
+ assert CONTROL_TOKEN not in previewed.text
+ assert "Preview · no historical change yet" in previewed.text
+ assert _policy_state(migration_engine, organization_id, resource_ref) == before
+ assert policy_epoch(migration_engine, organization_id) == before[1]
+
+ cancelled = client.post(
+ "/ui/articles/view",
+ content=f"resourceRef={resource_ref}&controlCredential={CONTROL_TOKEN}",
+ headers={"Content-Type": "application/x-www-form-urlencoded"},
+ )
+ assert cancelled.status_code == 200
+ assert _policy_state(migration_engine, organization_id, resource_ref) == before
+
+ match = re.search(
+ r'name="previewToken" value="([A-Za-z0-9_.-]+)"',
+ previewed.text,
+ )
+ assert match is not None
+ confirmed = client.post(
+ "/ui/articles/confirm",
+ content=(
+ f"previewToken={match.group(1)}&controlCredential={CONTROL_TOKEN}"
+ ),
+ headers={"Content-Type": "application/x-www-form-urlencoded"},
+ )
+ assert confirmed.status_code == 200
+ assert "Article policy changed" in confirmed.text
+ assert CONTROL_TOKEN not in confirmed.text
+ after = _policy_state(migration_engine, organization_id, resource_ref)
+ assert after == (before[0] + 1, before[1] + 1, "organization")
+
+ replay = client.post(
+ "/ui/articles/confirm",
+ content=(
+ f"previewToken={match.group(1)}&controlCredential={CONTROL_TOKEN}"
+ ),
+ headers={"Content-Type": "application/x-www-form-urlencoded"},
+ )
+ assert replay.status_code == 503
+ assert _policy_state(migration_engine, organization_id, resource_ref) == after
+ finally:
+ try:
+ with migration_engine.begin() as connection:
+ connection.execute(
+ text(
+ "DELETE FROM membership "
+ "WHERE organization_id = :organization_id"
+ ),
+ {"organization_id": organization_id},
+ )
+ connection.execute(
+ text("DELETE FROM user_account WHERE user_id = :user_id"),
+ {"user_id": user_id},
+ )
+ finally:
+ migration_engine.dispose()
+ delete_article_policy_scenario(migration_configuration, organization_id)
+
+
+def _policy_state(
+ engine: Engine,
+ organization_id: UUID,
+ resource_ref: str,
+) -> tuple[int, int, str | None]:
+ with engine.connect() as connection:
+ row = connection.execute(
+ text(
+ """
+ SELECT policy.policy_version, epoch.policy_epoch,
+ policy.policy_kind
+ FROM article_access_policy AS policy
+ JOIN organization_policy_epoch AS epoch
+ ON epoch.organization_id = policy.organization_id
+ WHERE policy.organization_id = :organization_id
+ AND policy.resource_ref = :resource_ref
+ """
+ ),
+ {
+ "organization_id": organization_id,
+ "resource_ref": resource_ref,
+ },
+ ).one()
+ return row.policy_version, row.policy_epoch, row.policy_kind
diff --git a/tests/integration/test_release_operator_process.py b/tests/integration/test_release_operator_process.py
index d16166ec..4e18617e 100644
--- a/tests/integration/test_release_operator_process.py
+++ b/tests/integration/test_release_operator_process.py
@@ -169,6 +169,7 @@ def _dogfood_environment(
user_id: UUID,
membership_id: UUID,
runtime_configuration: DatabaseConfiguration,
+ control_configuration: DatabaseConfiguration,
) -> dict[str, str]:
return {
DOGFOOD_COMPOSITION_ENV: DOGFOOD_COMPOSITION_VALUE,
@@ -186,6 +187,10 @@ def _dogfood_environment(
"CONTEXT_ENGINE_RUNTIME_DATABASE_URL": (
runtime_configuration.url.render_as_string(hide_password=False)
),
+ "CONTEXT_ENGINE_CONTROL_ROLE": control_configuration.expected_role,
+ "CONTEXT_ENGINE_CONTROL_DATABASE_URL": (
+ control_configuration.url.render_as_string(hide_password=False)
+ ),
}
@@ -424,6 +429,7 @@ def test_promote_release_activates_every_current_revision_and_dogfood_runtime(
release_evidence_file: Path,
migration_configuration: DatabaseConfiguration,
runtime_configuration: DatabaseConfiguration,
+ control_configuration: DatabaseConfiguration,
guarded_control_engine: Engine,
guarded_learning_engine: Engine,
guarded_release_operator_engine: Engine,
@@ -503,6 +509,7 @@ def test_promote_release_activates_every_current_revision_and_dogfood_runtime(
user_id=user_id,
membership_id=scenario.membership_id,
runtime_configuration=runtime_configuration,
+ control_configuration=control_configuration,
)
with pytest.raises(DogfoodConfigurationUnavailable):
create_served_app(dogfood_environment, host="127.0.0.1")
diff --git a/tests/integration/test_visibility_view_shows_rung.py b/tests/integration/test_visibility_view_shows_rung.py
new file mode 100644
index 00000000..f5b3fdbb
--- /dev/null
+++ b/tests/integration/test_visibility_view_shows_rung.py
@@ -0,0 +1,207 @@
+from __future__ import annotations
+
+from datetime import timedelta
+from uuid import uuid4
+
+import pytest
+from fastapi.testclient import TestClient
+from sqlalchemy import Engine, text
+
+from adapters.http.app import create_app
+from adapters.http.authentication import VerifiedAuthenticationContext
+from adapters.http.ui_api import PostgreSQLUiApi
+from engine.control import ControlOperation
+from engine.persistence import (
+ DatabaseConfiguration,
+ PostgreSQLMembershipAuthority,
+ create_database_engine,
+)
+from tests.support.article_access_policy import (
+ delete_article_policy_scenario,
+ ingest_article,
+ insert_organization,
+ observe_source_acl,
+ set_source_default,
+ set_tenant_default,
+)
+from tests.support.file_imports import NOW
+from tests.support.ui import authenticate_ui, ui_control_authority
+
+pytestmark = pytest.mark.integration
+TOKEN = "ui-visibility-token"
+CONTROL_TOKEN = "ui-visibility-control-token"
+
+
+class _Authenticator:
+ def __init__(
+ self, *, organization_id: object, user_id: object, membership_id: object
+ ):
+ self.organization_id = organization_id
+ self.user_id = user_id
+ self.membership_id = membership_id
+
+ def authenticate(self, opaque_credential: str) -> VerifiedAuthenticationContext:
+ assert opaque_credential == TOKEN
+ return VerifiedAuthenticationContext(
+ organization_ref=str(self.organization_id),
+ user_ref=str(self.user_id),
+ principal_ref="principal:ui-visibility",
+ membership_ref=str(self.membership_id),
+ membership_version=1,
+ agent_version_ref="agent:ui-visibility",
+ authenticated_application_ref="application:ui-visibility",
+ authentication_binding_ref="binding:ui-visibility",
+ )
+
+
+def test_visibility_view_shows_rung(
+ migration_configuration: DatabaseConfiguration,
+ guarded_runtime_engine: Engine,
+ guarded_control_engine: Engine,
+) -> None:
+ migration_engine = create_database_engine(migration_configuration)
+ organization_id, user_id, membership_id = uuid4(), uuid4(), uuid4()
+ articles = {
+ "explicit_article": (
+ f"source:explicit:{uuid4()}",
+ f"resource:explicit:{uuid4()}",
+ ),
+ "source_default": (f"source:default:{uuid4()}", f"resource:default:{uuid4()}"),
+ "tenant_default": (f"source:tenant:{uuid4()}", f"resource:tenant:{uuid4()}"),
+ "isolation": (f"source:isolation:{uuid4()}", f"resource:isolation:{uuid4()}"),
+ }
+ try:
+ insert_organization(migration_engine, organization_id)
+ with migration_engine.begin() as connection:
+ connection.execute(
+ text("INSERT INTO user_account (user_id) VALUES (:user_id)"),
+ {"user_id": user_id},
+ )
+ connection.execute(
+ text(
+ """
+ INSERT INTO membership (
+ organization_id, membership_id, user_id, status,
+ membership_version, valid_from
+ ) VALUES (
+ :organization_id, :membership_id, :user_id,
+ 'active', 1, :valid_from
+ )
+ """
+ ),
+ {
+ "organization_id": organization_id,
+ "membership_id": membership_id,
+ "user_id": user_id,
+ "valid_from": NOW - timedelta(days=1),
+ },
+ )
+ explicit_source, explicit_resource = articles["explicit_article"]
+ connection.execute(
+ text(
+ """
+ INSERT INTO article_explicit_policy_setting (
+ organization_id, source_ref, resource_ref,
+ policy_kind, group_refs
+ ) VALUES (
+ :organization_id, :source_ref, :resource_ref,
+ 'private', ARRAY[]::text[]
+ )
+ """
+ ),
+ {
+ "organization_id": organization_id,
+ "source_ref": explicit_source,
+ "resource_ref": explicit_resource,
+ },
+ )
+ set_tenant_default(migration_engine, organization_id, "organization")
+ source_source, _ = articles["source_default"]
+ set_source_default(migration_engine, organization_id, source_source, "private")
+ for rung in ("explicit_article", "source_default", "tenant_default"):
+ source_ref, resource_ref = articles[rung]
+ observe_source_acl(
+ migration_engine,
+ organization_id=organization_id,
+ source_ref=source_ref,
+ resource_ref=resource_ref,
+ )
+ ingest_article(
+ migration_engine,
+ organization_id=organization_id,
+ source_ref=source_ref,
+ resource_ref=resource_ref,
+ )
+ set_tenant_default(migration_engine, organization_id, None)
+ isolation_source, isolation_resource = articles["isolation"]
+ observe_source_acl(
+ migration_engine,
+ organization_id=organization_id,
+ source_ref=isolation_source,
+ resource_ref=isolation_resource,
+ )
+ ingest_article(
+ migration_engine,
+ organization_id=organization_id,
+ source_ref=isolation_source,
+ resource_ref=isolation_resource,
+ )
+ control_authority, control_gate = ui_control_authority(
+ organization_id=organization_id,
+ credential=CONTROL_TOKEN,
+ operations=frozenset({ControlOperation.READ_ARTICLE_POLICY}),
+ clock=lambda: NOW,
+ )
+ api = PostgreSQLUiApi(
+ PostgreSQLMembershipAuthority(guarded_runtime_engine),
+ guarded_control_engine,
+ preview_key=b"v" * 32,
+ control_gate=control_gate,
+ clock=lambda: NOW,
+ )
+ client = TestClient(
+ create_app(
+ authenticator=_Authenticator(
+ organization_id=organization_id,
+ user_id=user_id,
+ membership_id=membership_id,
+ ),
+ ui_bearer_token=TOKEN,
+ ui_control_authority=control_authority,
+ ui_api=api,
+ )
+ )
+ authenticate_ui(client, TOKEN)
+
+ for expected_rung, (_, resource_ref) in articles.items():
+ response = client.post(
+ "/ui/articles/view",
+ content=(
+ f"resourceRef={resource_ref}&controlCredential={CONTROL_TOKEN}"
+ ),
+ headers={"Content-Type": "application/x-www-form-urlencoded"},
+ )
+ assert response.status_code == 200
+ assert CONTROL_TOKEN not in response.text
+ assert f'{expected_rung}' in response.text
+ assert resource_ref in response.text
+ assert ("Effective policy{version('context-engine')}" in response.text
+
+
+@pytest.mark.parametrize(
+ ("path", "content", "safe_values", "secret_values"),
+ (
+ (
+ "/ui/import/preview",
+ "sourceRef=11111111-1111-4111-8111-111111111111"
+ "&path=docs%2Fguide.md&controlCredential=import-control-secret",
+ ("Source", "11111111-1111-4111-8111-111111111111", "Path", "docs/guide.md"),
+ ("import-control-secret",),
+ ),
+ (
+ "/ui/articles/preview",
+ "resourceRef=article%3Ahandbook&policyKind=groups"
+ "&groupRefs=group%3Aeng%2Cgroup%3Aops"
+ "&controlCredential=article-control-secret",
+ (
+ "Article",
+ "article:handbook",
+ "Policy",
+ "groups",
+ "Groups",
+ "group:eng, group:ops",
+ ),
+ ("article-control-secret",),
+ ),
+ (
+ "/ui/profiles",
+ "profileRef=embedding%3Aproposed&digest=" + "a" * 64,
+ ("Profile", "embedding:proposed", "Digest", "a" * 64),
+ (),
+ ),
+ (
+ "/ui/feedback",
+ "runRef=run%3Aauthorized&rating=not_helpful¬e=private+operator+note",
+ ("ContextRun", "run:authorized", "Rating", "not helpful"),
+ ("private operator note",),
+ ),
+ ),
+)
+def test_safe_non_secret_form_values_survive_refusal(
+ path: str,
+ content: str,
+ safe_values: tuple[str, ...],
+ secret_values: tuple[str, ...],
+) -> None:
+ credential = "configured-session"
+ client = TestClient(create_app(ui_bearer_token=credential))
+ authenticate_ui(client, credential)
+
+ response = client.post(
+ path,
+ content=content,
+ headers={"Content-Type": "application/x-www-form-urlencoded"},
+ )
+
+ assert response.status_code in {401, 403}
+ assert "Submitted non-secret input" in response.text
+ for value in safe_values:
+ assert value in response.text
+ for value in (*secret_values, credential):
+ assert value not in response.text
diff --git a/tests/unit/test_feedback_has_no_publication_authority.py b/tests/unit/test_feedback_has_no_publication_authority.py
new file mode 100644
index 00000000..bd2d6861
--- /dev/null
+++ b/tests/unit/test_feedback_has_no_publication_authority.py
@@ -0,0 +1,70 @@
+from __future__ import annotations
+
+import ast
+from pathlib import Path
+
+from fastapi.testclient import TestClient
+
+from adapters.http.app import create_app
+from adapters.http.ui_api import FeedbackCapture, RefusingUiApi, UiActor
+from tests.integration.test_runtime_authorized_evidence_integration import (
+ SeededAuthenticator,
+ _new_fixture,
+)
+from tests.support.ui import authenticate_ui
+
+REPOSITORY_ROOT = Path(__file__).resolve().parents[2]
+
+
+class _FeedbackApi(RefusingUiApi):
+ def __init__(self) -> None:
+ self.captured: list[FeedbackCapture] = []
+
+ def capture_feedback(
+ self,
+ actor: UiActor,
+ feedback: FeedbackCapture,
+ ) -> dict[str, object]:
+ del actor
+ self.captured.append(feedback)
+ return {"feedbackRef": "feedback_" + "5" * 64, "state": "recorded"}
+
+
+def test_feedback_has_no_publication_authority() -> None:
+ fixture = _new_fixture().org_a
+ api = _FeedbackApi()
+ client = TestClient(
+ create_app(
+ authenticator=SeededAuthenticator(fixture, token="feedback-token"),
+ ui_bearer_token="feedback-token",
+ ui_api=api,
+ )
+ )
+ authenticate_ui(client, "feedback-token")
+ response = client.post(
+ "/ui/feedback",
+ content="runRef=run_authorized&rating=helpful¬e=Clear+lineage",
+ headers={"Content-Type": "application/x-www-form-urlencoded"},
+ )
+
+ assert response.status_code == 200
+ assert "Feedback recorded" in response.text
+ assert api.captured == [
+ FeedbackCapture(
+ run_ref="run_authorized",
+ rating="helpful",
+ note="Clear lineage",
+ )
+ ]
+
+ forbidden = {"activate", "promote", "publish", "rollback"}
+ for root in (REPOSITORY_ROOT / "ui", REPOSITORY_ROOT / "adapters" / "http"):
+ for path in root.rglob("*.py"):
+ tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
+ feedback_owners = {
+ node.name.casefold()
+ for node in ast.walk(tree)
+ if isinstance(node, ast.FunctionDef | ast.AsyncFunctionDef)
+ and "feedback" in node.name.casefold()
+ }
+ assert not feedback_owners.intersection(forbidden), path
diff --git a/tests/unit/test_http_scope_authority.py b/tests/unit/test_http_scope_authority.py
index c9bb50c8..a7264577 100644
--- a/tests/unit/test_http_scope_authority.py
+++ b/tests/unit/test_http_scope_authority.py
@@ -249,18 +249,19 @@ def project(self, locator: object) -> None:
organization_id=ORGANIZATION_ID,
principal_ref="principal-from-auth",
agent_version_ref="agent-version-from-server",
- purpose="context.answer",
+ purposes=frozenset({"context.answer", "citation.open"}),
)
try:
- with authority.current_scope(bound) as snapshot:
- operands = _trusted_operands_from_snapshot(snapshot)
- assert operands.organization_boundary == ScopeSet(frozenset(targets))
- assert operands.membership_rights == ScopeSet(frozenset(targets[:4]))
- assert operands.principal_grants == ScopeSet(frozenset(targets[:3]))
- assert operands.source_native_acl == ScopeSet(frozenset(targets[:2]))
- assert operands.resource_acl == ScopeSet(frozenset(targets[:1]))
- assert operands.agent_ceiling == ScopeSet(frozenset(targets))
- assert operands.purpose_policy == ScopeSet(frozenset(targets))
+ for purpose in ("context.answer", "citation.open"):
+ with authority.current_scope(replace(bound, purpose=purpose)) as snapshot:
+ operands = _trusted_operands_from_snapshot(snapshot)
+ assert operands.organization_boundary == ScopeSet(frozenset(targets))
+ assert operands.membership_rights == ScopeSet(frozenset(targets[:4]))
+ assert operands.principal_grants == ScopeSet(frozenset(targets[:3]))
+ assert operands.source_native_acl == ScopeSet(frozenset(targets[:2]))
+ assert operands.resource_acl == ScopeSet(frozenset(targets[:1]))
+ assert operands.agent_ceiling == ScopeSet(frozenset(targets))
+ assert operands.purpose_policy == ScopeSet(frozenset(targets))
for mismatched in (
replace(bound, agent_version_ref="agent-version-not-authorized"),
replace(bound, purpose="context.not-authorized"),
diff --git a/tests/unit/test_local_operator_authentication.py b/tests/unit/test_local_operator_authentication.py
index dab3ea6b..66716bde 100644
--- a/tests/unit/test_local_operator_authentication.py
+++ b/tests/unit/test_local_operator_authentication.py
@@ -263,14 +263,17 @@ def test_authority_grants_one_allowed_operation_per_context_lifetime() -> None:
)
-def test_http_composition_cannot_reach_local_operator_authentication() -> None:
+def test_general_http_composition_cannot_reach_local_operator_authentication() -> None:
prohibited_module_names = {
"applications.control",
"applications.operator_authentication",
}
pending = [
- ROOT / "applications" / "api.py",
- *sorted((ROOT / "adapters" / "http").rglob("*.py")),
+ *sorted(
+ path
+ for path in (ROOT / "adapters" / "http").rglob("*.py")
+ if path.name != "dogfood.py"
+ ),
]
visited: set[Path] = set()
while pending:
@@ -299,3 +302,29 @@ def test_http_composition_cannot_reach_local_operator_authentication() -> None:
):
if candidate.is_file() and candidate not in visited:
pending.append(candidate)
+
+
+def test_dogfood_http_is_the_only_local_operator_authentication_composition() -> None:
+ api_path = ROOT / "applications" / "api.py"
+ api_tree = ast.parse(
+ api_path.read_text(encoding="utf-8"),
+ filename=api_path,
+ )
+ api_imports = {
+ node.module
+ for node in ast.walk(api_tree)
+ if isinstance(node, ast.ImportFrom) and node.module is not None
+ }
+ path = ROOT / "adapters" / "http" / "dogfood.py"
+ tree = ast.parse(path.read_text(encoding="utf-8"), filename=path)
+ imported = {
+ node.module
+ for node in ast.walk(tree)
+ if isinstance(node, ast.ImportFrom) and node.module is not None
+ }
+
+ assert "adapters.http.dogfood" in api_imports
+ assert "applications.operator_authentication" not in api_imports
+ assert "applications.control" not in api_imports
+ assert "applications.operator_authentication" in imported
+ assert "applications.control" not in imported
diff --git a/tests/unit/test_m0_rls_inventory.py b/tests/unit/test_m0_rls_inventory.py
index 034765a2..e1609775 100644
--- a/tests/unit/test_m0_rls_inventory.py
+++ b/tests/unit/test_m0_rls_inventory.py
@@ -31,6 +31,7 @@
"citation_open_locator",
"context_fragment",
"context_fragment_field",
+ "context_feedback",
"context_resource",
"context_revision",
"context_run",
@@ -164,7 +165,7 @@ def test_manifest_declares_exact_live_table_denominator_and_rls_evidence() -> No
assert global_tables == GLOBAL_TABLES
assert tenant_tables == TENANT_TABLES
- assert len(tables) == 70
+ assert len(tables) == 71
for name in sorted(GLOBAL_TABLES):
rationale = tables[name]["classificationRationale"]
@@ -187,8 +188,8 @@ def test_rls_auditor_requires_every_live_control_and_non_owner_evidence() -> Non
assert report["passed"] is True
assert report["coverage"] == {
- "numerator": 67,
- "denominator": 67,
+ "numerator": 68,
+ "denominator": 68,
"percent": 100.0,
}
inventory = cast(dict[str, object], report["inventory"])
@@ -213,7 +214,7 @@ def test_rls_auditor_does_not_count_force_rls_or_evidence_gaps() -> None:
assert report["passed"] is False
assert report["coverage"] == {
"numerator": 0,
- "denominator": 67,
+ "denominator": 68,
"percent": 0.0,
}
tenant_reports = cast(list[dict[str, Any]], report["tenantTables"])
diff --git a/tests/unit/test_profile_change_surfaces_reembed.py b/tests/unit/test_profile_change_surfaces_reembed.py
new file mode 100644
index 00000000..5a6b7e67
--- /dev/null
+++ b/tests/unit/test_profile_change_surfaces_reembed.py
@@ -0,0 +1,75 @@
+from __future__ import annotations
+
+from fastapi.testclient import TestClient
+
+from adapters.http.app import create_app
+from adapters.http.ui_api import RefusingUiApi, UiActor
+from tests.integration.test_runtime_authorized_evidence_integration import (
+ SeededAuthenticator,
+ _new_fixture,
+)
+from tests.support.ui import authenticate_ui
+
+
+class _ProfileApi(RefusingUiApi):
+ def profiles(self, actor: UiActor) -> dict[str, object]:
+ del actor
+ return {
+ "releaseGeneration": 7,
+ "releaseManifestRef": "release:current",
+ "contentProfile": {"profileRef": "content-v3", "digest": "1" * 64},
+ "indexProfile": {"profileRef": "embedding-v2", "digest": "2" * 64},
+ "runtimeProfile": {"profileRef": "runtime-v4", "digest": "3" * 64},
+ }
+
+
+def test_profile_change_surfaces_reembed() -> None:
+ fixture = _new_fixture().org_a
+ client = TestClient(
+ create_app(
+ authenticator=SeededAuthenticator(fixture, token="profile-token"),
+ ui_bearer_token="profile-token",
+ ui_api=_ProfileApi(),
+ )
+ )
+ authenticate_ui(client, "profile-token")
+
+ response = client.post(
+ "/ui/profiles",
+ content=(
+ "profileRef=embedding-v3&"
+ f"digest={'4' * 64}"
+ ),
+ headers={"Content-Type": "application/x-www-form-urlencoded"},
+ )
+
+ assert response.status_code == 200
+ assert "Re-embed required" in response.text
+ assert "No profile change was applied" in response.text
+ assert "embedding-v2" in response.text
+ assert "embedding-v3" in response.text
+ assert "Apply" not in response.text
+ assert "Confirm" not in response.text
+
+
+def test_current_embedding_profile_does_not_claim_reembed() -> None:
+ fixture = _new_fixture().org_a
+ client = TestClient(
+ create_app(
+ authenticator=SeededAuthenticator(fixture, token="profile-token"),
+ ui_bearer_token="profile-token",
+ ui_api=_ProfileApi(),
+ )
+ )
+ authenticate_ui(client, "profile-token")
+
+ response = client.post(
+ "/ui/profiles",
+ content=f"profileRef=embedding-v2&digest={'2' * 64}",
+ headers={"Content-Type": "application/x-www-form-urlencoded"},
+ )
+
+ assert response.status_code == 200
+ assert "Profile already active" in response.text
+ assert "No re-embed or Release change is required" in response.text
+ assert "Re-embed required" not in response.text
diff --git a/tests/unit/test_schema_security_manifest.py b/tests/unit/test_schema_security_manifest.py
index cef5e215..9c386203 100644
--- a/tests/unit/test_schema_security_manifest.py
+++ b/tests/unit/test_schema_security_manifest.py
@@ -68,6 +68,7 @@ def test_manifest_classifies_the_exact_current_release_schema() -> None:
"bulk_article_policy_change_audit",
"context_fragment",
"context_fragment_field",
+ "context_feedback",
"context_resource",
"context_revision",
"context_run",
@@ -1557,6 +1558,8 @@ def test_membership_manifest_requires_exact_user_actor_and_read_only_runtime() -
"context_engine_action_prepare_definer": ["SELECT"],
"context_engine_action_execute_definer": ["SELECT"],
"context_engine_citation_definer": ["SELECT"],
+ "context_engine_access_policy_definer": ["SELECT"],
+ "context_engine_context_run_reader_definer": ["SELECT"],
"context_engine_file_dispatch_definer": [
"SELECT",
"UPDATE status, valid_from, valid_until",
diff --git a/tests/unit/test_source_overview_matches_promoted_release.py b/tests/unit/test_source_overview_matches_promoted_release.py
new file mode 100644
index 00000000..87dfce78
--- /dev/null
+++ b/tests/unit/test_source_overview_matches_promoted_release.py
@@ -0,0 +1,72 @@
+from __future__ import annotations
+
+from fastapi.testclient import TestClient
+
+from adapters.http.app import create_app
+from adapters.http.ui_api import RefusingUiApi, UiActor
+from engine.control import ControlOperation, TrustedControlCall
+from tests.integration.test_runtime_authorized_evidence_integration import (
+ RECEIVED_AT,
+ SeededAuthenticator,
+ _new_fixture,
+)
+from tests.support.ui import authenticate_ui, ui_control_authority
+
+CONTROL_TOKEN = "overview-control-token"
+
+
+class _OverviewApi(RefusingUiApi):
+ def __init__(self, promoted_generation: int) -> None:
+ self.promoted_generation = promoted_generation
+
+ def overview(
+ self, actor: UiActor, control_call: TrustedControlCall
+ ) -> dict[str, object]:
+ del actor, control_call
+ return {
+ "releaseGeneration": self.promoted_generation,
+ "releaseManifestRef": "release:promoted",
+ "sources": [
+ {
+ "activeResourceCount": 3,
+ "displayName": "Handbook",
+ "lastSuccessfulAcquisitionAgeSeconds": 17,
+ "refusalCategories": ["invalid_utf8"],
+ "sourceRef": "source:handbook",
+ "status": "refused",
+ }
+ ],
+ }
+
+
+def test_source_overview_matches_promoted_release_and_surfaces_refusal() -> None:
+ promoted_generation = 11
+ fixture = _new_fixture().org_a
+ control_authority, _ = ui_control_authority(
+ organization_id=fixture.organization_id,
+ credential=CONTROL_TOKEN,
+ operations=frozenset({ControlOperation.READ_SOURCE_PROGRESS}),
+ clock=lambda: RECEIVED_AT,
+ )
+ client = TestClient(
+ create_app(
+ authenticator=SeededAuthenticator(fixture, token="overview-token"),
+ ui_bearer_token="overview-token",
+ ui_control_authority=control_authority,
+ ui_api=_OverviewApi(promoted_generation),
+ )
+ )
+ authenticate_ui(client, "overview-token")
+ response = client.post(
+ "/ui/overview",
+ content=f"controlCredential={CONTROL_TOKEN}",
+ headers={"Content-Type": "application/x-www-form-urlencoded"},
+ )
+
+ assert response.status_code == 200
+ assert f"Release generation {promoted_generation}" in response.text
+ assert f"Article is the authorization atom
+Inspect the effective policy and its exact resolution rung. Historical policy + changes require a separate preview and confirmation that advances Policy Epoch.
+{{ article.resource_ref }}{{ article.source_ref }}Preview · no historical change yet
+Proposed policy: {{ change.policy_kind }} + {% if change.group_refs %}({{ change.group_refs|join(', ') }}){% endif %}
+Confirmation rechecks the exact policy version and Policy Epoch atomically.
+ +ContextPackage · Citation closure
+Every clean answer Block includes a resolvable Article → Revision → Fragment + citation. Incomplete lineage refuses the answer.
+{{ hit.evidence.resource_ref }}{{ hit.evidence.revision_ref }}{{ hit.evidence.fragment_ref }}{{ hit.evidence.evidence_ref }}{{ hit.evidence.citation_open_ref }}{{ result.coverage_reason or 'no_authorized_evidence' }}
+Evidence only · no publication authority
+Record a minimal observation against one authorized ContextRun. Feedback cannot + activate, promote, publish, or roll back a Release.
+Evidence receipt {{ receipt }}
Authorized diagnostics
+Only exact-authorized Fragments render. Refused candidates and their rank are + not part of this view.
+{{ hit.evidence.evidence_ref }}{{ hit.evidence.resource_ref }}{{ hit.evidence.revision_ref }}{{ hit.evidence.fragment_ref }}{{ hit.evidence.source_ref }}{{ result.coverage_reason or 'no_authorized_evidence' }}
+Preview → explicit confirm
+The preview is compiled from the exact configured-root bytes. Nothing is + scheduled until confirmation, and the worker is bound to that byte identity.
+Exact job {{ receipt }}
Release promotion remains a separate operator path.
+Not published · not queued
+{{ preview.source_ref }}{{ preview.path }}{{ preview.compilation_digest }}{{ preview.fragment_digest }}{{ fragment.fragment_ref }}Request-scoped authority
+Enter the local dogfood credential to open a short-lived operator session.
+{% if current %}Release generation {{ current.release_generation }}{% else %}Control-authorized operational evidence{% endif %}
+Current Source health, ingest counts, and the exact promoted Release lineage.
+{{ current.release_manifest_ref }}{{ source.source_ref }}This is a first-run state, not a successful health result.
+Release generation {{ current.release_generation }}
+Exact profile identities make the active Release reproducible. This M1 view can + preview an embedding change; it cannot apply or publish one.
+{{ current.release_manifest_ref }}{{ current.content_profile.profile_ref }}{{ current.content_profile.digest }}{{ current.index_profile.profile_ref }}{{ current.index_profile.digest }}{{ current.runtime_profile.profile_ref }}{{ current.runtime_profile.digest }}Consequence preview · not applied
+Changing {{ current.index_profile.profile_ref }} to
+ {{ proposed.profile_ref }} invalidates compatibility with the current
+ corpus. The full active corpus must be re-embedded and evaluated in a versioned
+ Release candidate.
No profile change was applied. Release promotion remains outside + this UI.
+ {% else %} +No-op preview
+The proposed profile ref and digest exactly match the active index profile. + No re-embed or Release change is required.
+ {% endif %} +Fail closed
+The operation did not establish the trusted context it requires.
+ {% if submitted %} +{{ value }}{{ category }}