diff --git a/README.md b/README.md index ac37de43..46409db9 100644 --- a/README.md +++ b/README.md @@ -57,6 +57,7 @@ the running service's own `/health` response. | OpenAPI v0 wire contract + generated TypeScript SDK + breaking-change gate | Active | | Private File-backed bot delivery flow (deterministic twin) | Active | | Autonomous File import dispatch + bounded expired-lease reclaim | Active | +| Loopback single-Membership File dogfood `Acquire` | Active when explicitly configured | | Production authentication (OAuth/JWT) | `NOT_ACTIVE` | | Real source ACLs, general content retrieval, `Continue` / `OpenCitation` | `NOT_ACTIVE` | | Live Feishu / Slack / Google Docs connectors, group chat | `NOT_ACTIVE` | @@ -122,6 +123,50 @@ curl http://127.0.0.1:8137/health rejects every credential and performs zero content I/O. The public wire contract is `POST /v0/resolve`, frozen in [`openapi/v0/openapi.json`](./openapi/v0/openapi.json). +### Run the bounded dogfood API + +The only served content-bearing composition is a local, loopback-only dogfood +carrier. It is an explicit opt-in and is not production authentication. First +seed one Organization, User, and current Membership with the configured +migrator connection: + +```bash +uv run context-engine-dogfood-seed \ + --organization-id "$CONTEXT_ENGINE_DOGFOOD_ORGANIZATION_ID" \ + --user-id "$CONTEXT_ENGINE_DOGFOOD_USER_ID" \ + --membership-id "$CONTEXT_ENGINE_DOGFOOD_MEMBERSHIP_ID" +``` + +Configure the API with the Runtime database source and these dogfood settings: + +```text +CONTEXT_ENGINE_API_COMPOSITION=dogfood-local-v1 +CONTEXT_ENGINE_DOGFOOD_SECRET +CONTEXT_ENGINE_DOGFOOD_ORGANIZATION_ID +CONTEXT_ENGINE_DOGFOOD_USER_ID +CONTEXT_ENGINE_DOGFOOD_MEMBERSHIP_ID +CONTEXT_ENGINE_DOGFOOD_MEMBERSHIP_VERSION +CONTEXT_ENGINE_DOGFOOD_PRINCIPAL_REF +CONTEXT_ENGINE_DOGFOOD_AGENT_VERSION_REF +CONTEXT_ENGINE_DOGFOOD_APPLICATION_REF +CONTEXT_ENGINE_DOGFOOD_AUTHENTICATION_BINDING_REF +CONTEXT_ENGINE_DOGFOOD_EMBEDDING_PROVIDER=deterministic-twin-v1 +``` + +Before activation, freshly reimport the File corpus with the Supply worker's +network-free `twin` embedding mode and promote those exact Revision references +through the existing Learning release operation using the dogfood vector index +profile. The active Release binds the deterministic model and contextual- +fragment input profile; a mismatch fails composition. Then run the API with an +explicit loopback host. A valid composition reports `runtime_delivery: ACTIVE`; +missing or partial configuration fails closed. The dogfood secret must come +from one local secret source and must never be committed or printed. + +External query embeddings, production or multi-user authentication, remote +network exposure, group/public delivery, dogfood `OpenCitation`, `Continue`, hybrid retrieval, and +non-File providers remain `NOT_ACTIVE`; see +[ADR-0068](./docs/decisions/0068-activate-loopback-dogfood-runtime.md). + ### Run the worker The Supply worker is a separate process from the API, with one entry point and diff --git a/STATUS.md b/STATUS.md index a7b49338..397f6b2a 100644 --- a/STATUS.md +++ b/STATUS.md @@ -38,15 +38,17 @@ capability can never be reported as a passing one. ## Currently `NOT_ACTIVE` The default application **rejects every credential and performs zero content -I/O**. The following are known, designed, and deliberately not active: +I/O**. ADR-0068 separately activates one explicit loopback dogfood composition; +it does not widen the default. The following are known, designed, and +deliberately not active: | Capability | Note | |---|---| | Production authentication (OAuth / JWT) | Module-level default application is reject-all across all three production authorities (authentication, Organization, Membership) | -| Durable Principal / Agent grants | Scope authority returns seven missing trusted operands by default, so no deliverable scope can be produced | -| Real Source / Resource ACLs | Only synthetic conformance fixtures exist | -| General content retrieval | No production candidate path | -| `Continue` / `OpenCitation` carriers | The M0 *refusal* path is active; real issuance and redemption are not | +| Durable general Principal / Agent grants | The default scope authority returns seven missing operands; dogfood separately carries the bounded current File operands and binds one configured Agent/purpose to the Release ceiling only | +| General / multi-user Source and Resource ACLs | Dogfood uses current mirrored File access plus Membership field rights only; source-native and multi-user authorities remain absent | +| General content retrieval | Only the loopback File pgvector dogfood `Acquire` carrier is active | +| `Continue` / dogfood `OpenCitation` carriers | The bounded dogfood composition keeps both unavailable; other accepted ADRs own their narrower citation carriers | | Federated discovery, source-native authorization | Deterministic refusal only | | Live Feishu / Slack / Google Docs connectors | See [PLAN.md](./PLAN.md) milestones M4 / M6 / M7 | | Group-chat delivery, compensating deletes | M5 | @@ -56,6 +58,19 @@ I/O**. The following are known, designed, and deliberately not active: | Streaming delivery | Explicit V1 non-goal — placeholder + edit instead | | Answer generation inside the engine | Permanent non-goal — generation always lives above the engine boundary | +### Bounded dogfood Runtime + +| ADR | Activates | +|---|---| +| [0068](./docs/decisions/0068-activate-loopback-dogfood-runtime.md) | Explicit loopback single-Membership authentication plus File pgvector `Acquire`, with exact EffectiveScope removal before ANN `LIMIT`, sealed Kernel reauthorization, deterministic twin query embedding, and final Policy Epoch veto | + +`RUNTIME-DOGFOOD-AUTH-102`, `RUNTIME-DOGFOOD-CARRIER-102`, and +`RUNTIME-DOGFOOD-EPOCH-102` are registered release-veto evidence. The default +application remains reject-all and reports `NOT_ACTIVE`. Production +authentication, a second human, network exposure beyond the maintainer machine, +group/public audience, dogfood `OpenCitation`, `Continue`, hybrid retrieval, non-File providers, and +external query embeddings remain `NOT_ACTIVE`. + ## Activation ledger Each accepted ADR below activated a bounded, separately proven capability. diff --git a/adapters/exact_phrase.py b/adapters/exact_phrase.py index 6dca4088..65b024b9 100644 --- a/adapters/exact_phrase.py +++ b/adapters/exact_phrase.py @@ -9,6 +9,7 @@ MaterializedProjectionSession, _discover_materialized_exact_phrase, ) +from engine.runtime.scope import EffectiveScope class PostgreSQLExactPhraseCandidateIndex: @@ -18,9 +19,13 @@ def discover( self, request: Acquire, projection_session: MaterializedProjectionSession, + *, + effective_scope: EffectiveScope, ) -> tuple[CandidateRef, ...]: if type(request) is not Acquire: raise TypeError("exact phrase discovery requires Acquire") + if type(effective_scope) is not EffectiveScope: + raise TypeError("exact phrase discovery requires EffectiveScope") return _discover_materialized_exact_phrase( projection_session, exact_phrase_digest(request.need.query), diff --git a/adapters/http/app.py b/adapters/http/app.py index f52b8d70..3f292977 100644 --- a/adapters/http/app.py +++ b/adapters/http/app.py @@ -5,7 +5,7 @@ from datetime import UTC, datetime from hashlib import sha256 from json import loads -from typing import Annotated, Final +from typing import Annotated, Final, NoReturn from uuid import UUID, uuid4 from fastapi import Body, Depends, FastAPI, Header, Request, Response, Security @@ -175,6 +175,34 @@ class DuplicateJsonObjectKey(ValueError): """Strict JSON decoding found an ambiguous object member.""" +class _RuntimeDeliveryActivation: + """Nominal receipt issued only by a validated served composition.""" + + __slots__ = ("_seal",) + + def __init__(self) -> None: + raise TypeError("Runtime delivery activation is not constructible") + + def __reduce__(self) -> NoReturn: + raise TypeError("Runtime delivery activation is not serializable") + + +_RUNTIME_DELIVERY_ACTIVATION_SEAL = object() + + +def _construct_runtime_delivery_activation() -> _RuntimeDeliveryActivation: + activation = object.__new__(_RuntimeDeliveryActivation) + object.__setattr__(activation, "_seal", _RUNTIME_DELIVERY_ACTIVATION_SEAL) + return activation + + +def _is_runtime_delivery_active(value: object) -> bool: + return ( + type(value) is _RuntimeDeliveryActivation + and getattr(value, "_seal", None) is _RUNTIME_DELIVERY_ACTIVATION_SEAL + ) + + def _utc_now() -> datetime: return datetime.now(UTC) @@ -219,6 +247,7 @@ def create_app( clock: Callable[[], datetime] = _utc_now, request_id_factory: Callable[[], str] = _new_request_id, transport_profile: HttpTransportProfile = HTTP_TRANSPORT_PROFILE_V1, + runtime_delivery_activation: _RuntimeDeliveryActivation | None = None, ) -> FastAPI: """Construct API; the module-level composition remains reject-all.""" @@ -234,6 +263,10 @@ def create_app( ) if type(selected_runtime) is not Runtime: raise TypeError("runtime must be the sealed Runtime composition") + if runtime_delivery_activation is not None and not _is_runtime_delivery_active( + runtime_delivery_activation + ): + raise TypeError("Runtime delivery activation has the wrong nominal type") selected_authenticator = authenticator or RejectingAuthenticator() selected_organization_authority = ( organization_authority or RejectingOrganizationAuthority() @@ -393,7 +426,10 @@ def require_public_request_id( @app.get("/health", include_in_schema=False) def health() -> dict[str, str]: - return HEALTH_RESPONSE.copy() + response = HEALTH_RESPONSE.copy() + if _is_runtime_delivery_active(runtime_delivery_activation): + response["runtime_delivery"] = "ACTIVE" + return response @app.post( LEGACY_RESOLVE_PATH, @@ -545,6 +581,12 @@ def resolve_context( current_membership_verification.authentication_binding_ref ), checked_at=current_membership_verification.checked_at, + materialized_projection_session=( + current_membership_verification.materialized_projection_session + ), + active_runtime_release=( + current_membership_verification.active_runtime_release + ), ) except (TypeError, ValueError): raise TransportAuthenticationFailed from None diff --git a/adapters/http/authentication.py b/adapters/http/authentication.py index 2f10c0d2..9b3a59d6 100644 --- a/adapters/http/authentication.py +++ b/adapters/http/authentication.py @@ -1,5 +1,6 @@ """Trusted HTTP authentication adapter contracts and fail-closed default.""" +import hmac from dataclasses import dataclass, field from typing import Protocol from uuid import UUID @@ -115,3 +116,41 @@ def authenticate( opaque_credential: str, ) -> VerifiedAuthenticationContext: raise AuthenticationRejected + + +class DogfoodAuthenticator: + """Constant-time local-secret verifier for one fixed seeded identity.""" + + __slots__ = ("_authentication", "_secret") + + def __init__( + self, + *, + secret: str, + authentication: VerifiedAuthenticationContext, + ) -> None: + if ( + type(secret) is not str + or len(secret.encode("utf-8")) < 32 + or secret != secret.strip() + or any(character.isspace() for character in secret) + ): + raise ValueError("dogfood authentication configuration is unavailable") + if type(authentication) is not VerifiedAuthenticationContext: + raise TypeError("dogfood authentication identity is unavailable") + self._secret = secret.encode("utf-8") + self._authentication = authentication + + def authenticate(self, opaque_credential: str) -> VerifiedAuthenticationContext: + if type(opaque_credential) is not str: + raise AuthenticationRejected + try: + supplied = opaque_credential.encode("utf-8") + except UnicodeEncodeError: + raise AuthenticationRejected from None + if not hmac.compare_digest(supplied, self._secret): + raise AuthenticationRejected + return self._authentication + + def __repr__(self) -> str: + return "DogfoodAuthenticator()" diff --git a/adapters/http/dogfood.py b/adapters/http/dogfood.py new file mode 100644 index 00000000..351127a8 --- /dev/null +++ b/adapters/http/dogfood.py @@ -0,0 +1,307 @@ +"""Explicit local-only dogfood composition; absent configuration is reject-all.""" + +from __future__ import annotations + +import os +from collections.abc import Mapping +from dataclasses import dataclass, field +from datetime import UTC, datetime +from hashlib import sha256 +from uuid import UUID + +from fastapi import FastAPI + +from adapters.embeddings import DeterministicEmbeddingTwin +from adapters.http.authentication import ( + DogfoodAuthenticator, + VerifiedAuthenticationContext, +) +from adapters.http.organization_authority import DogfoodOrganizationAuthority +from adapters.http.scope_authority import DogfoodFileScopeAuthority +from adapters.pgvector import PostgreSQLVectorCandidateIndex +from engine.persistence import ( + DatabasePurpose, + PostgreSQLMembershipAuthority, + create_database_engine, + load_database_configuration, +) +from engine.persistence.membership_context import ( + MembershipAuthorityUnavailable, + MembershipIdentity, + MembershipNotCurrent, +) +from engine.runtime import Runtime +from engine.runtime.construction import required_kernel_dependencies +from engine.runtime.package_digest import QueryDigestKeyring +from engine.runtime.release_lineage import ( + DOGFOOD_VECTOR_INDEX_PROFILE_DIGEST_V1, + DOGFOOD_VECTOR_INDEX_PROFILE_REF_V1, + ActiveReleaseUnavailable, +) + +DOGFOOD_COMPOSITION_ENV = "CONTEXT_ENGINE_API_COMPOSITION" +DOGFOOD_COMPOSITION_VALUE = "dogfood-local-v1" +DOGFOOD_SECRET_ENV = "CONTEXT_ENGINE_DOGFOOD_SECRET" +DOGFOOD_ORGANIZATION_ENV = "CONTEXT_ENGINE_DOGFOOD_ORGANIZATION_ID" +DOGFOOD_USER_ENV = "CONTEXT_ENGINE_DOGFOOD_USER_ID" +DOGFOOD_MEMBERSHIP_ENV = "CONTEXT_ENGINE_DOGFOOD_MEMBERSHIP_ID" +DOGFOOD_MEMBERSHIP_VERSION_ENV = "CONTEXT_ENGINE_DOGFOOD_MEMBERSHIP_VERSION" +DOGFOOD_PRINCIPAL_ENV = "CONTEXT_ENGINE_DOGFOOD_PRINCIPAL_REF" +DOGFOOD_AGENT_ENV = "CONTEXT_ENGINE_DOGFOOD_AGENT_VERSION_REF" +DOGFOOD_APPLICATION_ENV = "CONTEXT_ENGINE_DOGFOOD_APPLICATION_REF" +DOGFOOD_BINDING_ENV = "CONTEXT_ENGINE_DOGFOOD_AUTHENTICATION_BINDING_REF" +DOGFOOD_EMBEDDING_PROVIDER_ENV = "CONTEXT_ENGINE_DOGFOOD_EMBEDDING_PROVIDER" +DOGFOOD_EMBEDDING_PROVIDER_VALUE = "deterministic-twin-v1" + +_QUERY_DIGEST_DERIVATION_DOMAIN = b"context-engine.dogfood.query-digest.v1\x00" + + +class DogfoodConfigurationUnavailable(ValueError): + """Local composition is incomplete, ambiguous, or attempts a wider carrier.""" + + +def _required(environment: Mapping[str, str], name: str) -> str: + value = environment.get(name) + if ( + value is None + or not value + or value != value.strip() + or any(character.isspace() for character in value) + ): + raise DogfoodConfigurationUnavailable( + "dogfood API configuration is unavailable" + ) + return value + + +@dataclass(frozen=True, slots=True) +class DogfoodConfiguration: + """One immutable local identity and network-free retrieval composition.""" + + secret: str = field(repr=False) + organization_id: UUID + user_id: UUID + membership_id: UUID + membership_version: int + principal_ref: str + agent_version_ref: str + application_ref: str + authentication_binding_ref: str + embedding_provider: str + + def __post_init__(self) -> None: + if ( + type(self.secret) is not str + or len(self.secret.encode("utf-8")) < 32 + or self.secret != self.secret.strip() + or any(character.isspace() for character in self.secret) + ): + raise DogfoodConfigurationUnavailable( + "dogfood API configuration is unavailable" + ) + for field_name in ("organization_id", "user_id", "membership_id"): + if type(getattr(self, field_name)) is not UUID: + raise DogfoodConfigurationUnavailable( + "dogfood API configuration is unavailable" + ) + if ( + type(self.membership_version) is not int + or not 1 <= self.membership_version < (1 << 63) + ): + raise DogfoodConfigurationUnavailable( + "dogfood API configuration is unavailable" + ) + for field_name in ( + "principal_ref", + "agent_version_ref", + "application_ref", + "authentication_binding_ref", + ): + value = getattr(self, field_name) + if type(value) is not str or not value or value.isspace(): + raise DogfoodConfigurationUnavailable( + "dogfood API configuration is unavailable" + ) + if self.embedding_provider != DOGFOOD_EMBEDDING_PROVIDER_VALUE: + raise DogfoodConfigurationUnavailable( + "external dogfood query embedding is NOT_ACTIVE" + ) + + @classmethod + def load(cls, environment: Mapping[str, str]) -> DogfoodConfiguration: + try: + membership_version_text = _required( + environment, + DOGFOOD_MEMBERSHIP_VERSION_ENV, + ) + if not membership_version_text.isdecimal(): + raise ValueError + return cls( + secret=_required(environment, DOGFOOD_SECRET_ENV), + organization_id=UUID( + _required(environment, DOGFOOD_ORGANIZATION_ENV) + ), + user_id=UUID(_required(environment, DOGFOOD_USER_ENV)), + membership_id=UUID( + _required(environment, DOGFOOD_MEMBERSHIP_ENV) + ), + membership_version=int(membership_version_text), + principal_ref=_required(environment, DOGFOOD_PRINCIPAL_ENV), + agent_version_ref=_required(environment, DOGFOOD_AGENT_ENV), + application_ref=_required(environment, DOGFOOD_APPLICATION_ENV), + authentication_binding_ref=_required( + environment, + DOGFOOD_BINDING_ENV, + ), + embedding_provider=_required( + environment, + DOGFOOD_EMBEDDING_PROVIDER_ENV, + ), + ) + except (TypeError, ValueError): + raise DogfoodConfigurationUnavailable( + "dogfood API configuration is unavailable" + ) from None + + def authentication(self) -> VerifiedAuthenticationContext: + return VerifiedAuthenticationContext( + organization_ref=str(self.organization_id), + user_ref=str(self.user_id), + principal_ref=self.principal_ref, + membership_ref=str(self.membership_id), + membership_version=self.membership_version, + agent_version_ref=self.agent_version_ref, + authenticated_application_ref=self.application_ref, + authentication_binding_ref=self.authentication_binding_ref, + ) + + def query_digest_keyring(self) -> QueryDigestKeyring: + return QueryDigestKeyring( + active_version=1, + keys={ + 1: sha256( + _QUERY_DIGEST_DERIVATION_DOMAIN + self.secret.encode("utf-8") + ).digest() + }, + ) + + +def create_dogfood_app( + configuration: DogfoodConfiguration, + environment: Mapping[str, str], + *, + host: str, +) -> FastAPI: + """Compose the exact local carrier; every dependency remains sealed.""" + + if type(configuration) is not DogfoodConfiguration: + raise TypeError("dogfood API configuration is required") + if host not in {"127.0.0.1", "::1"}: + raise DogfoodConfigurationUnavailable( + "dogfood API configuration is unavailable" + ) + database_configuration = load_database_configuration( + DatabasePurpose.API_RUNTIME, + environment, + ) + runtime_engine = create_database_engine(database_configuration) + runtime = Runtime( + required_kernel_dependencies(), + candidate_index=PostgreSQLVectorCandidateIndex(DeterministicEmbeddingTwin()), + query_digest_keyring=configuration.query_digest_keyring(), + ) + membership_authority = PostgreSQLMembershipAuthority(runtime_engine) + try: + with membership_authority.current_user_actor( + MembershipIdentity( + organization_id=configuration.organization_id, + user_id=configuration.user_id, + membership_id=configuration.membership_id, + membership_version=configuration.membership_version, + principal_ref=configuration.principal_ref, + request_id="dogfood-composition-activation", + authentication_binding_ref=( + configuration.authentication_binding_ref + ), + checked_at=datetime.now(UTC), + ) + ) as current_user_actor: + release = current_user_actor.active_runtime_release + if ( + release is None + or release.organization_id != configuration.organization_id + or release.index_profile_ref + != DOGFOOD_VECTOR_INDEX_PROFILE_REF_V1 + or release.index_profile_digest + != DOGFOOD_VECTOR_INDEX_PROFILE_DIGEST_V1 + or not release.active_revision_refs + ): + raise DogfoodConfigurationUnavailable( + "dogfood API configuration is unavailable" + ) + except DogfoodConfigurationUnavailable: + runtime_engine.dispose() + raise + except ( + ActiveReleaseUnavailable, + MembershipAuthorityUnavailable, + MembershipNotCurrent, + ): + runtime_engine.dispose() + raise DogfoodConfigurationUnavailable( + "dogfood API configuration is unavailable" + ) from None + + from adapters.http.app import ( + _construct_runtime_delivery_activation, + create_app, + ) + + app = create_app( + authenticator=DogfoodAuthenticator( + secret=configuration.secret, + authentication=configuration.authentication(), + ), + organization_authority=DogfoodOrganizationAuthority( + configuration.organization_id + ), + membership_authority=membership_authority, + scope_authority=DogfoodFileScopeAuthority( + organization_id=configuration.organization_id, + principal_ref=configuration.principal_ref, + agent_version_ref=configuration.agent_version_ref, + purpose="context.answer", + ), + runtime=runtime, + runtime_delivery_activation=_construct_runtime_delivery_activation(), + ) + app.add_event_handler("shutdown", runtime_engine.dispose) + return app + + +def create_served_app( + environment: Mapping[str, str] | None = None, + *, + host: str | None = None, +) -> FastAPI: + """Select reject-all by absence; reject partial or unsupported opt-in.""" + + from adapters.http.app import create_app + + source = os.environ if environment is None else environment + composition = source.get(DOGFOOD_COMPOSITION_ENV) + if composition is None: + return create_app() + if composition != DOGFOOD_COMPOSITION_VALUE: + raise DogfoodConfigurationUnavailable( + "API composition configuration is unavailable" + ) + if host is None: + raise DogfoodConfigurationUnavailable( + "dogfood API configuration is unavailable" + ) + return create_dogfood_app( + DogfoodConfiguration.load(source), + source, + host=host, + ) diff --git a/adapters/http/organization_authority.py b/adapters/http/organization_authority.py index 96bd78eb..5dae0b52 100644 --- a/adapters/http/organization_authority.py +++ b/adapters/http/organization_authority.py @@ -2,9 +2,13 @@ from datetime import datetime from typing import Protocol +from uuid import UUID from adapters.http.authentication import VerifiedAuthenticationContext -from engine.runtime.organization import ExistingOrganizationVerification +from engine.runtime.organization import ( + ExistingOrganizationVerification, + _construct_existing_http_organization_verification, +) class OrganizationVerificationRejected(Exception): @@ -34,3 +38,33 @@ def verify_existing( verified_at: datetime, ) -> ExistingOrganizationVerification: raise OrganizationVerificationRejected + + +class DogfoodOrganizationAuthority: + """Bind Organization proof to the sole locally configured identity.""" + + __slots__ = ("_organization_id",) + + def __init__(self, organization_id: UUID) -> None: + if type(organization_id) is not UUID: + raise TypeError("dogfood Organization must be UUID") + self._organization_id = organization_id + + def verify_existing( + self, + authentication: VerifiedAuthenticationContext, + *, + request_id: str, + verified_at: datetime, + ) -> ExistingOrganizationVerification: + if ( + type(authentication) is not VerifiedAuthenticationContext + or authentication.organization_ref != str(self._organization_id) + ): + raise OrganizationVerificationRejected + return _construct_existing_http_organization_verification( + organization_id=self._organization_id, + request_id=request_id, + authentication_binding_ref=authentication.authentication_binding_ref, + verified_at=verified_at, + ) diff --git a/adapters/http/scope_authority.py b/adapters/http/scope_authority.py index 3bcdfc18..0cfa63f7 100644 --- a/adapters/http/scope_authority.py +++ b/adapters/http/scope_authority.py @@ -10,8 +10,19 @@ from uuid import UUID from engine.runtime.actor import MAX_MEMBERSHIP_VERSION +from engine.runtime.materialized import ( + MaterializedProjectionSession, + MaterializedScopeUnavailable, + _current_materialized_scope_operands, + _require_active_materialized_projection_session, +) from engine.runtime.policy_epoch import MAX_POLICY_EPOCH -from engine.runtime.scope import MISSING_TRUSTED_SCOPE +from engine.runtime.release_lineage import ( + DOGFOOD_VECTOR_INDEX_PROFILE_DIGEST_V1, + DOGFOOD_VECTOR_INDEX_PROFILE_REF_V1, + ActiveRuntimeRelease, +) +from engine.runtime.scope import MISSING_TRUSTED_SCOPE, ScopeSet from engine.runtime.scope_authority import ( TrustedScopeSnapshot, _close_scope_authority_scope, @@ -39,6 +50,14 @@ class ScopeAuthorityIdentity: request_id: str = field(repr=False) authentication_binding_ref: str = field(repr=False) checked_at: datetime = field(repr=False) + materialized_projection_session: MaterializedProjectionSession | None = field( + default=None, + repr=False, + ) + active_runtime_release: ActiveRuntimeRelease | None = field( + default=None, + repr=False, + ) def __post_init__(self) -> None: for field_name in ("organization_id", "user_id", "membership_id"): @@ -52,6 +71,14 @@ def __post_init__(self) -> None: "Scope authority Membership version must fit a positive " "signed 64-bit integer" ) + if self.materialized_projection_session is not None: + _require_active_materialized_projection_session( + self.materialized_projection_session + ) + if self.active_runtime_release is not None and type( + self.active_runtime_release + ) is not ActiveRuntimeRelease: + raise TypeError("Scope authority active release has the wrong nominal type") if ( type(self.policy_epoch) is not int or not 1 <= self.policy_epoch <= MAX_POLICY_EPOCH @@ -133,3 +160,99 @@ def current_scope( if type(identity) is not ScopeAuthorityIdentity: raise TypeError("Scope authority identity must be ScopeAuthorityIdentity") return _missing_trusted_scope(identity) + + +class DogfoodFileScopeAuthority: + """Bound one local dogfood Agent/purpose to current File projection facts.""" + + __slots__ = ( + "_agent_version_ref", + "_organization_id", + "_principal_ref", + "_purpose", + ) + + def __init__( + self, + *, + organization_id: UUID, + principal_ref: str, + agent_version_ref: str, + purpose: str, + ) -> None: + if type(organization_id) is not UUID: + raise TypeError("dogfood scope Organization must be UUID") + for field_name, value in ( + ("principal_ref", principal_ref), + ("agent_version_ref", agent_version_ref), + ("purpose", purpose), + ): + if type(value) is not str or not value or value.isspace(): + raise ValueError(f"dogfood scope {field_name} must be non-empty") + self._organization_id = organization_id + self._principal_ref = principal_ref + self._agent_version_ref = agent_version_ref + self._purpose = purpose + + @contextmanager + def current_scope( + self, + identity: ScopeAuthorityIdentity, + ) -> Iterator[TrustedScopeSnapshot]: + if type(identity) is not ScopeAuthorityIdentity: + raise TypeError("Scope authority identity must be ScopeAuthorityIdentity") + if ( + identity.organization_id != self._organization_id + or identity.principal_ref != self._principal_ref + or identity.agent_version_ref != self._agent_version_ref + or identity.purpose != self._purpose + or identity.materialized_projection_session is None + or identity.active_runtime_release is None + or identity.active_runtime_release.organization_id + != identity.organization_id + or identity.active_runtime_release.index_profile_ref + != DOGFOOD_VECTOR_INDEX_PROFILE_REF_V1 + or identity.active_runtime_release.index_profile_digest + != DOGFOOD_VECTOR_INDEX_PROFILE_DIGEST_V1 + or not identity.active_runtime_release.active_revision_refs + ): + raise ScopeAuthorityUnavailable( + "dogfood scope binding is unavailable" + ) + try: + durable = _current_materialized_scope_operands( + identity.materialized_projection_session, + identity.active_runtime_release.active_revision_refs, + ) + except (MaterializedScopeUnavailable, TypeError, ValueError): + raise ScopeAuthorityUnavailable( + "dogfood scope authority is unavailable" + ) from None + authority_scope = _open_scope_authority_scope() + try: + yield _construct_trusted_scope_snapshot( + authority_scope=authority_scope, + organization_id=identity.organization_id, + user_id=identity.user_id, + membership_id=identity.membership_id, + membership_version=identity.membership_version, + policy_epoch=identity.policy_epoch, + principal_ref=identity.principal_ref, + agent_version_ref=identity.agent_version_ref, + purpose=identity.purpose, + request_id=identity.request_id, + authentication_binding_ref=identity.authentication_binding_ref, + checked_at=identity.checked_at, + organization_boundary=ScopeSet(durable.organization_boundary), + membership_rights=ScopeSet(durable.membership_rights), + principal_grants=ScopeSet(durable.principal_grants), + # This local composition binds the one configured Agent and + # purpose to the Release-selected Organization ceiling. A + # binding mismatch above is a missing operand, never a grant. + agent_ceiling=ScopeSet(durable.organization_boundary), + source_native_acl=ScopeSet(durable.source_native_acl), + resource_acl=ScopeSet(durable.resource_acl), + purpose_policy=ScopeSet(durable.organization_boundary), + ) + finally: + _close_scope_authority_scope(authority_scope) diff --git a/adapters/pgvector.py b/adapters/pgvector.py index 38f000c0..05cad7a5 100644 --- a/adapters/pgvector.py +++ b/adapters/pgvector.py @@ -9,6 +9,7 @@ MaterializedProjectionSession, _discover_materialized_vector, ) +from engine.runtime.scope import EffectiveScope from engine.supply import ( CONTEXT_FRAGMENT_EMBEDDING_DIMENSION, EmbeddingProfile, @@ -56,9 +57,13 @@ def discover( self, request: Acquire, projection_session: MaterializedProjectionSession, + *, + effective_scope: EffectiveScope, ) -> tuple[CandidateRef, ...]: if type(request) is not Acquire: raise TypeError("Vector candidate discovery requires Acquire") + if type(effective_scope) is not EffectiveScope: + raise TypeError("Vector candidate discovery requires EffectiveScope") try: query_embedding = validate_embedding_batch( (request.need.query,), @@ -79,6 +84,7 @@ def discover( if request.narrowing is not None else None ), + effective_scope=effective_scope, ) except EmbeddingProviderUnavailable: raise VectorCandidateIndexUnavailable( diff --git a/applications/api.py b/applications/api.py index 8a64ac73..6b3173c3 100644 --- a/applications/api.py +++ b/applications/api.py @@ -1,10 +1,17 @@ """Engine API process entry point.""" import argparse +import os from collections.abc import Sequence +from typing import Any import uvicorn +from adapters.http.dogfood import ( + DOGFOOD_COMPOSITION_ENV, + DOGFOOD_COMPOSITION_VALUE, +) + def main(argv: Sequence[str] | None = None) -> None: parser = argparse.ArgumentParser(description="ContextEngine API") @@ -12,8 +19,18 @@ def main(argv: Sequence[str] | None = None) -> None: parser.add_argument("--port", default=8000, type=int) parser.add_argument("--log-level", default="info") args = parser.parse_args(argv) + if ( + os.environ.get(DOGFOOD_COMPOSITION_ENV) == DOGFOOD_COMPOSITION_VALUE + and args.host not in {"127.0.0.1", "::1"} + ): + parser.error("dogfood composition accepts only an explicit loopback host") + served_app: Any = "adapters.http.app:app" + if os.environ.get(DOGFOOD_COMPOSITION_ENV) is not None: + from adapters.http.dogfood import create_served_app + + served_app = create_served_app(os.environ, host=args.host) uvicorn.run( - "adapters.http.app:app", + served_app, host=args.host, port=args.port, log_level=args.log_level, diff --git a/applications/dogfood.py b/applications/dogfood.py new file mode 100644 index 00000000..ef85e986 --- /dev/null +++ b/applications/dogfood.py @@ -0,0 +1,127 @@ +"""Explicit local identity seeding for the dogfood composition.""" + +from __future__ import annotations + +import argparse +from collections.abc import Sequence +from datetime import UTC, datetime +from uuid import UUID + +from sqlalchemy import text + +from engine.persistence import ( + DatabasePurpose, + create_database_engine, + load_database_configuration, +) +from engine.persistence.role_guard import assert_migrator_role + + +def _uuid(value: str) -> UUID: + try: + return UUID(value) + except ValueError as error: + raise argparse.ArgumentTypeError("expected a canonical UUID") from error + + +def main(argv: Sequence[str] | None = None) -> None: + parser = argparse.ArgumentParser( + description="Seed one local Organization/User/current Membership" + ) + parser.add_argument("--organization-id", required=True, type=_uuid) + parser.add_argument("--user-id", required=True, type=_uuid) + parser.add_argument("--membership-id", required=True, type=_uuid) + parser.add_argument("--membership-version", default=1, type=int) + args = parser.parse_args(argv) + if not 1 <= args.membership_version < (1 << 63): + parser.error("--membership-version must be a positive signed bigint") + + engine = create_database_engine( + load_database_configuration(DatabasePurpose.MIGRATION) + ) + seeded_at = datetime.now(UTC).replace(microsecond=0) + try: + with engine.begin() as connection: + assert_migrator_role(connection) + connection.execute( + text( + """ + INSERT INTO organization (organization_id) + VALUES (:organization_id) + ON CONFLICT (organization_id) DO NOTHING + """ + ), + {"organization_id": args.organization_id}, + ) + connection.execute( + text( + """ + INSERT INTO user_account (user_id) + VALUES (:user_id) + ON CONFLICT (user_id) DO NOTHING + """ + ), + {"user_id": args.user_id}, + ) + connection.execute( + text( + """ + INSERT INTO membership ( + organization_id, membership_id, user_id, status, + membership_version, valid_from, valid_until + ) VALUES ( + :organization_id, :membership_id, :user_id, 'active', + :membership_version, :valid_from, NULL + ) + ON CONFLICT (organization_id, membership_id) DO NOTHING + """ + ), + { + "organization_id": args.organization_id, + "user_id": args.user_id, + "membership_id": args.membership_id, + "membership_version": args.membership_version, + "valid_from": seeded_at, + }, + ) + exact = connection.execute( + text( + """ + SELECT EXISTS ( + SELECT 1 + FROM membership + WHERE organization_id = :organization_id + AND membership_id = :membership_id + AND user_id = :user_id + AND status = 'active' + AND membership_version = :membership_version + AND valid_from <= statement_timestamp() + AND ( + valid_until IS NULL + OR statement_timestamp() < valid_until + ) + ) + """ + ), + { + "organization_id": args.organization_id, + "user_id": args.user_id, + "membership_id": args.membership_id, + "membership_version": args.membership_version, + }, + ).scalar_one() + if exact is not True: + raise RuntimeError("dogfood identity conflicts with durable ownership") + finally: + engine.dispose() + print( + "dogfood identity ready: " + f"organization={args.organization_id} " + f"user={args.user_id} membership={args.membership_id} " + f"version={args.membership_version}", + flush=True, + ) + + +if __name__ == "__main__": + main() diff --git a/docs/decisions/0068-activate-loopback-dogfood-runtime.md b/docs/decisions/0068-activate-loopback-dogfood-runtime.md new file mode 100644 index 00000000..ce01c95d --- /dev/null +++ b/docs/decisions/0068-activate-loopback-dogfood-runtime.md @@ -0,0 +1,118 @@ +--- +name: adr-0068-activate-loopback-dogfood-runtime +version: "1.0.0" +description: > + Activate one loopback-only, single-Membership File pgvector Acquire carrier + with deterministic twin embeddings and EffectiveScope reduction before LIMIT. +--- + +# 0068. Activate the loopback dogfood Runtime + +- Status: accepted +- Date: 2026-07-27 +- Refines: ADR-0021, ADR-0025, ADR-0063, ADR-0067 + +## Context + +ADR-0063 admits one explicitly configured local authentication composition but +does not activate content delivery. ADR-0067 admits the pgvector candidate seam +but leaves it out of the served composition until the Kernel-computed +`EffectiveScope` can remove out-of-policy rows before the bounded ANN result. +The maintainer dogfood loop needs both decisions to become one runnable, +auditable carrier without widening the default application or creating a +production-authentication ancestor. + +Runtime cannot yet charge an external query embedding call to the effective +`PackageBudget`, and no external provider profile is active. Those gaps prohibit +an external provider in this carrier. The deterministic twin, by contrast, has +one Release-bound model/input profile identity. + +## Decision + +The served API may activate one bounded dogfood composition only when the exact +local composition selector and every required identity, secret, Runtime +database, and embedding setting are present and valid. + +1. The module-level ASGI application is always the existing reject-all + composition. The API CLI validates an explicit loopback host before it + constructs the dogfood application; direct `uvicorn adapters.http.app:app` + therefore cannot activate delivery. The default performs zero content I/O + and reports `runtime_delivery: NOT_ACTIVE`. +2. A constant-time bearer check maps one environment-held secret to one fixed + Organization, User, Membership/version, Principal, Agent, application, and + authentication binding. The secret is excluded from representations and is + never written to a response, log, `ContextRun`, or `DecisionAudit`. The + query-digest key is domain-separated and derived from that secret; rotating + the secret therefore also rotates the local digest-key material. +3. Only identity verification is simplified. Every accepted request opens the + ordinary current-Membership `UserActor` transaction, redeems the active + Release, computes trusted scope facts in that same transaction, traverses + the non-pluggable `AuthorizationKernel`, projects an `AuthorizedProjection`, + rechecks current Organization Policy Epoch, and persists the existing + authorized-only lineage. +4. The dogfood scope authority carries the operands separately: Organization + boundary and Mirrored File source lifecycle come from the current RLS-visible + Release selection, Membership rights come from current field rights, + Principal grant and Resource ACL come from current File access policy, and + the exact configured Agent and `context.answer` purpose each receive only + the Release-selected Organization ceiling. A missing or empty operand + absorbs the intersection. This is a local File composition, not a general + Principal, Agent, purpose, or source-native policy system. +5. The Kernel passes its computed `EffectiveScope` to candidate discovery. + PostgreSQL matches exact Organization/source/resource triples before ANN + ordering and `LIMIT`; the filter can only remove candidates. Every returned + `CandidateRef` still undergoes exact Kernel reauthorization and field + projection. Index, RLS, Release, and candidate rank never grant authority. +6. Query embedding is the deterministic network-free twin only. Its model and + contextual-fragment input profile are bound by the active Release and are + validated at composition activation and on every request. Its effective + Package usage is zero external provider calls, zero cost, and zero provider + elapsed time. Operators must freshly reimport the dogfood corpus with the + worker's same twin before publishing that Release. Any external or unknown + query-provider selection fails composition because exact external usage + accounting is not active. +7. Identity seeding is an explicit idempotent local operation using only the + configured migrator connection. It creates one Organization, one User, and + one current Membership if absent; it does not run at API startup and never + gives the Runtime process migration authority. + +The activation records name only the loopback single-Membership authentication +and File pgvector Acquire carriers. Production authentication, a second human, +network exposure beyond the maintainer machine, group/public audience, +dogfood `OpenCitation`, `Continue`, hybrid retrieval, external query embedding, +and non-File providers remain `NOT_ACTIVE`. + +## Rationale + +Passing exact effective scope into discovery closes the selective-filter recall +gap without confusing an optimization with authorization. Keeping scope +derivation, discovery, authorization, projection, and final epoch validation in +one current `UserActor` transaction prevents trusted facts from being rebuilt +across sessions. A network-free twin makes the first real dogfood loop honest +about both cost and profile compatibility while durable external-provider +lineage remains unavailable. + +## Consequences + +- A configured maintainer can receive real Evidence-bearing File + `ContextPackage`s over `POST /v0/resolve`; `/health` reports the carrier + `ACTIVE` only for that explicit composition. +- Revoked or expired Membership and mid-resolve Policy Epoch change fail closed + through the existing gates. +- Trusted scope materialization is exact but can grow linearly with the active + dogfood Release. That is acceptable for this bounded local carrier; a larger + corpus requires measurement and a durable policy representation. +- Empty or embedding-profile-mismatched active Release lineage, unavailable scope facts, malformed + configuration, non-loopback binding, and external provider selection fail + closed. +- This composition must be deleted or replaced, not widened, when production + authentication or a second human becomes necessary. + +## Revisit trigger + +Revisit before a second human, remote ingress, group/public delivery, +dogfood `OpenCitation`, `Continue`, hybrid or non-File discovery, a general +source-native ACL authority, or an +external query embedding provider. External-provider activation additionally +requires immutable publication/query model plus input-profile identity and +Runtime-enforced provider-call, cost, and elapsed accounting. diff --git a/docs/decisions/README.md b/docs/decisions/README.md index 373287dc..519fc8ce 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -53,6 +53,7 @@ kernel, capability separation, and publication visibility model. | Recursive File discovery | [0065 — Recurse File discovery with anchored descriptors](0065-recurse-file-discovery-with-anchored-descriptors.md) | Canonical nested Markdown paths are discovered through stable descriptor-relative no-follow traversal under one server-owned 1–64 MiB byte ceiling | Path-string traversal, symlink following, truncated baselines, unbounded or caller-owned read limits, non-Markdown discovery, or new polling/resync authority | | Fragment embedding publication | [0066 — Embed Fragments before publication](0066-embed-fragments-before-publication.md) | New immutable Fragments receive one validated 384-dimensional vector before activation through an explicit external provider or network-free CI twin | Implicit twin fallback, query-time retrieval, backfill authority, missing-vector activation, or vector-based authorization | | Vector candidate discovery | [0067 — Discover vector candidates in the current Runtime transaction](0067-discover-vector-candidates-in-the-current-runtime-transaction.md) | Query embedding uses the stored profile and bounded filtered HNSW discovery stays inside the retained UserActor transaction, returning content-free CandidateRefs only | Second retrieval connections, vector rank as authority, content/score-bearing candidates, implicit provider fallback, or served carrier activation | +| Loopback dogfood Runtime | [0068 — Activate the loopback dogfood Runtime](0068-activate-loopback-dogfood-runtime.md) | One explicit single-Membership File Acquire carrier applies exact EffectiveScope before ANN LIMIT, then traverses the sealed Kernel with deterministic twin queries | Production/multi-user authentication, remote ingress, group/public/Continue, hybrid/non-File retrieval, or external query embeddings | | Private delivery ingress | [0045 — Redeem private delivery evidence at ingress](0045-redeem-private-delivery-evidence-at-ingress.md) | One digest-only service/request/asker/audience/epoch-bound DeliveryEvidenceRef constructs private TrustedDeliveryContext inside the current UserActor transaction before content work | Raw trusted delivery facts on the wire, bearer persistence, application-role minting/table reads, alternate Runtime paths, or claiming later M2 carriers | | Exact Package egress | [0046 — Bind egress to one exact Package hop](0046-bind-egress-to-one-exact-package-hop.md) | One digest-only grant binds one exact audience-bound Package to one model or channel preflight hop and redeems atomically | Treating Package construction as disclosure authority, arbitrary content at egress, cross-hop reuse, or bypassing final policy | | Public OpenAPI v0 | [0047 — Freeze OpenAPI v0 through one Runtime path](0047-freeze-openapi-v0-through-one-runtime-path.md) | One public `/v0/resolve` schema and a hidden provisional v1 bridge share the same sealed Runtime; Package release lineage is read-only from the Learning-published active manifest | Two authorization compositions, caller-authored release facts, Runtime publication/fallback, or in-place mutation of historical snapshots | @@ -168,3 +169,4 @@ touched: - [0065 — Recursive File discovery with anchored descriptors](0065-recurse-file-discovery-with-anchored-descriptors.md) - [0066 — Embed Fragments before publication](0066-embed-fragments-before-publication.md) - [0067 — Discover vector candidates in the current Runtime transaction](0067-discover-vector-candidates-in-the-current-runtime-transaction.md) +- [0068 — Activate the loopback dogfood Runtime](0068-activate-loopback-dogfood-runtime.md) diff --git a/engine/persistence/membership_context.py b/engine/persistence/membership_context.py index 380efd26..af6dfa0d 100644 --- a/engine/persistence/membership_context.py +++ b/engine/persistence/membership_context.py @@ -66,6 +66,8 @@ MaterializedProjectionKind, MaterializedProjectionSession, MaterializedPublicationTrace, + MaterializedScopeOperands, + MaterializedScopeUnavailable, _close_materialized_projection_scope, _construct_materialized_projection_session, _open_materialized_projection_scope, @@ -79,6 +81,7 @@ _open_policy_epoch_authority_scope, ) from engine.runtime.release_lineage import ActiveRuntimeRelease +from engine.runtime.scope import EffectiveScope, ScopeTarget class MembershipNotCurrent(Exception): @@ -180,6 +183,19 @@ def _canonical_candidate_revision(value: str) -> UUID | None: AND resource.active_revision_id = fragment.revision_id AND resource.tombstoned IS FALSE WHERE fragment.embedding IS NOT NULL + AND ( + EXISTS ( + SELECT 1 + FROM unnest( + CAST(:scope_resource_organization_ids AS uuid[]), + CAST(:scope_resource_source_refs AS text[]), + CAST(:scope_resource_refs AS text[]) + ) AS resource_scope(organization_id, source_ref, resource_ref) + WHERE resource_scope.organization_id = resource.organization_id + AND resource_scope.source_ref = resource.source_ref + AND resource_scope.resource_ref = fragment.resource_ref + ) + ) AND ( CAST(:source_refs AS text[]) IS NULL OR resource.source_ref = ANY(CAST(:source_refs AS text[])) @@ -213,13 +229,164 @@ def source_is_active(self, source_ref: UUID) -> bool: ).scalar_one() return observed is True + def current_scope_operands( + self, + active_revision_ids: tuple[UUID, ...], + ) -> MaterializedScopeOperands: + """Observe each durable File scope operand without pre-intersection.""" + + try: + rows = self._connection.execute( + text( + """ + SELECT DISTINCT + resource.organization_id, + resource.source_ref, + resource.resource_ref, + access_policy.resource_ref IS NOT NULL AS principal_granted, + EXISTS ( + SELECT 1 + FROM context_fragment AS fragment + WHERE fragment.organization_id = resource.organization_id + AND fragment.resource_ref = resource.resource_ref + AND fragment.revision_id = resource.active_revision_id + AND ( + ( + fragment.projection_kind = 'body' + AND EXISTS ( + SELECT 1 + FROM membership_resource_field_right AS field_right + WHERE field_right.organization_id = + fragment.organization_id + AND field_right.membership_id = NULLIF( + current_setting('app.membership_id'), '' + )::uuid + AND field_right.membership_version = NULLIF( + current_setting('app.membership_version'), '' + )::bigint + AND field_right.resource_ref = + fragment.resource_ref + AND field_right.field_ref = 'body' + ) + ) + OR ( + fragment.projection_kind = 'fields' + AND EXISTS ( + SELECT 1 + FROM context_fragment_field AS fragment_field + JOIN membership_resource_field_right AS field_right + ON field_right.organization_id = + fragment_field.organization_id + AND field_right.membership_id = NULLIF( + current_setting('app.membership_id'), '' + )::uuid + AND field_right.membership_version = NULLIF( + current_setting('app.membership_version'), '' + )::bigint + AND field_right.resource_ref = + fragment_field.resource_ref + AND field_right.field_ref = + fragment_field.field_ref + WHERE fragment_field.organization_id = + fragment.organization_id + AND fragment_field.resource_ref = + fragment.resource_ref + AND fragment_field.revision_id = + fragment.revision_id + AND fragment_field.fragment_ref = + fragment.fragment_ref + ) + ) + ) + ) AS membership_allowed + FROM context_resource AS resource + LEFT JOIN resource_access_policy AS access_policy + ON access_policy.organization_id = resource.organization_id + AND access_policy.resource_ref = resource.resource_ref + AND access_policy.principal_ref = current_setting( + 'app.principal_ref' + ) + AND access_policy.access_state = 'allowed' + WHERE resource.tombstoned IS FALSE + AND resource.active_revision_id IS NOT NULL + AND resource.active_revision_id = ANY( + CAST(:active_revision_ids AS uuid[]) + ) + ORDER BY resource.organization_id, + resource.source_ref, + resource.resource_ref + """ + ), + {"active_revision_ids": list(active_revision_ids)}, + ) + except SQLAlchemyError: + raise MaterializedScopeUnavailable( + "materialized scope authority is unavailable" + ) from None + try: + rows_with_targets = tuple( + ( + ScopeTarget( + row.organization_id, + row.source_ref, + row.resource_ref, + ), + row.membership_allowed is True, + row.principal_granted is True, + ) + for row in rows + ) + organization_boundary = frozenset( + target for target, _membership, _principal in rows_with_targets + ) + membership_rights = frozenset( + target + for target, membership, _principal in rows_with_targets + if membership + ) + principal_grants = frozenset( + target + for target, _membership, principal in rows_with_targets + if principal + ) + # context_resource RLS already enforces the current File source + # lifecycle. That visible set is the local Mirrored source-native + # ACL; resource_access_policy separately carries the principal's + # grant and the resource ACL in the current File access model. + return MaterializedScopeOperands( + organization_boundary=organization_boundary, + membership_rights=membership_rights, + principal_grants=principal_grants, + source_native_acl=organization_boundary, + resource_acl=principal_grants, + ) + except (TypeError, ValueError): + raise MaterializedScopeUnavailable( + "materialized scope authority is unavailable" + ) from None + def discover_vector( self, query_embedding: tuple[float, ...], limit: int, source_refs: tuple[str, ...] | None, resource_refs: tuple[str, ...] | None, + effective_scope: EffectiveScope, ) -> tuple[CandidateRef, ...]: + resource_targets = tuple( + sorted( + ( + target + for target in effective_scope.targets + if target.resource_ref is not None + ), + key=lambda target: ( + target.organization_id.bytes, + target.source_ref, + target.resource_ref or "", + ), + ) + ) self._connection.execute( text( "SELECT set_config('hnsw.iterative_scan', :iterative_scan, true), " @@ -241,6 +408,15 @@ def discover_vector( "resource_refs": ( list(resource_refs) if resource_refs is not None else None ), + "scope_resource_organization_ids": [ + target.organization_id for target in resource_targets + ], + "scope_resource_source_refs": [ + target.source_ref for target in resource_targets + ], + "scope_resource_refs": [ + target.resource_ref for target in resource_targets + ], }, ) candidates: list[CandidateRef] = [] diff --git a/engine/persistence/role_guard.py b/engine/persistence/role_guard.py index f03a1bc9..0b64f53d 100644 --- a/engine/persistence/role_guard.py +++ b/engine/persistence/role_guard.py @@ -106,6 +106,30 @@ def assert_control_role(connection: Connection) -> None: _assert_non_owner_role(connection, CONTROL_ROLE) +def assert_migrator_role(connection: Connection) -> None: + """Require the explicit migration login for the local seeding operation.""" + + row = connection.execute( + text( + """ + SELECT current_user AS current_role, + session_user AS session_role, + role.rolsuper AS is_superuser, + role.rolbypassrls AS bypasses_rls + FROM pg_roles AS role + WHERE role.rolname = current_user + """ + ) + ).mappings().one() + if dict(row) != { + "current_role": MIGRATOR_ROLE, + "session_role": MIGRATOR_ROLE, + "is_superuser": False, + "bypasses_rls": False, + }: + raise AssertionError("dogfood seeding requires the exact migrator login") + + def assert_identity_role(connection: Connection) -> None: """Require the dedicated trusted-identity evidence issuer login.""" diff --git a/engine/runtime/construction.py b/engine/runtime/construction.py index 2806f2c9..19762d6a 100644 --- a/engine/runtime/construction.py +++ b/engine/runtime/construction.py @@ -625,7 +625,11 @@ def authorize_acquire( raise RuntimeConfigurationError( "candidate discovery requires same-transaction projection session" ) - discovered = candidate_index.discover(request, projection_session) + discovered = candidate_index.discover( + request, + projection_session, + effective_scope=policy_receipt.effective_scope, + ) if type(discovered) is not tuple or any( type(candidate) is not CandidateRef for candidate in discovered ): diff --git a/engine/runtime/content_io.py b/engine/runtime/content_io.py index 0137539c..601a0f05 100644 --- a/engine/runtime/content_io.py +++ b/engine/runtime/content_io.py @@ -7,6 +7,7 @@ from engine.runtime.contracts import Acquire from engine.runtime.evidence import CandidateRef from engine.runtime.materialized import MaterializedProjectionSession +from engine.runtime.scope import EffectiveScope __all__ = [ "CandidateIndex", @@ -36,6 +37,8 @@ def discover( self, request: Acquire, projection_session: MaterializedProjectionSession, + *, + effective_scope: EffectiveScope, ) -> tuple[CandidateRef, ...]: ... @@ -69,7 +72,10 @@ def discover( self, request: Acquire, projection_session: MaterializedProjectionSession, + *, + effective_scope: EffectiveScope, ) -> tuple[()]: + del request, projection_session, effective_scope raise RuntimeError("candidate index is prohibited on the empty Package path") diff --git a/engine/runtime/materialized.py b/engine/runtime/materialized.py index d3f85be7..4b5e5995 100644 --- a/engine/runtime/materialized.py +++ b/engine/runtime/materialized.py @@ -2,7 +2,7 @@ from dataclasses import dataclass, field from enum import StrEnum -from typing import Final, NoReturn, Protocol +from typing import Final, NoReturn, Protocol, runtime_checkable from uuid import UUID from engine.runtime.evidence import ( @@ -11,6 +11,7 @@ CandidateRef, validate_projected_field_refs, ) +from engine.runtime.scope import EffectiveScope, ScopeTarget __all__ = [ "MaterializedFieldValue", @@ -20,6 +21,9 @@ "MaterializedProjectionPort", "MaterializedProjectionSession", "MaterializedPublicationTrace", + "MaterializedScopeOperands", + "MaterializedScopePort", + "MaterializedScopeUnavailable", ] _STRUCTURED_FIELD_LINE_BREAKS: Final = frozenset( @@ -73,6 +77,38 @@ class MaterializedProjectionKind(StrEnum): STRUCTURED_FIELDS = "fields" +class MaterializedScopeUnavailable(RuntimeError): + """Same-transaction scope facts could not be established.""" + + +@dataclass(frozen=True, slots=True) +class MaterializedScopeOperands: + """Independent durable operands observed by one UserActor transaction.""" + + organization_boundary: frozenset[ScopeTarget] + membership_rights: frozenset[ScopeTarget] + principal_grants: frozenset[ScopeTarget] + source_native_acl: frozenset[ScopeTarget] + resource_acl: frozenset[ScopeTarget] + + def __post_init__(self) -> None: + for field_name in ( + "organization_boundary", + "membership_rights", + "principal_grants", + "source_native_acl", + "resource_acl", + ): + targets = getattr(self, field_name) + if type(targets) is not frozenset or any( + type(target) is not ScopeTarget or target.resource_ref is None + for target in targets + ): + raise TypeError( + f"materialized {field_name} must contain exact Resource targets" + ) + + @dataclass(frozen=True, slots=True) class MaterializedFieldValue: """One already-authorized field returned by the retained transaction.""" @@ -202,6 +238,7 @@ def discover_vector( limit: int, source_refs: tuple[str, ...] | None, resource_refs: tuple[str, ...] | None, + effective_scope: EffectiveScope, ) -> tuple[CandidateRef, ...]: ... def discover_exact_phrase( @@ -226,6 +263,16 @@ def project( locator: MaterializedFragmentLocator, ) -> MaterializedFragmentProjection | None: ... + +@runtime_checkable +class MaterializedScopePort(Protocol): + """Explicit optional capability for independent trusted scope operands.""" + + def current_scope_operands( + self, + active_revision_ids: tuple[UUID, ...], + ) -> MaterializedScopeOperands: ... + class _MaterializedProjectionScope: """Private lifetime token owned by one current UserActor transaction.""" @@ -267,6 +314,7 @@ class MaterializedProjectionSession: _authority_scope: _MaterializedProjectionScope = field(repr=False) _port: MaterializedProjectionPort = field(repr=False) + _scope_port: MaterializedScopePort | None = field(repr=False) def __init__(self, *args: object, **kwargs: object) -> None: raise TypeError( @@ -322,9 +370,49 @@ def _construct_materialized_projection_session( session = object.__new__(MaterializedProjectionSession) object.__setattr__(session, "_authority_scope", authority_scope) object.__setattr__(session, "_port", port) + object.__setattr__( + session, + "_scope_port", + port if isinstance(port, MaterializedScopePort) else None, + ) return session +def _current_materialized_scope_operands( + session: MaterializedProjectionSession, + active_revision_refs: tuple[str, ...], +) -> MaterializedScopeOperands: + """Read independent File-scope facts in the retained transaction.""" + + _require_active_materialized_projection_session(session) + if type(active_revision_refs) is not tuple or not active_revision_refs: + raise MaterializedScopeUnavailable( + "materialized scope release selection is unavailable" + ) + try: + revision_ids = tuple(UUID(value) for value in active_revision_refs) + except (AttributeError, TypeError, ValueError): + raise MaterializedScopeUnavailable( + "materialized scope release selection is unavailable" + ) from None + if any(str(value) != reference for value, reference in zip( + revision_ids, + active_revision_refs, + strict=True, + )): + raise MaterializedScopeUnavailable( + "materialized scope release selection is unavailable" + ) + if session._scope_port is None: + raise MaterializedScopeUnavailable( + "materialized scope authority is unavailable" + ) + operands = session._scope_port.current_scope_operands(revision_ids) + if type(operands) is not MaterializedScopeOperands: + raise TypeError("materialized scope authority returned the wrong nominal type") + return operands + + def _is_materialized_source_active( session: MaterializedProjectionSession, source_ref: UUID, @@ -379,6 +467,7 @@ def _discover_materialized_vector( *, source_refs: tuple[str, ...] | None = None, resource_refs: tuple[str, ...] | None = None, + effective_scope: EffectiveScope, ) -> tuple[CandidateRef, ...]: """Discover bounded content-free ANN lineage in the retained transaction.""" @@ -387,6 +476,8 @@ def _discover_materialized_vector( raise ValueError("vector discovery requires a nonempty query embedding") if type(limit) is not int or limit <= 0: raise ValueError("vector discovery requires a positive exact limit") + if type(effective_scope) is not EffectiveScope: + raise TypeError("vector discovery requires EffectiveScope") for field_name, refs in ( ("source_refs", source_refs), ("resource_refs", resource_refs), @@ -402,6 +493,7 @@ def _discover_materialized_vector( limit, source_refs, resource_refs, + effective_scope, ) if ( type(candidates) is not tuple diff --git a/engine/runtime/release_lineage.py b/engine/runtime/release_lineage.py index e4d527b7..694a59f7 100644 --- a/engine/runtime/release_lineage.py +++ b/engine/runtime/release_lineage.py @@ -18,6 +18,9 @@ class ActiveReleaseUnavailable(RuntimeError): CONTENT_PROFILE_REF_V0: Final = "content-materialized-v0" CONTENT_SCHEMA_REF_V0: Final = "context-content-schema-v1" INDEX_PROFILE_REF_V0: Final = "index-exact-phrase-v0" +DOGFOOD_VECTOR_INDEX_PROFILE_REF_V1: Final = ( + "index-file-pgvector-deterministic-twin-v1" +) INDEX_SCHEMA_REF_V0: Final = "context-index-schema-v1" CONTENT_PROFILE_DIGEST_V0: Final = sha256( b"context-engine.content-profile.materialized-v0" @@ -25,6 +28,11 @@ class ActiveReleaseUnavailable(RuntimeError): INDEX_PROFILE_DIGEST_V0: Final = sha256( b"context-engine.index-profile.exact-phrase-v0" ).hexdigest() +DOGFOOD_VECTOR_INDEX_PROFILE_DIGEST_V1: Final = sha256( + b"context-engine.index-profile.file-pgvector-v1\x00" + b"embedding-model:deterministic-twin-v1\x00" + b"embedding-input:contextual-fragment-v1" +).hexdigest() RUNTIME_PROFILE_DIGEST_V0: Final = sha256( b"context-engine.runtime-profile.materialized-openapi-v0" ).hexdigest() @@ -148,14 +156,23 @@ def __post_init__(self) -> None: raise ValueError( "active release Revisions must be unique and canonical" ) + supported_index_profile = ( + self.index_profile_ref, + self.index_profile_digest, + ) in { + (INDEX_PROFILE_REF_V0, INDEX_PROFILE_DIGEST_V0), + ( + DOGFOOD_VECTOR_INDEX_PROFILE_REF_V1, + DOGFOOD_VECTOR_INDEX_PROFILE_DIGEST_V1, + ), + } if ( self.content_profile_ref != CONTENT_PROFILE_REF_V0 or self.content_schema_ref != CONTENT_SCHEMA_REF_V0 - or self.index_profile_ref != INDEX_PROFILE_REF_V0 + or not supported_index_profile or self.index_schema_ref != INDEX_SCHEMA_REF_V0 or self.runtime_profile_ref != RUNTIME_PROFILE_REF_V0 or self.content_profile_digest != CONTENT_PROFILE_DIGEST_V0 - or self.index_profile_digest != INDEX_PROFILE_DIGEST_V0 or self.runtime_profile_digest != RUNTIME_PROFILE_DIGEST_V0 or self.tokenizer_ref != RUNTIME_TOKENIZER_REF_V0 or self.package_schema_ref != PACKAGE_SCHEMA_REF_V0 @@ -183,6 +200,8 @@ def __post_init__(self) -> None: "CONTENT_SCHEMA_REF_V0", "CURATION_PROFILE_DIGEST_V0", "CURATION_PROFILE_REF_V0", + "DOGFOOD_VECTOR_INDEX_PROFILE_DIGEST_V1", + "DOGFOOD_VECTOR_INDEX_PROFILE_REF_V1", "INDEX_PROFILE_DIGEST_V0", "INDEX_PROFILE_REF_V0", "INDEX_SCHEMA_REF_V0", diff --git a/eval/catalogs/m0-security-evidence.yaml b/eval/catalogs/m0-security-evidence.yaml index 5e14c1b4..5317f650 100644 --- a/eval/catalogs/m0-security-evidence.yaml +++ b/eval/catalogs/m0-security-evidence.yaml @@ -559,6 +559,21 @@ "id": "HTTP-FILE-MIXED-UPSERT-NO-DELETE-089", "layer": "runtime", "selector": "tests/integration/test_z_egress_grant_file.py::test_mixed_file_upsert_scheduling_has_zero_delete_effect_over_generated_sdk" + }, + { + "id": "RUNTIME-DOGFOOD-AUTH-102", + "layer": "runtime", + "selector": "tests/integration/test_dogfood_runtime_activation.py::test_dogfood_secret_and_membership_fail_closed_without_secret_retention" + }, + { + "id": "RUNTIME-DOGFOOD-CARRIER-102", + "layer": "runtime", + "selector": "tests/integration/test_dogfood_runtime_activation.py::test_dogfood_served_composition_delivers_release_scoped_file_evidence_before_limit" + }, + { + "id": "RUNTIME-DOGFOOD-EPOCH-102", + "layer": "runtime", + "selector": "tests/integration/test_dogfood_runtime_activation.py::test_dogfood_mid_resolve_policy_epoch_change_vetoes_stale_evidence" } ], "invariantMappings": [ @@ -653,7 +668,9 @@ "PG-FILE-SOURCE-OFFBOARD-030" ], "runtime": [ - "RUNTIME-INDEX-NOT-AUTHORITY-005" + "RUNTIME-INDEX-NOT-AUTHORITY-005", + "RUNTIME-DOGFOOD-CARRIER-102", + "RUNTIME-DOGFOOD-EPOCH-102" ] } }, @@ -732,7 +749,8 @@ "HTTP-DELIVERY-EVIDENCE-063", "FILE-DELIVERY-EVIDENCE-063", "SDK-LIVE-FILE-064", - "SDK-PRIVATE-BOT-FLOW-071" + "SDK-PRIVATE-BOT-FLOW-071", + "RUNTIME-DOGFOOD-AUTH-102" ] } }, diff --git a/eval/catalogs/security-catalog.schema.json b/eval/catalogs/security-catalog.schema.json index 6814622d..e8ed8d8d 100644 --- a/eval/catalogs/security-catalog.schema.json +++ b/eval/catalogs/security-catalog.schema.json @@ -95,8 +95,8 @@ }, "activations": { "type": "array", - "minItems": 22, - "maxItems": 22, + "minItems": 24, + "maxItems": 24, "uniqueItems": true, "prefixItems": [ { @@ -1073,6 +1073,87 @@ "Runtime authorization from Supply retry or lease state" ] } + }, + { + "const": { + "issueRef": "#102", + "invariantRef": "TRANSPORT-UNTRUSTED-008", + "carrier": "loopback-only single-Membership dogfood HTTP authentication", + "status": "active_fail_closed", + "policyEpochScope": "organization-v0", + "controlBoundary": "explicit local environment opt-in -> constant-time bearer verification -> one fixed configured identity -> current UserActor Membership transaction", + "testEvidence": [ + { + "id": "RUNTIME-DOGFOOD-AUTH-102", + "surface": "tests/integration/test_dogfood_runtime_activation.py::test_dogfood_secret_and_membership_fail_closed_without_secret_retention", + "oracle": "Absent, wrong, or truncated secrets and revoked or expired configured Membership authority return the same generic authentication failure; the secret is absent from responses, logs, ContextRun, and DecisionAudit." + } + ], + "deferredEvidence": [ + "production identity-provider authentication and key lifecycle", + "second-human and multi-Membership authentication", + "non-loopback network exposure and remote deployment" + ], + "futureCarriers": [ + "production authentication composition", + "multi-user identity and Membership selection", + "authenticated remote network ingress" + ], + "notActive": [ + "production authentication", + "a second human identity", + "network exposure beyond the maintainer machine", + "group or public audience", + "Continue", + "dogfood OpenCitation", + "hybrid retrieval", + "external query embedding", + "non-File providers" + ] + } + }, + { + "const": { + "issueRef": "#102", + "invariantRef": "INDEX-NOT-AUTHORITY-005", + "carrier": "loopback-only File pgvector dogfood Acquire delivery", + "status": "active_fail_closed", + "policyEpochScope": "organization-v0", + "controlBoundary": "current UserActor transaction -> database-derived exact EffectiveScope -> pre-LIMIT pgvector scope reduction -> CandidateRef -> sealed AuthorizationKernel -> AuthorizedProjection -> ContextPackage", + "testEvidence": [ + { + "id": "RUNTIME-DOGFOOD-CARRIER-102", + "surface": "tests/integration/test_dogfood_runtime_activation.py::test_dogfood_served_composition_delivers_release_scoped_file_evidence_before_limit", + "oracle": "The configured maintainer receives one real File Evidence block through the served POST /v0/resolve carrier even when more than the ANN limit of strictly closer RLS-visible distractors are excluded by EffectiveScope; the deterministic twin records zero external provider usage." + }, + { + "id": "RUNTIME-DOGFOOD-EPOCH-102", + "surface": "tests/integration/test_dogfood_runtime_activation.py::test_dogfood_mid_resolve_policy_epoch_change_vetoes_stale_evidence", + "oracle": "An access revocation that advances Policy Epoch after authorization but before delivery yields an authorized empty package with zero stale Evidence, while the sealed Kernel remains the final authority." + } + ], + "deferredEvidence": [ + "external-provider embedding metering and profile identity", + "multi-source or hybrid discovery and non-File source-native ACL semantics", + "group, public, Continue, and remote deployment carriers" + ], + "futureCarriers": [ + "metered external query embedding", + "hybrid and non-File retrieval", + "group, public, and Continue delivery" + ], + "notActive": [ + "production authentication", + "a second human identity", + "network exposure beyond the maintainer machine", + "group or public audience", + "Continue", + "dogfood OpenCitation", + "hybrid retrieval", + "external query embedding", + "non-File providers" + ] + } } ], "items": false @@ -1380,7 +1461,10 @@ "PROC-FILE-DISPATCH-091", "PG-FILE-RECLAIM-093", "PG-FILE-RECLAIM-CONCURRENCY-093", - "PROC-FILE-RECLAIM-093" + "PROC-FILE-RECLAIM-093", + "RUNTIME-DOGFOOD-AUTH-102", + "RUNTIME-DOGFOOD-CARRIER-102", + "RUNTIME-DOGFOOD-EPOCH-102" ] }, "surface": { @@ -1430,7 +1514,8 @@ "#87", "#89", "#91", - "#93" + "#93", + "#102" ] }, "invariantRef": { @@ -1469,7 +1554,9 @@ "explicit current File delete execution through sole tombstone authority", "explicit current mixed File page upsert-projection scheduling", "autonomous first-attempt dispatch of explicit scheduled File upserts", - "bounded autonomous reclaim of expired scheduled File upserts" + "bounded autonomous reclaim of expired scheduled File upserts", + "loopback-only single-Membership dogfood HTTP authentication", + "loopback-only File pgvector dogfood Acquire delivery" ] }, "status": { @@ -1506,7 +1593,9 @@ "exact trusted Source/SourceVersion/page/ordinal locator -> current complete v4 scan revalidation -> server-derived Resource/event lineage -> existing tombstone authority -> immutable exact execution binding", "complete current v4 upsert/delete page validation -> trusted ContextControl explicit FileImportAudience -> nonempty original-ordinal upsert projection -> exact existing acquisition/import-job lineage -> existing WorkerLease and current-scan publication fences", "function-only scheduler login -> current page/acquisition/audience/receiver eligibility -> deterministic SKIP LOCKED selector -> database-timed generation-one lease -> existing WorkerLease and File worker", - "function-only scheduler login -> expired lease plus database-owned exponential backoff -> current page/acquisition/audience/receiver revalidation -> deterministic recovery-first SKIP LOCKED selector -> exact next-generation WorkerLease -> existing durable-boundary File worker resume" + "function-only scheduler login -> expired lease plus database-owned exponential backoff -> current page/acquisition/audience/receiver revalidation -> deterministic recovery-first SKIP LOCKED selector -> exact next-generation WorkerLease -> existing durable-boundary File worker resume", + "explicit local environment opt-in -> constant-time bearer verification -> one fixed configured identity -> current UserActor Membership transaction", + "current UserActor transaction -> database-derived exact EffectiveScope -> pre-LIMIT pgvector scope reduction -> CandidateRef -> sealed AuthorizationKernel -> AuthorizedProjection -> ContextPackage" ] }, "testEvidence": { diff --git a/eval/catalogs/security-invariants.yaml b/eval/catalogs/security-invariants.yaml index e6f00fb3..ef464901 100644 --- a/eval/catalogs/security-invariants.yaml +++ b/eval/catalogs/security-invariants.yaml @@ -25,7 +25,8 @@ "#87", "#89", "#91", - "#93" + "#93", + "#102" ], "documentRefs": [ "README.md", @@ -55,9 +56,10 @@ "docs/decisions/0057-execute-current-file-deletes-through-tombstone-authority.md", "docs/decisions/0058-schedule-only-upserts-from-mixed-file-pages.md", "docs/decisions/0059-dispatch-scheduled-file-imports-through-exact-leases.md", - "docs/decisions/0060-reclaim-expired-file-imports-with-bounded-retries.md" + "docs/decisions/0060-reclaim-expired-file-imports-with-bounded-retries.md", + "docs/decisions/0068-activate-loopback-dogfood-runtime.md" ], - "reconciliation": "Issue #2 fixes the product and testing decisions, issue #5 requires exactly fifteen release invariants and twelve canonical acceptance fixtures, and ADR-0019 resolves the later nineteen-label prose expansion without weakening any safeguard. Issue #15 activates only Organization-level next-request resolve(Acquire) revocation evidence under REVOCATION-006: at that activation, PG-REVOCATION-006, RUN-006, and CACHE-002 are active while BLOB-002 and Continue, citation, Policy-Epoch-bound WorkerLease, production ContextAccessTicket/ActionTicket, audit, outbox, cleanup, finer-epoch, UI, and external-admin carriers remain future or NOT_ACTIVE; later issue records are authoritative for subsequently activated carriers. Issue #16 activates only the M0 refusal gate for unavailable Continue, profile-disabled OpenCitation, and server-owned unavailable Acquire plans: at that activation its real continuation, profile-enabled citation, federated/source-native, and File carriers remain future, while its Runtime and HTTP refusal surfaces prove generic outcomes before content I/O; Issue #69 later activates the private/direct File profile-enabled citation carrier. Issue #17 activates only the signed one-shot persistent no-op durable-job WorkerLease subcarrier under WORKER-LEASE-007. It binds one exact worker audience but no end-user delivery audience or Policy Epoch, and proves only LEASE-SIGNING-017, PG-WORKER-LEASE-NOOP-017, and WORKER-LEASE-REPLAY-007; Source, Resource, Revision, Policy Epoch, end-user delivery audience, idempotency, generation, business mutation, outbox, File publication, and the full ACCEPT-008 matrix remain deferred or NOT_ACTIVE. Issue #18 activates only distinct signed synthetic ContextAccessTicket Provider-read and ActionTicket no-op channel-action subcarriers under ACTION-SEPARATION-014, with current Organization-v0 Policy Epoch validation. TICKET-AUDIENCE-018 and PG-TICKET-EPOCH-018 do not activate production ContextProvider integration, ContextRuntime ticket integration, BotDelivery, full M2 ActionPlane.prepare/perform, a real Sender or external effect, payload/destination/approval/idempotency binding, durable one-shot/replay/reconciliation, or full ACCEPT-012 PASS; those remain future or NOT_ACTIVE. Issue #19 activates only the current Acquire authorized-only ContextRun and restricted delivered-empty DecisionAudit subcarrier under TRACE-REDACTION-012. DIGEST-019, RUN-LINEAGE-019, AUTHORIZED-RUN-019, and PG-TRACE-REDACTION-012 prove deterministic Package and Organization-bound query digests, retained-UserActor-transaction persistence, decisionRef resolution, redaction, and short-lived exact-Organization operator ticket reads with no application-role table access; the supported reader commits deletion before returning, while a direct caller rollback is not claimed as durable exactly-once redemption. Raw query retention, full ContextPackage body retention, unauthenticated transport failures as ContextRuns, cross-Organization analytics, and general observability redaction remain NOT_ACTIVE. Issue #48 activates only the current ACCEPT-002 authenticated HTTP Acquire Membership field-projection carrier under SCOPE-INTERSECTION-004, INDEX-NOT-AUTHORITY-005, and TRACE-REDACTION-012. PROP-FIELD-PROJECTION-048, PG-FIELD-PROJECTION-048, and HTTP-ACCEPT-002-048 bind one current Membership/version field ceiling to same-transaction FORCE-RLS reduction, the sealed AuthorizationKernel, AuthorizedProjection and Evidence integrity, and authorized-only ContextRun/audit persistence. General permission DSLs, caller-authored projection lists, CandidateRef or index field authority, production Provider/source-native ACL negotiation, Supply publication, File/Base field ACL, typed fields, Continue, and Issue #20 runner substitution remain future or NOT_ACTIVE; Issue #69 later activates private/direct File OpenCitation through the same field-projection gates. Issue #63 activates only the digest-only private authenticated HTTP Acquire DeliveryEvidenceRef carrier under TRANSPORT-UNTRUSTED-008. PROP-DELIVERY-EVIDENCE-063, PG-DELIVERY-EVIDENCE-063, HTTP-DELIVERY-EVIDENCE-063, and FILE-DELIVERY-EVIDENCE-063 prove exact service/request/Organization/asker/Membership-version/destination/consumer/purpose/audience/epoch/lifetime binding, stable identical retry identity, role isolation, expiry cleanup, pre-content generic rejection, and one File-backed sealed Runtime delivery. Group AudienceSnapshot, group/public DeliveryEvidenceRef, production ModelGateway, ActionPlane, and the BotDelivery application remain future or NOT_ACTIVE; Issues #64, #66, #69, and #70 later activate the frozen OpenAPI, generated TypeScript SDK, private/direct OpenCitation, and deterministic private model-egress carriers. Issue #65 activates only one opaque digest-only model or channel EgressGrant after final Package policy, exact atomic PostgreSQL redemption and restricted audit, nominal BotDelivery inputs, and deterministic network-free ModelGateway or Sender-preflight spies under EGRESS-011. PROP-EGRESS-011, PG-EGRESS-011, and RUNTIME-EGRESS-011 prove exact Package/Organization/purpose/audience/epoch/hop/profile/lifetime binding and zero additional bytes on replay. Real model/provider calls, a real Sender or channel write, ActionTicket effects, group AudienceSnapshot revalidation, and the BotDelivery application process remain future or NOT_ACTIVE; Issue #64 later activates the generated SDK consumer and Issue #70 later activates the deterministic private TypeScript ModelGateway. Issue #66 activates the frozen public POST /v0/resolve OpenAPI carrier under TRANSPORT-UNTRUSTED-008. OPENAPI-CONTRACT-066, OPENAPI-BREAKING-066, HTTP-V0-066, and PG-RUNTIME-RELEASE-066 prove one public closed operation, deterministic immutable snapshot and breaking-change refusal, a hidden v1 bridge through the same handler and sealed Runtime path, and exact read-only observation of the active Learning-promoted release with fail-closed missing-release behavior before content work. A production BotDelivery caller, Continue redemption, MCP, group AudienceSnapshot, and external effects remain future or NOT_ACTIVE; Issues #64 and #69 later activate the generated TypeScript SDK and private/direct OpenCitation redemption through this frozen operation. Issue #64 activates only the packaged generated TypeScript POST /v0/resolve client under TRANSPORT-UNTRUSTED-008. SDK-CONTRACT-064 and SDK-LIVE-FILE-064 prove deterministic pinned generation, strict closed types, a narrow export map and metadata-only facade, installable tarball consumption, and one real PostgreSQL/File-backed Acquire through CandidateRef, AuthorizationKernel, AuthorizedProjection, ContextPackage, and opaque model egress grant. Issue #69 later extends SDK-LIVE-FILE-064 with a successful private/direct File OpenCitation through a second request-bound DeliveryEvidenceRef; Issue #70 extends the installed SDK fixture into deterministic Package-bound model generation; generated Continue remains generic unavailable. External package publication, production provider access, MCP, group AudienceSnapshot, real Continue redemption, and external effects remain future or NOT_ACTIVE. Issue #67 activates only private ActionPlane.prepare for create-placeholder, finalize-reply, and private-follow-up operation-specific tickets under ACTION-SEPARATION-014. PG-ACTION-PREPARE-067 proves exact current delivery, Organization, destination, audience, source, payload, approval, epoch, lifetime, and idempotency binding under a dedicated non-owner PostgreSQL role with digest-only FORCE-RLS persistence and zero effects. Issue #68 activates private ActionPlane.perform only through a deterministic Sender twin. PG-ACTION-PERFORM-068 proves one pre-Sender current-authority validation, one provider-attempt identity, immutable receipt replay, zero-effect ticket/payload mutation and stale-audience refusal, same-label cross-Organization isolation, and monotonic applied/rejected reconciliation including crash interleavings. Real provider or channel network effects, group AudienceSnapshot, compensation/delete, production BotDelivery orchestration, and the full ACCEPT-012 pass remain future or NOT_ACTIVE. The canonical set is IDs 001 through 012, 014, 015, and 019: CACHE-SCOPE-013 remains a preregistered conditional extension; AUDIENCE-016 is absorbed by SCOPE-INTERSECTION-004 and EGRESS-011; ACL-PROOF-017 is absorbed by INDEX-NOT-AUTHORITY-005 and REVOCATION-006; DELIVERY-EVIDENCE-018 is absorbed by TRANSPORT-UNTRUSTED-008. ACCEPT-001 through ACCEPT-012 follow ADR-0019's category order. Protected-asset references A-01 through A-08 refer, in order, to the eight bullets in the threat model's Protected assets section. Every expectedEvidence value below is a stable planned case identifier, not a claim that the case ran or passed; only an exact activation record upgrades named evidence, while fixture carrier status and the explicit M0 oracle preserve every other accepted-versus-active distinction. Issue #69 activates private/direct File CitationOpenRef issuance and OpenCitation under CITATION-AUTH-010: digest-only multi-use locators reveal only prior Package/Evidence and Fragment location lineage, every open obtains a current UserActor and trusted delivery context then traverses CandidateRef, AuthorizationKernel, AuthorizedProjection, a replacement ContextPackage, EgressGrant, ContextRun, and restricted DecisionAudit. PG-CITATION-AUTH-010, RUNTIME-CITATION-AUTH-010, and SDK-LIVE-FILE-064 prove A/B/A reauthorization, non-consumption on denial, database-clock expiry, cross-kind and cross-Organization opacity, and the generated SDK carrier. Group/public AudienceSnapshot, non-File providers, raw source URL locators, and Continue remain future or NOT_ACTIVE. Issue #70 activates only the private deterministic TypeScript ModelGateway under EGRESS-011. TS-MODEL-EGRESS-070, SDK-MODEL-EGRESS-070, and PG-MODEL-EGRESS-070 prove one current Package, exact grant redemption, closed provider input, bounded Package-subset citations, replay zero bytes, and digest-only retained audit. Real providers, streaming, group AudienceSnapshot, model-authored ActionPlane authority, and external effects remain future or NOT_ACTIVE. Issue #71 activates the complete private File-backed deterministic-twin BotDelivery carrier under TRANSPORT-UNTRUSTED-008, CITATION-AUTH-010, EGRESS-011, and ACTION-SEPARATION-014. TS-PRIVATE-BOT-FLOW-071, SDK-PRIVATE-BOT-FLOW-071, and PG-PRIVATE-BOT-FLOW-071 prove the independent TypeScript process and import boundary, exact verified private event binding, opaque DeliveryEvidenceRef transport, installed generated-SDK HTTP resolve, sealed File Package path, controlled model generation, distinct placeholder/final ActionPlane effects, digest-only DeliveryReceipt audit, citation reopening, and the composed wrong-binding oracles. The historical FIXTURE-ACCEPT-012 continues to prove the M0 unavailable cross-capability baseline separately. Live Feishu/model/Sender network carriers, group/public delivery, compensation/delete, Continue, and MCP remain NOT_ACTIVE. Issue #81 activates deterministic shallow File readChanges and whole-page acknowledgement under WORKER-LEASE-007. PG-FILE-CHANGE-ACTIVATE-081, PG-FILE-CHANGE-PAGE-081, and PG-FILE-CHANGE-DENY-081 prove immutable v3 activation, provider-authenticated content-free paging, exact replay, predecessor ordering, post-commit cursor issuance, FORCE-RLS tenant isolation, source/version invalidation, and zero implicit job or publication effect. Automatic scheduling, deletion execution, recursive discovery, full resync, and Runtime authority from cursor or checkpoint metadata remain NOT_ACTIVE. Issue #83 activates only explicit accepted File page scheduling through the existing file.import acquisition, WorkerLease, and publication path under WORKER-LEASE-007. PG-FILE-CHANGE-SCHEDULE-083, PG-FILE-CHANGE-SCHEDULE-DENY-083, and PG-FILE-CHANGE-SUPERSESSION-083 prove whole-page atomicity, exact job replay, explicit current FileImportAudience and receiver validation, immutable raw-byte observation lineage, pre-compiler drift refusal, and scan-epoch fences before content read and visible publication with zero stale publication effect. Autonomous polling, implicit audience inheritance, deletion execution, automatic retry/reclaim, dead-letter handling, full resync, and Runtime authority from provider checkpoints remain NOT_ACTIVE. Issue #85 activates bounded File delete observations only under WORKER-LEASE-007. PG-FILE-DELETE-DETECT-085, PG-FILE-DELETE-PAGE-085, and PG-FILE-DELETE-NO-EFFECT-085 prove exact latest-complete same-SourceVersion baseline binding, stable shallow diff, immutable canonical persistence/replay, forged/incomplete/stale/cross-Organization refusal, mixed-page scheduling refusal, and zero tombstone, Policy Epoch, cleanup, watermark, or generated-SDK Runtime visibility effect. At that activation, deletion execution remains NOT_ACTIVE; autonomous polling, retry/reclaim/dead-letter, full resync, recursive scan, and Runtime authority from baseline or delete metadata remain NOT_ACTIVE. Issue #87 later activates only exact trusted current File delete execution through the existing #28 tombstone authority under REVOCATION-006. PG-FILE-DELETE-EXECUTE-087, PG-FILE-DELETE-REPLAY-087, and HTTP-FILE-DELETE-INVISIBLE-087 prove current complete-scan revalidation, server-derived effect identity, atomic tombstone/epoch/cleanup/binding, exact replay, mismatch rollback, and immediate sealed-Runtime invisibility. Issue #89 later activates only the exact upsert projection of one current mixed v4 page under WORKER-LEASE-007. PG-FILE-MIXED-UPSERT-SCHEDULE-089, PG-FILE-MIXED-UPSERT-REPLAY-089, and HTTP-FILE-MIXED-UPSERT-NO-DELETE-089 prove complete-page validation, gapped original-ordinal job binding, exact replay, partial-lineage refusal, and zero delete/tombstone authority through the generated SDK. Autonomous scheduling, automatic upsert/delete ordering, batch deletion execution, Provider deletion authority, physical cleanup, restore/recreate, retry/reclaim/dead-letter, full resync, and Runtime authority from observation, page, checkpoint, or execution metadata remain NOT_ACTIVE. Issue #91 activates only scheduler-owned first-attempt File dispatch through the existing exact WorkerLease. Issue #93 extends that carrier with database-timed exponential backoff and at most three automatic higher-generation reclaims through the existing durable-boundary recovery path. Exhausted attempts remain untouched; dead-letter handling, operator remediation, provider polling, and delete ordering remain NOT_ACTIVE." + "reconciliation": "Issue #2 fixes the product and testing decisions, issue #5 requires exactly fifteen release invariants and twelve canonical acceptance fixtures, and ADR-0019 resolves the later nineteen-label prose expansion without weakening any safeguard. Issue #15 activates only Organization-level next-request resolve(Acquire) revocation evidence under REVOCATION-006: at that activation, PG-REVOCATION-006, RUN-006, and CACHE-002 are active while BLOB-002 and Continue, citation, Policy-Epoch-bound WorkerLease, production ContextAccessTicket/ActionTicket, audit, outbox, cleanup, finer-epoch, UI, and external-admin carriers remain future or NOT_ACTIVE; later issue records are authoritative for subsequently activated carriers. Issue #16 activates only the M0 refusal gate for unavailable Continue, profile-disabled OpenCitation, and server-owned unavailable Acquire plans: at that activation its real continuation, profile-enabled citation, federated/source-native, and File carriers remain future, while its Runtime and HTTP refusal surfaces prove generic outcomes before content I/O; Issue #69 later activates the private/direct File profile-enabled citation carrier. Issue #17 activates only the signed one-shot persistent no-op durable-job WorkerLease subcarrier under WORKER-LEASE-007. It binds one exact worker audience but no end-user delivery audience or Policy Epoch, and proves only LEASE-SIGNING-017, PG-WORKER-LEASE-NOOP-017, and WORKER-LEASE-REPLAY-007; Source, Resource, Revision, Policy Epoch, end-user delivery audience, idempotency, generation, business mutation, outbox, File publication, and the full ACCEPT-008 matrix remain deferred or NOT_ACTIVE. Issue #18 activates only distinct signed synthetic ContextAccessTicket Provider-read and ActionTicket no-op channel-action subcarriers under ACTION-SEPARATION-014, with current Organization-v0 Policy Epoch validation. TICKET-AUDIENCE-018 and PG-TICKET-EPOCH-018 do not activate production ContextProvider integration, ContextRuntime ticket integration, BotDelivery, full M2 ActionPlane.prepare/perform, a real Sender or external effect, payload/destination/approval/idempotency binding, durable one-shot/replay/reconciliation, or full ACCEPT-012 PASS; those remain future or NOT_ACTIVE. Issue #19 activates only the current Acquire authorized-only ContextRun and restricted delivered-empty DecisionAudit subcarrier under TRACE-REDACTION-012. DIGEST-019, RUN-LINEAGE-019, AUTHORIZED-RUN-019, and PG-TRACE-REDACTION-012 prove deterministic Package and Organization-bound query digests, retained-UserActor-transaction persistence, decisionRef resolution, redaction, and short-lived exact-Organization operator ticket reads with no application-role table access; the supported reader commits deletion before returning, while a direct caller rollback is not claimed as durable exactly-once redemption. Raw query retention, full ContextPackage body retention, unauthenticated transport failures as ContextRuns, cross-Organization analytics, and general observability redaction remain NOT_ACTIVE. Issue #48 activates only the current ACCEPT-002 authenticated HTTP Acquire Membership field-projection carrier under SCOPE-INTERSECTION-004, INDEX-NOT-AUTHORITY-005, and TRACE-REDACTION-012. PROP-FIELD-PROJECTION-048, PG-FIELD-PROJECTION-048, and HTTP-ACCEPT-002-048 bind one current Membership/version field ceiling to same-transaction FORCE-RLS reduction, the sealed AuthorizationKernel, AuthorizedProjection and Evidence integrity, and authorized-only ContextRun/audit persistence. General permission DSLs, caller-authored projection lists, CandidateRef or index field authority, production Provider/source-native ACL negotiation, Supply publication, File/Base field ACL, typed fields, Continue, and Issue #20 runner substitution remain future or NOT_ACTIVE; Issue #69 later activates private/direct File OpenCitation through the same field-projection gates. Issue #63 activates only the digest-only private authenticated HTTP Acquire DeliveryEvidenceRef carrier under TRANSPORT-UNTRUSTED-008. PROP-DELIVERY-EVIDENCE-063, PG-DELIVERY-EVIDENCE-063, HTTP-DELIVERY-EVIDENCE-063, and FILE-DELIVERY-EVIDENCE-063 prove exact service/request/Organization/asker/Membership-version/destination/consumer/purpose/audience/epoch/lifetime binding, stable identical retry identity, role isolation, expiry cleanup, pre-content generic rejection, and one File-backed sealed Runtime delivery. Group AudienceSnapshot, group/public DeliveryEvidenceRef, production ModelGateway, ActionPlane, and the BotDelivery application remain future or NOT_ACTIVE; Issues #64, #66, #69, and #70 later activate the frozen OpenAPI, generated TypeScript SDK, private/direct OpenCitation, and deterministic private model-egress carriers. Issue #65 activates only one opaque digest-only model or channel EgressGrant after final Package policy, exact atomic PostgreSQL redemption and restricted audit, nominal BotDelivery inputs, and deterministic network-free ModelGateway or Sender-preflight spies under EGRESS-011. PROP-EGRESS-011, PG-EGRESS-011, and RUNTIME-EGRESS-011 prove exact Package/Organization/purpose/audience/epoch/hop/profile/lifetime binding and zero additional bytes on replay. Real model/provider calls, a real Sender or channel write, ActionTicket effects, group AudienceSnapshot revalidation, and the BotDelivery application process remain future or NOT_ACTIVE; Issue #64 later activates the generated SDK consumer and Issue #70 later activates the deterministic private TypeScript ModelGateway. Issue #66 activates the frozen public POST /v0/resolve OpenAPI carrier under TRANSPORT-UNTRUSTED-008. OPENAPI-CONTRACT-066, OPENAPI-BREAKING-066, HTTP-V0-066, and PG-RUNTIME-RELEASE-066 prove one public closed operation, deterministic immutable snapshot and breaking-change refusal, a hidden v1 bridge through the same handler and sealed Runtime path, and exact read-only observation of the active Learning-promoted release with fail-closed missing-release behavior before content work. A production BotDelivery caller, Continue redemption, MCP, group AudienceSnapshot, and external effects remain future or NOT_ACTIVE; Issues #64 and #69 later activate the generated TypeScript SDK and private/direct OpenCitation redemption through this frozen operation. Issue #64 activates only the packaged generated TypeScript POST /v0/resolve client under TRANSPORT-UNTRUSTED-008. SDK-CONTRACT-064 and SDK-LIVE-FILE-064 prove deterministic pinned generation, strict closed types, a narrow export map and metadata-only facade, installable tarball consumption, and one real PostgreSQL/File-backed Acquire through CandidateRef, AuthorizationKernel, AuthorizedProjection, ContextPackage, and opaque model egress grant. Issue #69 later extends SDK-LIVE-FILE-064 with a successful private/direct File OpenCitation through a second request-bound DeliveryEvidenceRef; Issue #70 extends the installed SDK fixture into deterministic Package-bound model generation; generated Continue remains generic unavailable. External package publication, production provider access, MCP, group AudienceSnapshot, real Continue redemption, and external effects remain future or NOT_ACTIVE. Issue #67 activates only private ActionPlane.prepare for create-placeholder, finalize-reply, and private-follow-up operation-specific tickets under ACTION-SEPARATION-014. PG-ACTION-PREPARE-067 proves exact current delivery, Organization, destination, audience, source, payload, approval, epoch, lifetime, and idempotency binding under a dedicated non-owner PostgreSQL role with digest-only FORCE-RLS persistence and zero effects. Issue #68 activates private ActionPlane.perform only through a deterministic Sender twin. PG-ACTION-PERFORM-068 proves one pre-Sender current-authority validation, one provider-attempt identity, immutable receipt replay, zero-effect ticket/payload mutation and stale-audience refusal, same-label cross-Organization isolation, and monotonic applied/rejected reconciliation including crash interleavings. Real provider or channel network effects, group AudienceSnapshot, compensation/delete, production BotDelivery orchestration, and the full ACCEPT-012 pass remain future or NOT_ACTIVE. The canonical set is IDs 001 through 012, 014, 015, and 019: CACHE-SCOPE-013 remains a preregistered conditional extension; AUDIENCE-016 is absorbed by SCOPE-INTERSECTION-004 and EGRESS-011; ACL-PROOF-017 is absorbed by INDEX-NOT-AUTHORITY-005 and REVOCATION-006; DELIVERY-EVIDENCE-018 is absorbed by TRANSPORT-UNTRUSTED-008. ACCEPT-001 through ACCEPT-012 follow ADR-0019's category order. Protected-asset references A-01 through A-08 refer, in order, to the eight bullets in the threat model's Protected assets section. Every expectedEvidence value below is a stable planned case identifier, not a claim that the case ran or passed; only an exact activation record upgrades named evidence, while fixture carrier status and the explicit M0 oracle preserve every other accepted-versus-active distinction. Issue #69 activates private/direct File CitationOpenRef issuance and OpenCitation under CITATION-AUTH-010: digest-only multi-use locators reveal only prior Package/Evidence and Fragment location lineage, every open obtains a current UserActor and trusted delivery context then traverses CandidateRef, AuthorizationKernel, AuthorizedProjection, a replacement ContextPackage, EgressGrant, ContextRun, and restricted DecisionAudit. PG-CITATION-AUTH-010, RUNTIME-CITATION-AUTH-010, and SDK-LIVE-FILE-064 prove A/B/A reauthorization, non-consumption on denial, database-clock expiry, cross-kind and cross-Organization opacity, and the generated SDK carrier. Group/public AudienceSnapshot, non-File providers, raw source URL locators, and Continue remain future or NOT_ACTIVE. Issue #70 activates only the private deterministic TypeScript ModelGateway under EGRESS-011. TS-MODEL-EGRESS-070, SDK-MODEL-EGRESS-070, and PG-MODEL-EGRESS-070 prove one current Package, exact grant redemption, closed provider input, bounded Package-subset citations, replay zero bytes, and digest-only retained audit. Real providers, streaming, group AudienceSnapshot, model-authored ActionPlane authority, and external effects remain future or NOT_ACTIVE. Issue #71 activates the complete private File-backed deterministic-twin BotDelivery carrier under TRANSPORT-UNTRUSTED-008, CITATION-AUTH-010, EGRESS-011, and ACTION-SEPARATION-014. TS-PRIVATE-BOT-FLOW-071, SDK-PRIVATE-BOT-FLOW-071, and PG-PRIVATE-BOT-FLOW-071 prove the independent TypeScript process and import boundary, exact verified private event binding, opaque DeliveryEvidenceRef transport, installed generated-SDK HTTP resolve, sealed File Package path, controlled model generation, distinct placeholder/final ActionPlane effects, digest-only DeliveryReceipt audit, citation reopening, and the composed wrong-binding oracles. The historical FIXTURE-ACCEPT-012 continues to prove the M0 unavailable cross-capability baseline separately. Live Feishu/model/Sender network carriers, group/public delivery, compensation/delete, Continue, and MCP remain NOT_ACTIVE. Issue #81 activates deterministic shallow File readChanges and whole-page acknowledgement under WORKER-LEASE-007. PG-FILE-CHANGE-ACTIVATE-081, PG-FILE-CHANGE-PAGE-081, and PG-FILE-CHANGE-DENY-081 prove immutable v3 activation, provider-authenticated content-free paging, exact replay, predecessor ordering, post-commit cursor issuance, FORCE-RLS tenant isolation, source/version invalidation, and zero implicit job or publication effect. Automatic scheduling, deletion execution, recursive discovery, full resync, and Runtime authority from cursor or checkpoint metadata remain NOT_ACTIVE. Issue #83 activates only explicit accepted File page scheduling through the existing file.import acquisition, WorkerLease, and publication path under WORKER-LEASE-007. PG-FILE-CHANGE-SCHEDULE-083, PG-FILE-CHANGE-SCHEDULE-DENY-083, and PG-FILE-CHANGE-SUPERSESSION-083 prove whole-page atomicity, exact job replay, explicit current FileImportAudience and receiver validation, immutable raw-byte observation lineage, pre-compiler drift refusal, and scan-epoch fences before content read and visible publication with zero stale publication effect. Autonomous polling, implicit audience inheritance, deletion execution, automatic retry/reclaim, dead-letter handling, full resync, and Runtime authority from provider checkpoints remain NOT_ACTIVE. Issue #85 activates bounded File delete observations only under WORKER-LEASE-007. PG-FILE-DELETE-DETECT-085, PG-FILE-DELETE-PAGE-085, and PG-FILE-DELETE-NO-EFFECT-085 prove exact latest-complete same-SourceVersion baseline binding, stable shallow diff, immutable canonical persistence/replay, forged/incomplete/stale/cross-Organization refusal, mixed-page scheduling refusal, and zero tombstone, Policy Epoch, cleanup, watermark, or generated-SDK Runtime visibility effect. At that activation, deletion execution remains NOT_ACTIVE; autonomous polling, retry/reclaim/dead-letter, full resync, recursive scan, and Runtime authority from baseline or delete metadata remain NOT_ACTIVE. Issue #87 later activates only exact trusted current File delete execution through the existing #28 tombstone authority under REVOCATION-006. PG-FILE-DELETE-EXECUTE-087, PG-FILE-DELETE-REPLAY-087, and HTTP-FILE-DELETE-INVISIBLE-087 prove current complete-scan revalidation, server-derived effect identity, atomic tombstone/epoch/cleanup/binding, exact replay, mismatch rollback, and immediate sealed-Runtime invisibility. Issue #89 later activates only the exact upsert projection of one current mixed v4 page under WORKER-LEASE-007. PG-FILE-MIXED-UPSERT-SCHEDULE-089, PG-FILE-MIXED-UPSERT-REPLAY-089, and HTTP-FILE-MIXED-UPSERT-NO-DELETE-089 prove complete-page validation, gapped original-ordinal job binding, exact replay, partial-lineage refusal, and zero delete/tombstone authority through the generated SDK. Autonomous scheduling, automatic upsert/delete ordering, batch deletion execution, Provider deletion authority, physical cleanup, restore/recreate, retry/reclaim/dead-letter, full resync, and Runtime authority from observation, page, checkpoint, or execution metadata remain NOT_ACTIVE. Issue #91 activates only scheduler-owned first-attempt File dispatch through the existing exact WorkerLease. Issue #93 extends that carrier with database-timed exponential backoff and at most three automatic higher-generation reclaims through the existing durable-boundary recovery path. Exhausted attempts remain untouched; dead-letter handling, operator remediation, provider polling, and delete ordering remain NOT_ACTIVE. Issue #102 activates only an explicit loopback single-Membership authentication composition and File pgvector Acquire carrier through the same current UserActor transaction and sealed Runtime. RUNTIME-DOGFOOD-AUTH-102, RUNTIME-DOGFOOD-CARRIER-102, and RUNTIME-DOGFOOD-EPOCH-102 prove generic secret or Membership refusal, exact EffectiveScope reduction before ANN LIMIT, deterministic network-free embedding with Release-bound model/input identity, and final Policy Epoch veto. Production authentication, a second human, network exposure beyond the maintainer machine, group/public audience, dogfood OpenCitation, Continue, hybrid retrieval, external query embedding, and non-File providers remain NOT_ACTIVE." }, "hardOracles": [ { @@ -1007,6 +1009,83 @@ "manual import or delete recovery", "Runtime authorization from Supply retry or lease state" ] + }, + { + "issueRef": "#102", + "invariantRef": "TRANSPORT-UNTRUSTED-008", + "carrier": "loopback-only single-Membership dogfood HTTP authentication", + "status": "active_fail_closed", + "policyEpochScope": "organization-v0", + "controlBoundary": "explicit local environment opt-in -> constant-time bearer verification -> one fixed configured identity -> current UserActor Membership transaction", + "testEvidence": [ + { + "id": "RUNTIME-DOGFOOD-AUTH-102", + "surface": "tests/integration/test_dogfood_runtime_activation.py::test_dogfood_secret_and_membership_fail_closed_without_secret_retention", + "oracle": "Absent, wrong, or truncated secrets and revoked or expired configured Membership authority return the same generic authentication failure; the secret is absent from responses, logs, ContextRun, and DecisionAudit." + } + ], + "deferredEvidence": [ + "production identity-provider authentication and key lifecycle", + "second-human and multi-Membership authentication", + "non-loopback network exposure and remote deployment" + ], + "futureCarriers": [ + "production authentication composition", + "multi-user identity and Membership selection", + "authenticated remote network ingress" + ], + "notActive": [ + "production authentication", + "a second human identity", + "network exposure beyond the maintainer machine", + "group or public audience", + "Continue", + "dogfood OpenCitation", + "hybrid retrieval", + "external query embedding", + "non-File providers" + ] + }, + { + "issueRef": "#102", + "invariantRef": "INDEX-NOT-AUTHORITY-005", + "carrier": "loopback-only File pgvector dogfood Acquire delivery", + "status": "active_fail_closed", + "policyEpochScope": "organization-v0", + "controlBoundary": "current UserActor transaction -> database-derived exact EffectiveScope -> pre-LIMIT pgvector scope reduction -> CandidateRef -> sealed AuthorizationKernel -> AuthorizedProjection -> ContextPackage", + "testEvidence": [ + { + "id": "RUNTIME-DOGFOOD-CARRIER-102", + "surface": "tests/integration/test_dogfood_runtime_activation.py::test_dogfood_served_composition_delivers_release_scoped_file_evidence_before_limit", + "oracle": "The configured maintainer receives one real File Evidence block through the served POST /v0/resolve carrier even when more than the ANN limit of strictly closer RLS-visible distractors are excluded by EffectiveScope; the deterministic twin records zero external provider usage." + }, + { + "id": "RUNTIME-DOGFOOD-EPOCH-102", + "surface": "tests/integration/test_dogfood_runtime_activation.py::test_dogfood_mid_resolve_policy_epoch_change_vetoes_stale_evidence", + "oracle": "An access revocation that advances Policy Epoch after authorization but before delivery yields an authorized empty package with zero stale Evidence, while the sealed Kernel remains the final authority." + } + ], + "deferredEvidence": [ + "external-provider embedding metering and profile identity", + "multi-source or hybrid discovery and non-File source-native ACL semantics", + "group, public, Continue, and remote deployment carriers" + ], + "futureCarriers": [ + "metered external query embedding", + "hybrid and non-File retrieval", + "group, public, and Continue delivery" + ], + "notActive": [ + "production authentication", + "a second human identity", + "network exposure beyond the maintainer machine", + "group or public audience", + "Continue", + "dogfood OpenCitation", + "hybrid retrieval", + "external query embedding", + "non-File providers" + ] } ], "invariants": [ diff --git a/pyproject.toml b/pyproject.toml index 36eff2a5..1065cc93 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -33,6 +33,7 @@ dev = [ [project.scripts] context-engine-api = "applications.api:main" +context-engine-dogfood-seed = "applications.dogfood:main" context-engine-worker = "applications.worker:main" [tool.hatch.build.targets.wheel] diff --git a/scripts/validate_security_catalog.py b/scripts/validate_security_catalog.py index 42b31a68..54a10043 100644 --- a/scripts/validate_security_catalog.py +++ b/scripts/validate_security_catalog.py @@ -2158,6 +2158,117 @@ ], } +CANONICAL_DOGFOOD_AUTHENTICATION_ACTIVATION: dict[str, object] = { + "issueRef": "#102", + "invariantRef": "TRANSPORT-UNTRUSTED-008", + "carrier": "loopback-only single-Membership dogfood HTTP authentication", + "status": "active_fail_closed", + "policyEpochScope": "organization-v0", + "controlBoundary": ( + "explicit local environment opt-in -> constant-time bearer verification -> " + "one fixed configured identity -> current UserActor Membership transaction" + ), + "testEvidence": [ + { + "id": "RUNTIME-DOGFOOD-AUTH-102", + "surface": ( + "tests/integration/test_dogfood_runtime_activation.py::" + "test_dogfood_secret_and_membership_fail_closed_without_secret_retention" + ), + "oracle": ( + "Absent, wrong, or truncated secrets and revoked or expired configured " + "Membership authority return the same generic authentication failure; " + "the secret is absent from responses, logs, ContextRun, and " + "DecisionAudit." + ), + }, + ], + "deferredEvidence": [ + "production identity-provider authentication and key lifecycle", + "second-human and multi-Membership authentication", + "non-loopback network exposure and remote deployment", + ], + "futureCarriers": [ + "production authentication composition", + "multi-user identity and Membership selection", + "authenticated remote network ingress", + ], + "notActive": [ + "production authentication", + "a second human identity", + "network exposure beyond the maintainer machine", + "group or public audience", + "Continue", + "dogfood OpenCitation", + "hybrid retrieval", + "external query embedding", + "non-File providers", + ], +} + +CANONICAL_DOGFOOD_RUNTIME_ACTIVATION: dict[str, object] = { + "issueRef": "#102", + "invariantRef": "INDEX-NOT-AUTHORITY-005", + "carrier": "loopback-only File pgvector dogfood Acquire delivery", + "status": "active_fail_closed", + "policyEpochScope": "organization-v0", + "controlBoundary": ( + "current UserActor transaction -> database-derived exact EffectiveScope -> " + "pre-LIMIT pgvector scope reduction -> CandidateRef -> sealed " + "AuthorizationKernel -> AuthorizedProjection -> ContextPackage" + ), + "testEvidence": [ + { + "id": "RUNTIME-DOGFOOD-CARRIER-102", + "surface": ( + "tests/integration/test_dogfood_runtime_activation.py::" + "test_dogfood_served_composition_delivers_release_scoped_file_evidence_before_limit" + ), + "oracle": ( + "The configured maintainer receives one real File Evidence block " + "through the served POST /v0/resolve carrier even when more than " + "the ANN limit of strictly closer RLS-visible distractors are " + "excluded by EffectiveScope; " + "the deterministic twin records zero external provider usage." + ), + }, + { + "id": "RUNTIME-DOGFOOD-EPOCH-102", + "surface": ( + "tests/integration/test_dogfood_runtime_activation.py::" + "test_dogfood_mid_resolve_policy_epoch_change_vetoes_stale_evidence" + ), + "oracle": ( + "An access revocation that advances Policy Epoch after authorization " + "but " + "before delivery yields an authorized empty package with zero stale " + "Evidence, while the sealed Kernel remains the final authority." + ), + }, + ], + "deferredEvidence": [ + "external-provider embedding metering and profile identity", + "multi-source or hybrid discovery and non-File source-native ACL semantics", + "group, public, Continue, and remote deployment carriers", + ], + "futureCarriers": [ + "metered external query embedding", + "hybrid and non-File retrieval", + "group, public, and Continue delivery", + ], + "notActive": [ + "production authentication", + "a second human identity", + "network exposure beyond the maintainer machine", + "group or public audience", + "Continue", + "dogfood OpenCitation", + "hybrid retrieval", + "external query embedding", + "non-File providers", + ], +} + CANONICAL_ACTIVATIONS: list[dict[str, object]] = [ CANONICAL_REVOCATION_ACTIVATION, CANONICAL_UNAVAILABLE_CAPABILITY_ACTIVATION, @@ -2181,6 +2292,8 @@ CANONICAL_FILE_MIXED_UPSERT_SCHEDULING_ACTIVATION, CANONICAL_FILE_DISPATCH_ACTIVATION, CANONICAL_FILE_RECLAIM_ACTIVATION, + CANONICAL_DOGFOOD_AUTHENTICATION_ACTIVATION, + CANONICAL_DOGFOOD_RUNTIME_ACTIVATION, ] CANONICAL_ACTIVATION_ISSUE_LIST = ", ".join( f"Issue {activation['issueRef']}" for activation in CANONICAL_ACTIVATIONS diff --git a/tests/catalog/test_validate_security_catalog.py b/tests/catalog/test_validate_security_catalog.py index 81e4f19d..23c6e826 100644 --- a/tests/catalog/test_validate_security_catalog.py +++ b/tests/catalog/test_validate_security_catalog.py @@ -27,6 +27,8 @@ CANONICAL_ACTIVATIONS, CANONICAL_CITATION_OPEN_ACTIVATION, CANONICAL_CONTEXT_RUN_ACTIVATION, + CANONICAL_DOGFOOD_AUTHENTICATION_ACTIVATION, + CANONICAL_DOGFOOD_RUNTIME_ACTIVATION, CANONICAL_EGRESS_GRANT_ACTIVATION, CANONICAL_FAIL_CLOSED_OUTCOMES, CANONICAL_FIELD_PROJECTION_ACTIVATION, @@ -591,6 +593,8 @@ def make_catalog() -> dict[str, object]: copy.deepcopy(CANONICAL_FILE_MIXED_UPSERT_SCHEDULING_ACTIVATION), copy.deepcopy(CANONICAL_FILE_DISPATCH_ACTIVATION), copy.deepcopy(CANONICAL_FILE_RECLAIM_ACTIVATION), + copy.deepcopy(CANONICAL_DOGFOOD_AUTHENTICATION_ACTIVATION), + copy.deepcopy(CANONICAL_DOGFOOD_RUNTIME_ACTIVATION), ], "invariants": invariants, "fixtures": fixtures, @@ -711,6 +715,12 @@ def make_schema() -> dict[str, object]: }, {"const": copy.deepcopy(CANONICAL_FILE_DISPATCH_ACTIVATION)}, {"const": copy.deepcopy(CANONICAL_FILE_RECLAIM_ACTIVATION)}, + { + "const": copy.deepcopy( + CANONICAL_DOGFOOD_AUTHENTICATION_ACTIVATION + ) + }, + {"const": copy.deepcopy(CANONICAL_DOGFOOD_RUNTIME_ACTIVATION)}, ], "items": False, }, @@ -1112,7 +1122,7 @@ def test_issue_71_activates_private_accept_012_without_rewriting_m0_history( assert isinstance(upgrade_trigger, str) self.assertIn("Issue #71 activates", upgrade_trigger) self.assertEqual( - object_list_at(catalog, "activations")[-8], + object_list_at(catalog, "activations")[-10], CANONICAL_PRIVATE_BOT_DELIVERY_ACTIVATION, ) @@ -1370,7 +1380,7 @@ def test_issue_93_file_reclaim_activation_is_independently_frozen(self) -> None: prefix_items = schema_activations["prefixItems"] assert isinstance(prefix_items, list) schema_activation = object_at( - cast(dict[str, object], prefix_items[-1]), "const" + cast(dict[str, object], prefix_items[-3]), "const" ) expected_boundary = ( @@ -1513,7 +1523,7 @@ def test_issue_70_model_egress_activation_stops_before_real_provider(self) -> No def test_issue_71_private_bot_activation_stops_before_live_providers(self) -> None: catalog = make_catalog() - activation = object_list_at(catalog, "activations")[-8] + activation = object_list_at(catalog, "activations")[-10] self.assertEqual(activation, CANONICAL_PRIVATE_BOT_DELIVERY_ACTIVATION) self.assertEqual(activation["invariantRef"], "ACTION-SEPARATION-014") @@ -1537,7 +1547,7 @@ def test_issue_71_private_bot_activation_stops_before_live_providers(self) -> No def test_issue_81_file_change_activation_stops_before_scheduling(self) -> None: catalog = make_catalog() - activation = object_list_at(catalog, "activations")[-7] + activation = object_list_at(catalog, "activations")[-9] self.assertEqual(activation, CANONICAL_FILE_CHANGE_FEED_ACTIVATION) self.assertEqual(activation["invariantRef"], "WORKER-LEASE-007") @@ -1558,7 +1568,7 @@ def test_issue_81_file_change_activation_stops_before_scheduling(self) -> None: def test_issue_83_file_change_scheduling_stays_explicit(self) -> None: catalog = make_catalog() - activation = object_list_at(catalog, "activations")[-6] + activation = object_list_at(catalog, "activations")[-8] self.assertEqual( activation, @@ -1580,7 +1590,7 @@ def test_issue_83_file_change_scheduling_stays_explicit(self) -> None: def test_issue_85_file_delete_observation_has_no_execution_authority(self) -> None: catalog = make_catalog() - activation = object_list_at(catalog, "activations")[-5] + activation = object_list_at(catalog, "activations")[-7] self.assertEqual( activation, @@ -1604,7 +1614,7 @@ def test_issue_85_file_delete_observation_has_no_execution_authority(self) -> No def test_issue_87_executes_only_current_exact_file_deletes(self) -> None: catalog = make_catalog() - activation = object_list_at(catalog, "activations")[-4] + activation = object_list_at(catalog, "activations")[-6] self.assertEqual( activation, @@ -1625,7 +1635,7 @@ def test_issue_87_executes_only_current_exact_file_deletes(self) -> None: def test_issue_89_schedules_only_the_mixed_page_upsert_projection(self) -> None: catalog = make_catalog() - activation = object_list_at(catalog, "activations")[-3] + activation = object_list_at(catalog, "activations")[-5] self.assertEqual( activation, @@ -2066,7 +2076,7 @@ def test_tracked_catalog_freezes_issue_19_authority_and_bounded_scope( self.assertEqual(catalog["catalogVersion"], "1.3.0") self.assertEqual( - issue_refs[-22:], + issue_refs[-23:], [ "#15", "#16", @@ -2090,6 +2100,7 @@ def test_tracked_catalog_freezes_issue_19_authority_and_bounded_scope( "#89", "#91", "#93", + "#102", ], ) self.assertIn( diff --git a/tests/integration/test_dogfood_runtime_activation.py b/tests/integration/test_dogfood_runtime_activation.py new file mode 100644 index 00000000..284042c9 --- /dev/null +++ b/tests/integration/test_dogfood_runtime_activation.py @@ -0,0 +1,675 @@ +from __future__ import annotations + +import logging +import subprocess +from concurrent.futures import ThreadPoolExecutor +from pathlib import Path +from threading import Event +from typing import Any, cast +from uuid import UUID, uuid4 + +import pytest +from fastapi.testclient import TestClient +from sqlalchemy import Engine, text + +import engine.persistence.membership_context as membership_context_module +from adapters.embeddings import DeterministicEmbeddingTwin +from adapters.http.dogfood import ( + DOGFOOD_AGENT_ENV, + DOGFOOD_APPLICATION_ENV, + DOGFOOD_BINDING_ENV, + DOGFOOD_COMPOSITION_ENV, + DOGFOOD_COMPOSITION_VALUE, + DOGFOOD_EMBEDDING_PROVIDER_ENV, + DOGFOOD_EMBEDDING_PROVIDER_VALUE, + DOGFOOD_MEMBERSHIP_ENV, + DOGFOOD_MEMBERSHIP_VERSION_ENV, + DOGFOOD_ORGANIZATION_ENV, + DOGFOOD_PRINCIPAL_ENV, + DOGFOOD_SECRET_ENV, + DOGFOOD_USER_ENV, + DogfoodConfiguration, + DogfoodConfigurationUnavailable, + create_dogfood_app, + create_served_app, +) +from adapters.pgvector import DEFAULT_VECTOR_CANDIDATE_LIMIT +from applications.api import main as api_main +from engine.persistence import ( + DatabaseConfiguration, + PostgreSQLAccessPolicyControl, + ResourceAccessRevocation, + create_database_engine, +) +from engine.runtime.evidence import CandidateRef +from engine.runtime.release_lineage import ( + DOGFOOD_VECTOR_INDEX_PROFILE_DIGEST_V1, + DOGFOOD_VECTOR_INDEX_PROFILE_REF_V1, + INDEX_PROFILE_DIGEST_V0, + INDEX_PROFILE_REF_V0, +) +from tests.integration.test_zz_file_revision_replacement import _scenario_user_id +from tests.support.file_imports import ( + FileImportScenario, + delete_file_import_scenario, + prepare_file_import_scenario, + run_file_import, +) +from tests.support.releases import ( + clear_test_runtime_release, + ensure_test_runtime_release, +) + +pytestmark = pytest.mark.integration +SECRET = "dogfood-secret-with-at-least-thirty-two-bytes" +TARGET_TEXT = "Dogfood delivery reaches the authorized target." +QUERY = "Which dogfood delivery target is authorized?" + + +def _configuration( + scenario: FileImportScenario, + user_id: UUID, +) -> DogfoodConfiguration: + return DogfoodConfiguration( + secret=SECRET, + organization_id=scenario.organization_id, + user_id=user_id, + membership_id=scenario.membership_id, + membership_version=1, + principal_ref="principal:file-reader", + agent_version_ref="agent:dogfood-local:v1", + application_ref="application:dogfood-local:v1", + authentication_binding_ref="binding:dogfood-local:v1", + embedding_provider=DOGFOOD_EMBEDDING_PROVIDER_VALUE, + ) + + +def _environment( + configuration: DogfoodConfiguration, + runtime_configuration: DatabaseConfiguration, +) -> dict[str, str]: + return { + DOGFOOD_COMPOSITION_ENV: DOGFOOD_COMPOSITION_VALUE, + DOGFOOD_SECRET_ENV: configuration.secret, + DOGFOOD_ORGANIZATION_ENV: str(configuration.organization_id), + DOGFOOD_USER_ENV: str(configuration.user_id), + DOGFOOD_MEMBERSHIP_ENV: str(configuration.membership_id), + DOGFOOD_MEMBERSHIP_VERSION_ENV: str(configuration.membership_version), + DOGFOOD_PRINCIPAL_ENV: configuration.principal_ref, + DOGFOOD_AGENT_ENV: configuration.agent_version_ref, + DOGFOOD_APPLICATION_ENV: configuration.application_ref, + DOGFOOD_BINDING_ENV: configuration.authentication_binding_ref, + DOGFOOD_EMBEDDING_PROVIDER_ENV: configuration.embedding_provider, + "CONTEXT_ENGINE_RUNTIME_ROLE": runtime_configuration.expected_role, + "CONTEXT_ENGINE_RUNTIME_DATABASE_URL": ( + runtime_configuration.url.render_as_string(hide_password=False) + ), + } + + +def _publish( + request: pytest.FixtureRequest, + tmp_path: Path, + migration_configuration: DatabaseConfiguration, + guarded_control_engine: Engine, + guarded_worker_engine: Engine, + *, + dogfood_index_profile: bool = True, +) -> tuple[FileImportScenario, UUID, CandidateRef]: + scenario = prepare_file_import_scenario( + tmp_path, + migration_configuration, + guarded_control_engine, + payload=f"# Dogfood\n\n{TARGET_TEXT}\n".encode(), + ) + request.addfinalizer( + lambda: _delete(migration_configuration, scenario.organization_id) + ) + assert scenario.token is not None + published = run_file_import( + scenario, + scenario.prepared, + scenario.token, + guarded_worker_engine, + ) + ensure_test_runtime_release( + scenario.organization_id, + active_revision_refs=(published.candidate_ref.revision_ref,), + index_profile_ref=( + DOGFOOD_VECTOR_INDEX_PROFILE_REF_V1 + if dogfood_index_profile + else INDEX_PROFILE_REF_V0 + ), + index_profile_digest=( + DOGFOOD_VECTOR_INDEX_PROFILE_DIGEST_V1 + if dogfood_index_profile + else INDEX_PROFILE_DIGEST_V0 + ), + ) + return ( + scenario, + _scenario_user_id(scenario, migration_configuration), + published.candidate_ref, + ) + + +def _delete( + migration_configuration: DatabaseConfiguration, + organization_id: UUID, +) -> None: + clear_test_runtime_release(organization_id) + delete_file_import_scenario(migration_configuration, organization_id) + + +def _add_policy_out_of_scope_distractors( + migration_configuration: DatabaseConfiguration, + scenario: FileImportScenario, +) -> None: + embedding = DeterministicEmbeddingTwin().embed((QUERY,))[0] + parameters = [ + { + "organization_id": scenario.organization_id, + "membership_id": scenario.membership_id, + "resource_ref": f"resource:dogfood-distractor:{ordinal:03d}", + "source_ref": str(scenario.source_ref.value), + "revision_id": uuid4(), + "fragment_ref": f"fragment:dogfood-distractor:{ordinal:03d}", + "embedding": "[" + ",".join(repr(value) for value in embedding) + "]", + } + for ordinal in range(DEFAULT_VECTOR_CANDIDATE_LIMIT + 4) + ] + engine = create_database_engine(migration_configuration) + try: + with engine.begin() as connection: + connection.execute(text("SET CONSTRAINTS ALL DEFERRED")) + connection.execute( + text( + """ + INSERT INTO context_resource ( + organization_id, resource_ref, source_ref, + active_revision_id, tombstoned + ) VALUES ( + :organization_id, :resource_ref, :source_ref, + :revision_id, false + ) + """ + ), + parameters, + ) + connection.execute( + text( + """ + INSERT INTO context_revision ( + organization_id, resource_ref, revision_id + ) VALUES ( + :organization_id, :resource_ref, :revision_id + ) + """ + ), + parameters, + ) + connection.execute( + text( + """ + INSERT INTO context_fragment ( + organization_id, resource_ref, revision_id, + fragment_ref, ordinal, content, projection_kind, + embedding + ) VALUES ( + :organization_id, :resource_ref, :revision_id, + :fragment_ref, 0, :fragment_ref, 'body', + CAST(:embedding AS vector) + ) + """ + ), + parameters, + ) + connection.execute( + text( + """ + INSERT INTO resource_access_policy ( + organization_id, resource_ref, principal_ref, + access_version, access_state, revoked_at + ) VALUES ( + :organization_id, :resource_ref, + 'principal:file-reader', 1, 'allowed', NULL + ) + """ + ), + parameters, + ) + connection.execute( + text( + """ + INSERT INTO membership_resource_field_right ( + organization_id, membership_id, membership_version, + resource_ref, field_ref + ) VALUES ( + :organization_id, :membership_id, 1, + :resource_ref, 'body' + ) + """ + ), + parameters, + ) + finally: + engine.dispose() + + +def _strictly_closer_distractor_count( + migration_configuration: DatabaseConfiguration, + scenario: FileImportScenario, + target: CandidateRef, +) -> int: + query_embedding = DeterministicEmbeddingTwin().embed((QUERY,))[0] + encoded_embedding = "[" + ",".join(repr(value) for value in query_embedding) + "]" + engine = create_database_engine(migration_configuration) + try: + with engine.connect() as connection: + return int( + connection.execute( + text( + """ + SELECT count(*) + FROM context_fragment AS distractor + WHERE distractor.organization_id = :organization_id + AND distractor.fragment_ref LIKE + 'fragment:dogfood-distractor:%' + AND distractor.embedding <=> CAST(:embedding AS vector) + < ( + SELECT target.embedding <=> + CAST(:embedding AS vector) + FROM context_fragment AS target + WHERE target.organization_id = :organization_id + AND target.resource_ref = :target_resource_ref + AND target.revision_id = :target_revision_id + AND target.fragment_ref = :target_fragment_ref + ) + """ + ), + { + "organization_id": scenario.organization_id, + "embedding": encoded_embedding, + "target_resource_ref": target.resource_ref, + "target_revision_id": UUID(target.revision_ref), + "target_fragment_ref": target.fragment_ref, + }, + ).scalar_one() + ) + finally: + engine.dispose() + + +def _resolve(client: TestClient, secret: str = SECRET) -> Any: + return client.post( + "/v0/resolve", + headers={ + "Authorization": f"Bearer {secret}", + "X-Context-Request-Id": f"dogfood-{uuid4()}", + }, + json={"kind": "acquire", "need": {"query": QUERY}}, + ) + + +@pytest.mark.security_evidence(id="RUNTIME-DOGFOOD-CARRIER-102", layer="runtime") +def test_dogfood_served_composition_delivers_release_scoped_file_evidence_before_limit( + request: pytest.FixtureRequest, + tmp_path: Path, + migration_configuration: DatabaseConfiguration, + runtime_configuration: DatabaseConfiguration, + guarded_control_engine: Engine, + guarded_worker_engine: Engine, + monkeypatch: pytest.MonkeyPatch, +) -> None: + scenario, user_id, target = _publish( + request, + tmp_path, + migration_configuration, + guarded_control_engine, + guarded_worker_engine, + ) + _add_policy_out_of_scope_distractors(migration_configuration, scenario) + assert _strictly_closer_distractor_count( + migration_configuration, + scenario, + target, + ) > DEFAULT_VECTOR_CANDIDATE_LIMIT + configuration = _configuration(scenario, user_id) + served: dict[str, object] = {} + for name, value in _environment(configuration, runtime_configuration).items(): + monkeypatch.setenv(name, value) + + def observe(app: object, **kwargs: object) -> None: + served["app"] = app + served.update(kwargs) + + monkeypatch.setattr("applications.api.uvicorn.run", observe) + api_main(["--host", "127.0.0.1", "--port", "9123"]) + client = TestClient(cast(Any, served["app"])) + + assert client.get("/health").json()["runtime_delivery"] == "ACTIVE" + assert served["host"] == "127.0.0.1" + response = _resolve(client) + + assert response.status_code == 200 + package = cast(dict[str, Any], response.json()["package"]) + assert [block["text"] for block in package["blocks"]] == [TARGET_TEXT] + assert len(package["evidence"]) == 1 + assert package["evidence"][0]["revisionRef"] == target.revision_ref + assert "dogfood-distractor" not in response.text + assert package["budgetUsage"] == { + "tokens": len(TARGET_TEXT.encode()), + "providerCalls": 0, + "costMicrounits": 0, + "elapsedMs": 0, + } + + +def test_dogfood_rejects_an_active_release_with_an_unbound_embedding_profile( + request: pytest.FixtureRequest, + tmp_path: Path, + migration_configuration: DatabaseConfiguration, + runtime_configuration: DatabaseConfiguration, + guarded_control_engine: Engine, + guarded_worker_engine: Engine, +) -> None: + scenario, user_id, _target = _publish( + request, + tmp_path, + migration_configuration, + guarded_control_engine, + guarded_worker_engine, + dogfood_index_profile=False, + ) + configuration = _configuration(scenario, user_id) + + with pytest.raises(DogfoodConfigurationUnavailable): + create_served_app( + _environment(configuration, runtime_configuration), + host="127.0.0.1", + ) + + +@pytest.mark.security_evidence(id="RUNTIME-DOGFOOD-AUTH-102", layer="runtime") +def test_dogfood_secret_and_membership_fail_closed_without_secret_retention( + request: pytest.FixtureRequest, + tmp_path: Path, + migration_configuration: DatabaseConfiguration, + runtime_configuration: DatabaseConfiguration, + guarded_control_engine: Engine, + guarded_worker_engine: Engine, + caplog: pytest.LogCaptureFixture, +) -> None: + scenario, user_id, _revision_ref = _publish( + request, + tmp_path, + migration_configuration, + guarded_control_engine, + guarded_worker_engine, + ) + configuration = _configuration(scenario, user_id) + client = TestClient( + create_dogfood_app( + configuration, + _environment(configuration, runtime_configuration), + host="127.0.0.1", + ) + ) + successful = _resolve(client) + assert successful.status_code == 200 + assert successful.json()["package"]["evidence"] + + with caplog.at_level(logging.DEBUG): + responses = ( + client.post( + "/v0/resolve", + headers={"X-Context-Request-Id": "dogfood-absent"}, + json={"kind": "acquire", "need": {"query": QUERY}}, + ), + _resolve(client, "wrong-dogfood-secret"), + _resolve(client, SECRET[:-1]), + ) + for response in responses: + assert response.status_code == 401 + assert response.json() == {"code": "authentication_failed"} + assert SECRET not in response.text + + engine = create_database_engine(migration_configuration) + try: + with engine.begin() as connection: + connection.execute( + text( + """ + UPDATE membership + SET status = 'revoked' + WHERE organization_id = :organization_id + AND membership_id = :membership_id + """ + ), + { + "organization_id": scenario.organization_id, + "membership_id": scenario.membership_id, + }, + ) + finally: + engine.dispose() + revoked = _resolve(client) + assert revoked.status_code == 401 + assert revoked.json() == {"code": "authentication_failed"} + engine = create_database_engine(migration_configuration) + try: + with engine.begin() as connection: + connection.execute( + text( + """ + UPDATE membership + SET status = 'active', + valid_until = clock_timestamp() - interval '1 second' + WHERE organization_id = :organization_id + AND membership_id = :membership_id + """ + ), + { + "organization_id": scenario.organization_id, + "membership_id": scenario.membership_id, + }, + ) + finally: + engine.dispose() + expired = _resolve(client) + assert expired.status_code == 401 + assert expired.json() == {"code": "authentication_failed"} + assert SECRET not in caplog.text + + engine = create_database_engine(migration_configuration) + try: + with engine.connect() as connection: + for table_name in ("context_run", "decision_audit"): + values = connection.execute( + text(f"SELECT row_to_json(row)::text FROM {table_name} AS row") + ).scalars() + assert all(SECRET not in value for value in values) + finally: + engine.dispose() + + +@pytest.mark.security_evidence(id="RUNTIME-DOGFOOD-EPOCH-102", layer="runtime") +def test_dogfood_mid_resolve_policy_epoch_change_vetoes_stale_evidence( + request: pytest.FixtureRequest, + tmp_path: Path, + migration_configuration: DatabaseConfiguration, + runtime_configuration: DatabaseConfiguration, + control_configuration: DatabaseConfiguration, + guarded_control_engine: Engine, + guarded_worker_engine: Engine, + monkeypatch: pytest.MonkeyPatch, +) -> None: + scenario, user_id, target = _publish( + request, + tmp_path, + migration_configuration, + guarded_control_engine, + guarded_worker_engine, + ) + configuration = _configuration(scenario, user_id) + client = TestClient( + create_dogfood_app( + configuration, + _environment(configuration, runtime_configuration), + host="127.0.0.1", + ) + ) + final_read_reached = Event() + release_final_read = Event() + original_read = ( + membership_context_module._PostgreSQLPolicyEpochPort.read_current_epoch + ) + reads = 0 + + def block_final_epoch_read( + port: membership_context_module._PostgreSQLPolicyEpochPort, + organization_id: UUID, + ) -> object: + nonlocal reads + reads += 1 + if reads == 3: + final_read_reached.set() + if not release_final_read.wait(timeout=10): + raise RuntimeError("final Policy Epoch read was not released") + return original_read(port, organization_id) + + monkeypatch.setattr( + membership_context_module._PostgreSQLPolicyEpochPort, + "read_current_epoch", + block_final_epoch_read, + ) + control_engine = create_database_engine(control_configuration) + try: + with ThreadPoolExecutor(max_workers=1) as executor: + pending = executor.submit(_resolve, client) + assert final_read_reached.wait(timeout=10) + epoch = PostgreSQLAccessPolicyControl(control_engine).change_access( + ResourceAccessRevocation( + organization_id=scenario.organization_id, + resource_ref=target.resource_ref, + principal_ref="principal:file-reader", + expected_access_version=1, + ) + ) + assert epoch.value == 2 + release_final_read.set() + response = pending.result(timeout=10) + finally: + release_final_read.set() + control_engine.dispose() + + assert response.status_code == 200 + package = response.json()["package"] + assert package["blocks"] == package["evidence"] == [] + assert package["coverage"] == { + "status": "empty", + "reason": "no_authorized_evidence", + } + assert TARGET_TEXT not in response.text + assert reads == 3 + + +def test_dogfood_seed_cli_creates_one_idempotent_current_membership( + migration_configuration: DatabaseConfiguration, +) -> None: + organization_id = uuid4() + user_id = uuid4() + membership_id = uuid4() + command = ( + "context-engine-dogfood-seed", + "--organization-id", + str(organization_id), + "--user-id", + str(user_id), + "--membership-id", + str(membership_id), + ) + engine = create_database_engine(migration_configuration) + try: + first = subprocess.run(command, check=True, capture_output=True, text=True) + with engine.connect() as connection: + first_row = connection.execute( + text( + """ + SELECT user_id, status, membership_version, valid_from, + valid_until, xmin::text + FROM membership + WHERE organization_id = :organization_id + AND membership_id = :membership_id + """ + ), + { + "organization_id": organization_id, + "membership_id": membership_id, + }, + ).one() + second = subprocess.run(command, check=True, capture_output=True, text=True) + assert first.stdout == second.stdout + with engine.connect() as connection: + row = connection.execute( + text( + """ + SELECT user_id, status, membership_version, valid_from, + valid_until, xmin::text + FROM membership + WHERE organization_id = :organization_id + AND membership_id = :membership_id + """ + ), + { + "organization_id": organization_id, + "membership_id": membership_id, + }, + ).one() + assert row == first_row + assert tuple(row)[:3] == (user_id, "active", 1) + assert row.valid_until is None + with engine.begin() as connection: + connection.execute( + text( + """ + UPDATE membership + SET valid_from = statement_timestamp() + interval '1 day' + WHERE organization_id = :organization_id + AND membership_id = :membership_id + """ + ), + { + "organization_id": organization_id, + "membership_id": membership_id, + }, + ) + future = subprocess.run(command, check=False, capture_output=True, text=True) + assert future.returncode != 0 + assert "dogfood identity ready" not in future.stdout + finally: + with engine.begin() as connection: + connection.execute( + text( + """ + DELETE FROM membership + WHERE organization_id = :organization_id + AND membership_id = :membership_id + """ + ), + { + "organization_id": organization_id, + "membership_id": membership_id, + }, + ) + connection.execute( + text( + "DELETE FROM organization WHERE organization_id = :organization_id" + ), + {"organization_id": organization_id}, + ) + connection.execute( + text("DELETE FROM user_account WHERE user_id = :user_id"), + {"user_id": user_id}, + ) + engine.dispose() diff --git a/tests/integration/test_file_import_tracer.py b/tests/integration/test_file_import_tracer.py index 7e4a5fa1..b6691aab 100644 --- a/tests/integration/test_file_import_tracer.py +++ b/tests/integration/test_file_import_tracer.py @@ -76,7 +76,7 @@ _construct_existing_http_organization_verification, ) from engine.runtime.package_digest import QueryDigestKeyring -from engine.runtime.scope import ScopeSet, ScopeTarget +from engine.runtime.scope import EffectiveScope, ScopeSet, ScopeTarget from engine.runtime.scope_authority import ( TrustedScopeSnapshot, _close_scope_authority_scope, @@ -300,8 +300,14 @@ def discover( self, request: Acquire, projection_session: MaterializedProjectionSession, + *, + effective_scope: EffectiveScope, ) -> tuple[CandidateRef, ...]: - exact = self.exact.discover(request, projection_session) + exact = self.exact.discover( + request, + projection_session, + effective_scope=effective_scope, + ) return exact or (self.replay,) @@ -1784,6 +1790,16 @@ def test_exact_phrase_discovery_does_not_hide_a_match_after_sixty_four_rows( discovered = PostgreSQLExactPhraseCandidateIndex().discover( Acquire(need=ContextNeed(query="same exact paragraph")), projection_session, + effective_scope=EffectiveScope( + frozenset( + { + ScopeTarget( + organization_id, + "source:exact-limit", + ) + } + ) + ), ) finally: _close_materialized_projection_scope(projection_scope) diff --git a/tests/integration/test_membership_field_projection_integration.py b/tests/integration/test_membership_field_projection_integration.py index a07df96a..e4c356d9 100644 --- a/tests/integration/test_membership_field_projection_integration.py +++ b/tests/integration/test_membership_field_projection_integration.py @@ -185,9 +185,13 @@ def __init__(self, candidate: CandidateRef) -> None: self.returned_candidates: list[CandidateRef] = [] def discover( - self, request: Acquire, projection_session: object + self, + request: Acquire, + projection_session: object, + *, + effective_scope: object, ) -> tuple[CandidateRef, ...]: - del projection_session + del projection_session, effective_scope self.calls.append(request) self.returned_candidates.append(self.candidate) return (self.candidate,) diff --git a/tests/integration/test_pgvector_candidate_index.py b/tests/integration/test_pgvector_candidate_index.py index 03b46eac..a6b9b939 100644 --- a/tests/integration/test_pgvector_candidate_index.py +++ b/tests/integration/test_pgvector_candidate_index.py @@ -32,6 +32,7 @@ from engine.runtime.evidence import CandidateRef from engine.runtime.materialized import MaterializedProjectionSession from engine.runtime.package_digest import QueryDigestKeyring +from engine.runtime.scope import EffectiveScope, ScopeTarget from engine.supply import EmbeddingProfile, EmbeddingProviderUnavailable from tests.integration.test_file_import_tracer import ( _ExactScopeAuthority, @@ -68,8 +69,14 @@ def discover( self, request: Acquire, projection_session: MaterializedProjectionSession, + *, + effective_scope: EffectiveScope, ) -> tuple[CandidateRef, ...]: - candidates = self.inner.discover(request, projection_session) + candidates = self.inner.discover( + request, + projection_session, + effective_scope=effective_scope, + ) self.calls.append(candidates) return candidates @@ -85,8 +92,14 @@ def discover( self, request: Acquire, projection_session: MaterializedProjectionSession, + *, + effective_scope: EffectiveScope, ) -> tuple[CandidateRef, ...]: - candidates = self.inner.discover(request, projection_session) + candidates = self.inner.discover( + request, + projection_session, + effective_scope=effective_scope, + ) self.calls.append(candidates) self.discovered.set() if not self.release.wait(timeout=10): @@ -312,12 +325,27 @@ def _ann_plan_and_exact_result( *, source_refs: tuple[str, ...] | None = None, resource_refs: tuple[str, ...] | None = None, + effective_scope: EffectiveScope | None = None, ) -> tuple[ str, tuple[tuple[object, ...], ...], tuple[tuple[object, ...], ...], ]: embedding = DeterministicEmbeddingTwin().embed((QUERY,))[0] + if effective_scope is None: + effective_scope = EffectiveScope( + frozenset( + { + ScopeTarget( + scenario.organization_id, + str(scenario.source_ref.value), + ) + } + ) + ) + resource_targets = tuple( + target for target in effective_scope.targets if target.resource_ref is not None + ) parameters = { "query_embedding": "[" + ",".join(repr(value) for value in embedding) @@ -325,6 +353,13 @@ def _ann_plan_and_exact_result( "limit": 1, "source_refs": list(source_refs) if source_refs is not None else None, "resource_refs": list(resource_refs) if resource_refs is not None else None, + "scope_resource_organization_ids": [ + target.organization_id for target in resource_targets + ], + "scope_resource_source_refs": [ + target.source_ref for target in resource_targets + ], + "scope_resource_refs": [target.resource_ref for target in resource_targets], } with guarded_runtime_engine.begin() as connection: for name, value in _actor_settings(scenario, user_id).items(): @@ -502,7 +537,6 @@ def test_vector_candidate_http_chain_is_rls_scoped_content_free_and_bounded( index, request_id="issue-101-vector-authorized", ), - resource_refs=(candidate_a.resource_ref,), ) assert response.status_code == 200 @@ -553,7 +587,17 @@ def test_vector_candidate_http_chain_is_rls_scoped_content_free_and_bounded( guarded_runtime_engine, org_a, user_a, - resource_refs=(candidate_a.resource_ref,), + effective_scope=EffectiveScope( + frozenset( + { + ScopeTarget( + candidate_a.organization_id, + candidate_a.source_ref, + candidate_a.resource_ref, + ) + } + ) + ), ) assert "Index Scan using ix_context_fragment_embedding_hnsw" in plan assert approximate == exact @@ -608,7 +652,7 @@ def test_vector_candidate_denials_have_one_non_enumerating_empty_shape( ) ) ) - assert index.calls == [(candidate,)] + assert index.calls == [()] migration_engine = create_database_engine(migration_configuration) try: with migration_engine.begin() as connection: @@ -627,7 +671,7 @@ def test_vector_candidate_denials_have_one_non_enumerating_empty_shape( migration_engine.dispose() tombstoned_shape = _assert_empty(_resolve(client)) - assert index.calls == [(candidate,), ()] + assert index.calls == [(), ()] assert tombstoned_shape == unknown_shape migration_engine = create_database_engine(migration_configuration) @@ -650,7 +694,7 @@ def test_vector_candidate_denials_have_one_non_enumerating_empty_shape( revoked = _resolve(client) assert revoked.status_code == 401 assert revoked.json() == {"code": "authentication_failed"} - assert index.calls == [(candidate,), ()] + assert index.calls == [(), ()] for forbidden in ( candidate.source_ref, candidate.resource_ref, diff --git a/tests/integration/test_runtime_authorized_evidence_integration.py b/tests/integration/test_runtime_authorized_evidence_integration.py index 9e4da4cc..2e72d5db 100644 --- a/tests/integration/test_runtime_authorized_evidence_integration.py +++ b/tests/integration/test_runtime_authorized_evidence_integration.py @@ -195,9 +195,13 @@ def __init__( self.calls: list[Acquire] = [] def discover( - self, request: Acquire, projection_session: object + self, + request: Acquire, + projection_session: object, + *, + effective_scope: object, ) -> tuple[CandidateRef, ...]: - del projection_session + del projection_session, effective_scope self.calls.append(request) return self._ranked diff --git a/tests/integration/test_runtime_empty_package_integration.py b/tests/integration/test_runtime_empty_package_integration.py index 3b50eeab..54078dfd 100644 --- a/tests/integration/test_runtime_empty_package_integration.py +++ b/tests/integration/test_runtime_empty_package_integration.py @@ -111,8 +111,14 @@ class ContentIoSpy: def __init__(self) -> None: self.calls = 0 - def discover(self, request: Acquire, projection_session: object) -> tuple[()]: - del projection_session + def discover( + self, + request: Acquire, + projection_session: object, + *, + effective_scope: object, + ) -> tuple[()]: + del request, projection_session, effective_scope self.calls += 1 return () diff --git a/tests/integration/test_runtime_non_enumeration_integration.py b/tests/integration/test_runtime_non_enumeration_integration.py index 4a0d9afd..a171ef87 100644 --- a/tests/integration/test_runtime_non_enumeration_integration.py +++ b/tests/integration/test_runtime_non_enumeration_integration.py @@ -127,9 +127,13 @@ def __init__( self.calls: list[Acquire] = [] def discover( - self, request: Acquire, projection_session: object + self, + request: Acquire, + projection_session: object, + *, + effective_scope: object, ) -> tuple[CandidateRef, ...]: - del projection_session + del projection_session, effective_scope call_index = len(self.calls) self.calls.append(request) if call_index >= len(self.rankings): diff --git a/tests/integration/test_zz_file_resource_tombstone.py b/tests/integration/test_zz_file_resource_tombstone.py index 3b4532dc..c464fffa 100644 --- a/tests/integration/test_zz_file_resource_tombstone.py +++ b/tests/integration/test_zz_file_resource_tombstone.py @@ -34,6 +34,7 @@ from engine.runtime.evidence import CandidateRef from engine.runtime.materialized import MaterializedProjectionSession from engine.runtime.package_digest import QueryDigestKeyring +from engine.runtime.scope import EffectiveScope from tests.integration.test_zz_file_revision_replacement import ( OLD_MARKDOWN, _resolve, @@ -66,8 +67,10 @@ def discover( self, request: Acquire, projection_session: MaterializedProjectionSession, + *, + effective_scope: EffectiveScope, ) -> tuple[CandidateRef, ...]: - del request, projection_session + del request, projection_session, effective_scope return (self.candidate,) diff --git a/tests/integration/test_zz_file_revision_replacement.py b/tests/integration/test_zz_file_revision_replacement.py index 755ba295..64cb2573 100644 --- a/tests/integration/test_zz_file_revision_replacement.py +++ b/tests/integration/test_zz_file_revision_replacement.py @@ -30,6 +30,7 @@ from engine.runtime.evidence import CandidateRef from engine.runtime.materialized import MaterializedProjectionSession from engine.runtime.package_digest import QueryDigestKeyring +from engine.runtime.scope import EffectiveScope from engine.supply import ( MarkdownCompilerConfig, ParsedDocument, @@ -298,8 +299,14 @@ def discover( self, request: Acquire, projection_session: MaterializedProjectionSession, + *, + effective_scope: EffectiveScope, ) -> tuple[CandidateRef, ...]: - candidates = self._inner.discover(request, projection_session) + candidates = self._inner.discover( + request, + projection_session, + effective_scope=effective_scope, + ) self.discovered.set() if not self.release.wait(timeout=5): raise AssertionError("candidate discovery barrier timed out") diff --git a/tests/integration/test_zz_file_source_offboarding.py b/tests/integration/test_zz_file_source_offboarding.py index b6302d5d..c4ec9ad6 100644 --- a/tests/integration/test_zz_file_source_offboarding.py +++ b/tests/integration/test_zz_file_source_offboarding.py @@ -51,6 +51,7 @@ _construct_existing_http_organization_verification, ) from engine.runtime.package_digest import QueryDigestKeyring +from engine.runtime.scope import EffectiveScope from engine.runtime.ticket_identity import ( TicketExecutionIdentity, _construct_ticket_execution_identity, @@ -101,8 +102,10 @@ def discover( self, request: Acquire, projection_session: MaterializedProjectionSession, + *, + effective_scope: EffectiveScope, ) -> tuple[CandidateRef, ...]: - del request, projection_session + del request, projection_session, effective_scope return (self.candidate,) diff --git a/tests/support/releases.py b/tests/support/releases.py index 61efceda..f630ec2e 100644 --- a/tests/support/releases.py +++ b/tests/support/releases.py @@ -74,6 +74,8 @@ def active_runtime_release( *, suffix: str = "test-v0", active_revision_refs: tuple[str, ...] = (), + index_profile_ref: str = INDEX_PROFILE_REF_V0, + index_profile_digest: str = INDEX_PROFILE_DIGEST_V0, ) -> ActiveRuntimeRelease: return ActiveRuntimeRelease( organization_id=organization_id, @@ -81,12 +83,12 @@ def active_runtime_release( active_generation=1, content_profile_ref=CONTENT_PROFILE_REF_V0, content_schema_ref=CONTENT_SCHEMA_REF_V0, - index_profile_ref=INDEX_PROFILE_REF_V0, + index_profile_ref=index_profile_ref, index_schema_ref=INDEX_SCHEMA_REF_V0, runtime_profile_ref=RUNTIME_PROFILE_REF_V0, runtime_profile_digest=RUNTIME_PROFILE_DIGEST_V0, content_profile_digest=CONTENT_PROFILE_DIGEST_V0, - index_profile_digest=INDEX_PROFILE_DIGEST_V0, + index_profile_digest=index_profile_digest, tokenizer_ref=RUNTIME_TOKENIZER_REF_V0, package_schema_ref=PACKAGE_SCHEMA_REF_V0, curation_profile_ref=CURATION_PROFILE_REF_V0, @@ -104,6 +106,8 @@ def ensure_test_runtime_release( *, active_revision_refs: tuple[str, ...] | None = None, runtime_profile_ref: str = RUNTIME_PROFILE_REF_V0, + index_profile_ref: str = INDEX_PROFILE_REF_V0, + index_profile_digest: str = INDEX_PROFILE_DIGEST_V0, tokenizer_ref: str = RUNTIME_TOKENIZER_REF_V0, package_schema_ref: str = PACKAGE_SCHEMA_REF_V0, ) -> ActiveRuntimeRelease: @@ -269,8 +273,8 @@ def ensure_test_runtime_release( content_schema_ref=CONTENT_SCHEMA_REF_V0, ) index = IndexProfileRef( - profile_ref=INDEX_PROFILE_REF_V0, - profile_digest=INDEX_PROFILE_DIGEST_V0, + profile_ref=index_profile_ref, + profile_digest=index_profile_digest, content_profile_digest=content.profile_digest, content_schema_ref=content.content_schema_ref, index_schema_ref=INDEX_SCHEMA_REF_V0, diff --git a/tests/unit/test_dogfood_composition.py b/tests/unit/test_dogfood_composition.py new file mode 100644 index 00000000..d7c51ec9 --- /dev/null +++ b/tests/unit/test_dogfood_composition.py @@ -0,0 +1,183 @@ +from __future__ import annotations + +import logging +from collections.abc import Mapping +from uuid import UUID + +import pytest +from fastapi.testclient import TestClient + +from adapters.http.app import app as default_served_app +from adapters.http.authentication import AuthenticationRejected, DogfoodAuthenticator +from adapters.http.dogfood import ( + DOGFOOD_AGENT_ENV, + DOGFOOD_APPLICATION_ENV, + DOGFOOD_BINDING_ENV, + DOGFOOD_COMPOSITION_ENV, + DOGFOOD_COMPOSITION_VALUE, + DOGFOOD_EMBEDDING_PROVIDER_ENV, + DOGFOOD_EMBEDDING_PROVIDER_VALUE, + DOGFOOD_MEMBERSHIP_ENV, + DOGFOOD_MEMBERSHIP_VERSION_ENV, + DOGFOOD_ORGANIZATION_ENV, + DOGFOOD_PRINCIPAL_ENV, + DOGFOOD_SECRET_ENV, + DOGFOOD_USER_ENV, + DogfoodConfiguration, + DogfoodConfigurationUnavailable, + create_served_app, +) +from applications.api import main as api_main + +SECRET = "dogfood-secret-with-at-least-thirty-two-bytes" +ORGANIZATION_ID = UUID("81e18bca-86a1-478a-937d-7675c6fe69b0") +USER_ID = UUID("d3d9893f-82d2-4890-8cb2-4c7e57a56f16") +MEMBERSHIP_ID = UUID("9c9e9f4c-a5ec-4417-9408-0346e1c6c998") + + +def environment() -> dict[str, str]: + return { + DOGFOOD_COMPOSITION_ENV: DOGFOOD_COMPOSITION_VALUE, + DOGFOOD_SECRET_ENV: SECRET, + DOGFOOD_ORGANIZATION_ENV: str(ORGANIZATION_ID), + DOGFOOD_USER_ENV: str(USER_ID), + DOGFOOD_MEMBERSHIP_ENV: str(MEMBERSHIP_ID), + DOGFOOD_MEMBERSHIP_VERSION_ENV: "1", + DOGFOOD_PRINCIPAL_ENV: "principal:file-reader", + DOGFOOD_AGENT_ENV: "agent:dogfood-local:v1", + DOGFOOD_APPLICATION_ENV: "application:dogfood-local:v1", + DOGFOOD_BINDING_ENV: "binding:dogfood-local:v1", + DOGFOOD_EMBEDDING_PROVIDER_ENV: DOGFOOD_EMBEDDING_PROVIDER_VALUE, + } + + +def _load(source: Mapping[str, str] | None = None) -> DogfoodConfiguration: + return DogfoodConfiguration.load(environment() if source is None else source) + + +def test_absent_configuration_preserves_the_reject_all_served_composition() -> None: + client = TestClient(create_served_app({})) + + assert client.get("/health").json()["runtime_delivery"] == "NOT_ACTIVE" + response = client.post( + "/v0/resolve", + headers={ + "Authorization": f"Bearer {SECRET}", + "X-Context-Request-Id": "dogfood-default-rejects", + }, + json={"kind": "acquire", "need": {"query": "probe"}}, + ) + assert response.status_code == 401 + assert response.json() == {"code": "authentication_failed"} + + +def test_module_level_asgi_app_is_always_reject_all() -> None: + client = TestClient(default_served_app) + + assert client.get("/health").json()["runtime_delivery"] == "NOT_ACTIVE" + response = client.post( + "/v0/resolve", + headers={ + "Authorization": f"Bearer {SECRET}", + "X-Context-Request-Id": "direct-uvicorn-must-reject", + }, + json={"kind": "acquire", "need": {"query": "probe"}}, + ) + assert response.status_code == 401 + + +@pytest.mark.parametrize( + "credential", + ("wrong-dogfood-secret", SECRET[:-1]), +) +def test_dogfood_secret_rejections_are_generic_and_redacted( + credential: str, + caplog: pytest.LogCaptureFixture, +) -> None: + authenticator = DogfoodAuthenticator( + secret=SECRET, + authentication=_load().authentication(), + ) + + with caplog.at_level(logging.DEBUG), pytest.raises(AuthenticationRejected): + authenticator.authenticate(credential) + + assert SECRET not in caplog.text + assert SECRET not in repr(authenticator) + + +def test_dogfood_secret_authenticates_only_the_fixed_identity() -> None: + configuration = _load() + context = DogfoodAuthenticator( + secret=SECRET, + authentication=configuration.authentication(), + ).authenticate(SECRET) + + assert context.organization_ref == str(ORGANIZATION_ID) + assert context.user_ref == str(USER_ID) + assert context.membership_ref == str(MEMBERSHIP_ID) + assert context.principal_ref == "principal:file-reader" + + +@pytest.mark.parametrize( + ("changed_name", "changed_value"), + ( + (DOGFOOD_SECRET_ENV, "short"), + (DOGFOOD_MEMBERSHIP_VERSION_ENV, "0"), + (DOGFOOD_MEMBERSHIP_VERSION_ENV, "not-an-integer"), + (DOGFOOD_ORGANIZATION_ENV, "not-a-uuid"), + (DOGFOOD_EMBEDDING_PROVIDER_ENV, "external"), + ), +) +def test_partial_or_widening_dogfood_configuration_fails_closed( + changed_name: str, + changed_value: str, +) -> None: + source = environment() + source[changed_name] = changed_value + + with pytest.raises(DogfoodConfigurationUnavailable): + DogfoodConfiguration.load(source) + + +def test_missing_required_dogfood_configuration_fails_closed() -> None: + source = environment() + del source[DOGFOOD_SECRET_ENV] + + with pytest.raises(DogfoodConfigurationUnavailable): + DogfoodConfiguration.load(source) + + +def test_query_digest_key_is_derived_without_exposing_the_dogfood_secret() -> None: + configuration = _load() + + assert repr(configuration.query_digest_keyring()) == ( + "QueryDigestKeyring()" + ) + assert SECRET not in repr(configuration) + + +def test_dogfood_api_refuses_non_loopback_binding( + monkeypatch: pytest.MonkeyPatch, +) -> None: + monkeypatch.setenv(DOGFOOD_COMPOSITION_ENV, DOGFOOD_COMPOSITION_VALUE) + + with pytest.raises(SystemExit) as failure: + api_main(["--host", "0.0.0.0"]) + + assert failure.value.code == 2 + + +def test_default_api_host_policy_is_not_changed( + monkeypatch: pytest.MonkeyPatch, +) -> None: + monkeypatch.delenv(DOGFOOD_COMPOSITION_ENV, raising=False) + called: dict[str, object] = {} + + def observe(*args: object, **kwargs: object) -> None: + called.update(kwargs) + + monkeypatch.setattr("applications.api.uvicorn.run", observe) + api_main(["--host", "0.0.0.0"]) + + assert called["host"] == "0.0.0.0" diff --git a/tests/unit/test_effective_scope_runtime.py b/tests/unit/test_effective_scope_runtime.py index 1e69957b..d2eaf4df 100644 --- a/tests/unit/test_effective_scope_runtime.py +++ b/tests/unit/test_effective_scope_runtime.py @@ -64,8 +64,14 @@ class ContentIoSpy: def __init__(self) -> None: self.calls = 0 - def discover(self, request: Acquire, projection_session: object) -> tuple[()]: - del request, projection_session + def discover( + self, + request: Acquire, + projection_session: object, + *, + effective_scope: object, + ) -> tuple[()]: + del request, projection_session, effective_scope self.calls += 1 return () diff --git a/tests/unit/test_http_scope_authority.py b/tests/unit/test_http_scope_authority.py index e6811c4b..c9bb50c8 100644 --- a/tests/unit/test_http_scope_authority.py +++ b/tests/unit/test_http_scope_authority.py @@ -1,7 +1,7 @@ from __future__ import annotations import pickle -from dataclasses import FrozenInstanceError +from dataclasses import FrozenInstanceError, replace from datetime import UTC, datetime, timedelta, timezone from typing import Any, cast from uuid import UUID @@ -9,16 +9,30 @@ import pytest from adapters.http.scope_authority import ( + DogfoodFileScopeAuthority, MissingTrustedScopeAuthority, ScopeAuthority, ScopeAuthorityIdentity, + ScopeAuthorityUnavailable, ) -from engine.runtime.scope import MISSING_TRUSTED_SCOPE +from engine.runtime.materialized import ( + MaterializedProjectionPort, + MaterializedScopeOperands, + _close_materialized_projection_scope, + _construct_materialized_projection_session, + _open_materialized_projection_scope, +) +from engine.runtime.release_lineage import ( + DOGFOOD_VECTOR_INDEX_PROFILE_DIGEST_V1, + DOGFOOD_VECTOR_INDEX_PROFILE_REF_V1, +) +from engine.runtime.scope import MISSING_TRUSTED_SCOPE, ScopeSet, ScopeTarget from engine.runtime.scope_authority import ( TrustedScopeSnapshot, _require_active_trusted_scope_snapshot, _trusted_operands_from_snapshot, ) +from tests.support.releases import active_runtime_release CHECKED_AT = datetime(2026, 7, 21, 9, 30, tzinfo=UTC) ORGANIZATION_ID = UUID("81e18bca-86a1-478a-937d-7675c6fe69b0") @@ -170,3 +184,94 @@ def test_scope_authority_identity_repr_does_not_expose_trusted_refs() -> None: assert "context.answer" not in rendered assert "request-1" not in rendered assert "binding-from-auth" not in rendered + + +def test_dogfood_scope_carries_independent_durable_operands() -> None: + targets = tuple( + ScopeTarget(ORGANIZATION_ID, "source:file", f"resource:{name}") + for name in ("a", "b", "c", "d", "e") + ) + + class OperandPort: + def current_scope_operands( + self, + active_revision_ids: tuple[UUID, ...], + ) -> MaterializedScopeOperands: + assert active_revision_ids == ( + UUID("0425904c-480f-4022-930f-15e8dd949a7e"), + ) + return MaterializedScopeOperands( + organization_boundary=frozenset(targets), + membership_rights=frozenset(targets[:4]), + principal_grants=frozenset(targets[:3]), + source_native_acl=frozenset(targets[:2]), + resource_acl=frozenset(targets[:1]), + ) + + def source_is_active(self, source_ref: UUID) -> bool: + del source_ref + return True + + def discover_vector(self, *args: object, **kwargs: object) -> tuple[()]: + del args, kwargs + return () + + def discover_exact_phrase(self, phrase_digest: str) -> tuple[()]: + del phrase_digest + return () + + def observe_publication(self, candidate_ref: object) -> None: + del candidate_ref + + def locate(self, candidate_ref: object) -> None: + del candidate_ref + + def project(self, locator: object) -> None: + del locator + + projection_scope = _open_materialized_projection_scope() + session = _construct_materialized_projection_session( + authority_scope=projection_scope, + port=cast(MaterializedProjectionPort, OperandPort()), + ) + release = active_runtime_release( + ORGANIZATION_ID, + active_revision_refs=("0425904c-480f-4022-930f-15e8dd949a7e",), + index_profile_ref=DOGFOOD_VECTOR_INDEX_PROFILE_REF_V1, + index_profile_digest=DOGFOOD_VECTOR_INDEX_PROFILE_DIGEST_V1, + ) + bound = replace( + identity(), + materialized_projection_session=session, + active_runtime_release=release, + ) + authority = DogfoodFileScopeAuthority( + organization_id=ORGANIZATION_ID, + principal_ref="principal-from-auth", + agent_version_ref="agent-version-from-server", + purpose="context.answer", + ) + try: + with authority.current_scope(bound) as snapshot: + operands = _trusted_operands_from_snapshot(snapshot) + assert operands.organization_boundary == ScopeSet(frozenset(targets)) + assert operands.membership_rights == ScopeSet(frozenset(targets[:4])) + assert operands.principal_grants == ScopeSet(frozenset(targets[:3])) + assert operands.source_native_acl == ScopeSet(frozenset(targets[:2])) + assert operands.resource_acl == ScopeSet(frozenset(targets[:1])) + assert operands.agent_ceiling == ScopeSet(frozenset(targets)) + assert operands.purpose_policy == ScopeSet(frozenset(targets)) + for mismatched in ( + replace(bound, agent_version_ref="agent-version-not-authorized"), + replace(bound, purpose="context.not-authorized"), + ): + with ( + pytest.raises( + ScopeAuthorityUnavailable, + match="scope binding is unavailable", + ), + authority.current_scope(mismatched), + ): + pass + finally: + _close_materialized_projection_scope(projection_scope) diff --git a/tests/unit/test_http_trust_boundary.py b/tests/unit/test_http_trust_boundary.py index 81390636..0975d40a 100644 --- a/tests/unit/test_http_trust_boundary.py +++ b/tests/unit/test_http_trust_boundary.py @@ -313,8 +313,14 @@ def __init__(self) -> None: self.provider_calls = 0 self.source_content_calls = 0 - def discover(self, request: Acquire, projection_session: object) -> tuple[()]: - del request, projection_session + def discover( + self, + request: Acquire, + projection_session: object, + *, + effective_scope: Any, + ) -> tuple[()]: + del request, projection_session, effective_scope self.index_calls += 1 return () diff --git a/tests/unit/test_materialized_projection.py b/tests/unit/test_materialized_projection.py index 664d1dc6..03d95628 100644 --- a/tests/unit/test_materialized_projection.py +++ b/tests/unit/test_materialized_projection.py @@ -22,6 +22,7 @@ _open_materialized_projection_scope, _project_materialized_fragment, ) +from engine.runtime.scope import EffectiveScope ORGANIZATION_ID = UUID("81e18bca-86a1-478a-937d-7675c6fe69b0") @@ -61,8 +62,9 @@ def discover_vector( limit: int, source_refs: tuple[str, ...] | None, resource_refs: tuple[str, ...] | None, + effective_scope: EffectiveScope, ) -> tuple[CandidateRef, ...]: - del query_embedding, limit, source_refs, resource_refs + del query_embedding, limit, source_refs, resource_refs, effective_scope return () def source_is_active(self, source_ref: UUID) -> bool: @@ -177,11 +179,25 @@ def test_vector_discovery_is_bounded_and_lifetime_bound() -> None: port=cast(MaterializedProjectionPort, port), ) - assert _discover_materialized_vector(session, (0.25,), 1) == () + effective_scope = EffectiveScope(frozenset()) + assert ( + _discover_materialized_vector( + session, + (0.25,), + 1, + effective_scope=effective_scope, + ) + == () + ) _close_materialized_projection_scope(scope) with pytest.raises(ValueError, match="active materialized projection scope"): - _discover_materialized_vector(session, (0.25,), 1) + _discover_materialized_vector( + session, + (0.25,), + 1, + effective_scope=effective_scope, + ) def test_materialized_projection_rejects_more_than_the_public_field_bound() -> None: diff --git a/tests/unit/test_membership_context.py b/tests/unit/test_membership_context.py index 7a559a50..e809be89 100644 --- a/tests/unit/test_membership_context.py +++ b/tests/unit/test_membership_context.py @@ -43,6 +43,7 @@ RUNTIME_TOKENIZER_REF_V0, public_release_manifest_ref, ) +from engine.runtime.scope import EffectiveScope, ScopeTarget CHECKED_AT = datetime(2026, 7, 21, 8, 0, tzinfo=UTC) @@ -368,6 +369,17 @@ def test_postgres_vector_discovery_uses_one_content_free_bounded_ann_query() -> 1, ("source:vector",), ("resource:vector",), + EffectiveScope( + frozenset( + { + ScopeTarget( + organization_id, + "source:vector", + "resource:vector", + ) + } + ) + ), ) assert candidates == ( @@ -404,6 +416,9 @@ def test_postgres_vector_discovery_uses_one_content_free_bounded_ann_query() -> "limit": 1, "source_refs": ["source:vector"], "resource_refs": ["resource:vector"], + "scope_resource_organization_ids": [organization_id], + "scope_resource_source_refs": ["source:vector"], + "scope_resource_refs": ["resource:vector"], } @@ -424,7 +439,23 @@ def test_postgres_vector_discovery_rejects_invalid_revision_lineage() -> None: ) with pytest.raises(TypeError, match="invalid revision lineage"): - port.discover_vector((0.25, -0.5), 1, None, None) + port.discover_vector( + (0.25, -0.5), + 1, + None, + None, + EffectiveScope( + frozenset( + { + ScopeTarget( + identity().organization_id, + "source:vector", + "resource:vector", + ) + } + ) + ), + ) @pytest.mark.parametrize("invalid_value", (None, "", " \t")) diff --git a/tests/unit/test_membership_field_projection.py b/tests/unit/test_membership_field_projection.py index 448fee39..edbbe30a 100644 --- a/tests/unit/test_membership_field_projection.py +++ b/tests/unit/test_membership_field_projection.py @@ -50,8 +50,9 @@ def discover_vector( limit: int, source_refs: tuple[str, ...] | None, resource_refs: tuple[str, ...] | None, + effective_scope: object, ) -> tuple[()]: - del query_embedding, limit, source_refs, resource_refs + del query_embedding, limit, source_refs, resource_refs, effective_scope return () def source_is_active(self, source_ref: UUID) -> bool: diff --git a/tests/unit/test_pgvector_candidate_index.py b/tests/unit/test_pgvector_candidate_index.py index 6a27d3b2..dd50a050 100644 --- a/tests/unit/test_pgvector_candidate_index.py +++ b/tests/unit/test_pgvector_candidate_index.py @@ -19,6 +19,7 @@ _construct_materialized_projection_session, _open_materialized_projection_scope, ) +from engine.runtime.scope import EffectiveScope, ScopeTarget from engine.supply import EmbeddingProfile, EmbeddingProviderUnavailable @@ -31,6 +32,7 @@ def __init__(self, candidates: tuple[CandidateRef, ...]) -> None: int, tuple[str, ...] | None, tuple[str, ...] | None, + EffectiveScope, ] ] = [] @@ -40,8 +42,17 @@ def discover_vector( limit: int, source_refs: tuple[str, ...] | None, resource_refs: tuple[str, ...] | None, + effective_scope: EffectiveScope, ) -> tuple[CandidateRef, ...]: - self.calls.append((query_embedding, limit, source_refs, resource_refs)) + self.calls.append( + ( + query_embedding, + limit, + source_refs, + resource_refs, + effective_scope, + ) + ) return self.candidates[:limit] def discover_exact_phrase(self, phrase_digest: str) -> tuple[()]: @@ -80,6 +91,21 @@ def _candidate() -> CandidateRef: ) +def _effective_scope() -> EffectiveScope: + candidate = _candidate() + return EffectiveScope( + frozenset( + { + ScopeTarget( + candidate.organization_id, + candidate.source_ref, + candidate.resource_ref, + ) + } + ) + ) + + def test_vector_index_embeds_query_and_returns_only_bounded_candidate_refs() -> None: port = _RecordingPort((_candidate(),)) scope = _open_materialized_projection_scope() @@ -94,17 +120,19 @@ def test_vector_index_embeds_query_and_returns_only_bounded_candidate_refs() -> ).discover( Acquire(need=ContextNeed(query="semantic query")), session, + effective_scope=_effective_scope(), ) finally: _close_materialized_projection_scope(scope) assert candidates == (_candidate(),) assert len(port.calls) == 1 - query_embedding, limit, source_refs, resource_refs = port.calls[0] + query_embedding, limit, source_refs, resource_refs, effective_scope = port.calls[0] assert len(query_embedding) == 384 assert limit == 1 assert source_refs is None assert resource_refs is None + assert effective_scope == _effective_scope() assert set(CandidateRef.__dataclass_fields__) == { "organization_id", "source_ref", @@ -134,11 +162,12 @@ def test_vector_index_applies_request_narrowing_before_ann_limit() -> None: ), ), session, + effective_scope=_effective_scope(), ) finally: _close_materialized_projection_scope(scope) - assert port.calls[0][2:] == ( + assert port.calls[0][2:4] == ( ("source:vector",), ("resource:vector",), ) @@ -159,6 +188,7 @@ def test_vector_index_genericizes_query_embedding_failure_before_database_io() - PostgreSQLVectorCandidateIndex(_UnavailableProvider()).discover( Acquire(need=ContextNeed(query="semantic query")), session, + effective_scope=_effective_scope(), ) finally: _close_materialized_projection_scope(scope) diff --git a/tests/unit/test_runtime_authorized_evidence.py b/tests/unit/test_runtime_authorized_evidence.py index 56c01b14..4655435c 100644 --- a/tests/unit/test_runtime_authorized_evidence.py +++ b/tests/unit/test_runtime_authorized_evidence.py @@ -153,9 +153,13 @@ def __init__(self, ranked: tuple[CandidateRef, ...]) -> None: self.calls = 0 def discover( - self, request: Acquire, projection_session: object + self, + request: Acquire, + projection_session: object, + *, + effective_scope: object, ) -> tuple[CandidateRef, ...]: - del request, projection_session + del request, projection_session, effective_scope self.calls += 1 return self.ranked @@ -181,8 +185,9 @@ def discover_vector( limit: int, source_refs: tuple[str, ...] | None, resource_refs: tuple[str, ...] | None, + effective_scope: object, ) -> tuple[()]: - del query_embedding, limit, source_refs, resource_refs + del query_embedding, limit, source_refs, resource_refs, effective_scope return () def source_is_active(self, source_ref: UUID) -> bool: diff --git a/tests/unit/test_runtime_empty_package.py b/tests/unit/test_runtime_empty_package.py index 05779d1a..d8f720fc 100644 --- a/tests/unit/test_runtime_empty_package.py +++ b/tests/unit/test_runtime_empty_package.py @@ -84,8 +84,14 @@ def __init__(self) -> None: self.provider_calls = 0 self.source_content_calls = 0 - def discover(self, request: Acquire, projection_session: object) -> tuple[()]: - del projection_session + def discover( + self, + request: Acquire, + projection_session: object, + *, + effective_scope: Any, + ) -> tuple[()]: + del request, projection_session, effective_scope self.index_calls += 1 return () @@ -480,7 +486,11 @@ def test_content_io_spy_would_detect_every_runtime_dependency_call() -> None: candidate = runtime(spy) request = Acquire(need=ContextNeed(query="mutation control")) - candidate._content_io.index.discover(request, cast(Any, None)) + candidate._content_io.index.discover( + request, + cast(Any, None), + effective_scope=cast(Any, None), + ) candidate._content_io.provider.authorize_and_project() candidate._content_io.source_content.read_content() diff --git a/tests/unit/test_runtime_unavailable_capabilities.py b/tests/unit/test_runtime_unavailable_capabilities.py index 2a6e67e2..b8aaa704 100644 --- a/tests/unit/test_runtime_unavailable_capabilities.py +++ b/tests/unit/test_runtime_unavailable_capabilities.py @@ -87,8 +87,14 @@ def __init__(self) -> None: self.provider_calls = 0 self.source_calls = 0 - def discover(self, request: Acquire, projection_session: object) -> tuple[()]: - del request, projection_session + def discover( + self, + request: Acquire, + projection_session: object, + *, + effective_scope: Any, + ) -> tuple[()]: + del request, projection_session, effective_scope self.index_calls += 1 return () @@ -321,7 +327,9 @@ def test_content_twins_are_observable_controls_for_every_prohibited_call() -> No content_io = RuntimeContentIo(index=twin, provider=twin, source_content=twin) content_io.index.discover( - Acquire(need=ContextNeed(query="control")), cast(Any, None) + Acquire(need=ContextNeed(query="control")), + cast(Any, None), + effective_scope=cast(Any, None), ) content_io.provider.authorize_and_project() content_io.source_content.read_content()