Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 49 additions & 14 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,55 @@

## Code-owner review gates — disabled (on hold)

As of 2026-08-04, code-owner review requirements (`require_code_owner_reviews` in branch
protection, `require_code_owner_review` in rulesets) are disabled across the ContextualWisdomLab
org: there is a single maintainer (solo developer), so a code-owner approval gate can never be
satisfied. This is ON HOLD until the org has multiple maintainers — do NOT re-enable these
settings or add CODEOWNERS-based merge gates before then.
As of 2026-08-04, code-owner review requirements
(`require_code_owner_reviews` in branch protection and
`require_code_owner_review` in rulesets) are disabled across the
ContextualWisdomLab organization. There is currently one maintainer, so a
code-owner approval gate cannot be satisfied. Do not re-enable these settings or
add CODEOWNERS-based merge gates until multiple independent maintainers exist.

## Tenant lifecycle security contract

- Preserve standalone operation under the exact `standalone` tenant scope and
keep the existing four-argument lifecycle-recorder interface source
compatible.
- Derive `tenant_scope` only from an authenticated and authorized host boundary.
Provider metadata, remote identifiers, request payloads, model output,
transport headers, and endpoint aliases are never tenant authorities.
- Validate tenant context before observation reservation, credential
resolution, provider I/O, or database I/O.
- Bind tenant context with parameterized, transaction-local `set_config`; every
lifecycle lookup, conflict target, and operational index must be
tenant-qualified.
- Treat the custom PostgreSQL setting as a trusted application boundary, not a
credential. A role with arbitrary SQL can select an arbitrary tenant scope;
do not expose the lifecycle application role through generic tenant-controlled
SQL, and never describe RLS as a substitute for authorization or
SQL-injection prevention.
- Keep PostgreSQL row-level security enabled and forced. Application roles must
be `NOSUPERUSER NOBYPASSRLS`; administrative bypass identities are outside
the application isolation guarantee.
- Migrations must restore forced RLS within the same atomic SQL statement that
relaxes owner enforcement, preserve legacy rows under `standalone`, remain
idempotent, and keep the packaged and Docker initialization schemas
byte-for-byte identical.
- Update the README, operator guide, architecture, ADR, doctoring, and CHANGELOG
whenever tenant identity, role, migration, direct-SQL, or rollback contracts
change.
- Maintain 100% production statement, branch, and public-docstring coverage with
realistic tenant-isolation, migration, rollback, compatibility, and
concurrency tests.

## Provider retry invariant

Automatic provider retries are restricted to idempotent GET operations. The reviewed default
HTTP status set is exactly `{408, 425, 429, 502, 503, 504}`; HTTP 425 `Too Early` uses the same
bounded `Retry-After` or equal-jitter delay path as the other statuses. TLS handshake and
certificate failures are never retried automatically; a repeated request cannot repair peer
identity or TLS policy. Certificate fingerprint mismatches are never retried automatically for
the same peer-identity reason. Provider POST operations remain single-attempt, and HTTP 500 is
not retryable by default without a separately reviewed provider-specific contract. Do not widen
this replay boundary without deterministic regression tests and authoritative protocol/security
documentation.
Automatic provider retries are restricted to idempotent GET operations. The
reviewed default HTTP status set is exactly `{408, 425, 429, 502, 503, 504}`;
HTTP 425 `Too Early` uses the same bounded `Retry-After` or equal-jitter delay
path as the other statuses. TLS handshake and certificate failures are never
retried automatically; a repeated request cannot repair peer identity or TLS
policy. Certificate fingerprint mismatches are never retried automatically for
the same peer-identity reason. Provider POST operations remain single-attempt,
and HTTP 500 is not retryable by default without a separately reviewed
provider-specific contract. Do not widen this replay boundary without
deterministic regression tests and authoritative protocol/security
documentation.
76 changes: 76 additions & 0 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# Architecture

## Deployment boundary

`pg-llm-batch` remains independently deployable and embeddable. PostgreSQL owns
configuration, encrypted secrets, token counting, JSONL payloads, and durable
provider lifecycle state. Provider HTTP behavior remains behind
`BatchAPIClient`, while host services may inject credential, observation-order,
and lifecycle-persistence seams without changing provider semantics.

## Durable lifecycle tenancy

`DurableBatchAPIClient` is the backward-compatible standalone facade.
`TenantDurableBatchAPIClient` requires a tenant scope selected by a trusted host
after authentication and authorization. The durable business identity is:

```text
(tenant_scope, endpoint_alias, remote_batch_id)
```

Package reads and writes bind the validated scope with parameterized,
transaction-local `set_config('pg_llm_batch.tenant_scope', ..., true)`.
PostgreSQL row-level security is enabled and forced, so missing context is
default-deny for ordinary application roles. PostgreSQL superusers and roles
with `BYPASSRLS` remain administrative escape hatches and must not be used as
application identities.

The custom setting is part of a **trusted application boundary**. It is not a
credential and is not a substitute for authentication, authorization,
SQL-injection prevention, or restricted direct database access. A role capable
of arbitrary SQL can call `set_config` with an arbitrary tenant scope. CWL hosts
must therefore expose tenant lifecycle operations through parameterized package
APIs or a separately reviewed identity-to-role interface, not a generic SQL
surface.

Provider metadata, endpoint aliases, provider resource identifiers, payloads,
model output, and transport headers never select tenant authorization context.

## Migration and rollback

Legacy lifecycle rows are backfilled to `standalone` without deletion or
identity merging. The prior endpoint/provider unique key is replaced by a
tenant-qualified key. The owner-enforcement transition, backfill, constraint
migration, and forced-RLS restoration execute in one PostgreSQL anonymous block
so psql autocommit cannot commit an intermediate owner-bypass state.

Enabling RLS changes the behavior of direct SQL integrations: an ordinary role
that does not bind an authorized transaction-local scope sees no lifecycle
rows. Those integrations must move to `get_remote_batch_state`,
`get_tenant_remote_batch_state`, or a reviewed tenant-binding database interface
before deployment.

Rollback to the former two-column key is unsafe until an operator proves that no
`(endpoint_alias, remote_batch_id)` pair exists in more than one tenant scope.
The packaged schema and Docker initialization schema are maintained as exact
mirrors and must be reapplied successfully more than once.

## Modular interoperability

CWL hosts such as `contextual-orchestrator` and `naruon` supply tenant context
only after their own authentication and authorization boundary. The package
does not require either host and retains standalone operation. When embedded,
tenant scope is a local control-plane identity and not model- or
provider-returned data.

## Verification boundary

Deterministic gates cover strict tenant validation, standalone compatibility,
tenant-qualified SQL parameters, current-state reconciliation, migration
idempotency, malformed database rows, default-deny policy text, schema
mirroring, operator documentation, and 100% production statement and branch
coverage. Live PostgreSQL isolation tests use a `NOSUPERUSER NOBYPASSRLS` role
and prove that identical provider identifiers in different tenants remain
independently addressable and mutually invisible when access occurs through the
trusted package boundary. They do not claim isolation after arbitrary SQL is
granted.
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

- Trusted tenant-scoped durable lifecycle identities for shared-table MSA deployments, including `TenantDurableBatchAPIClient`, tenant-qualified persistence and read helpers, transaction-local PostgreSQL context, forced default-deny row-level security, and explicit standalone compatibility.
- Independent 1 MiB bounded-stream decoding for Files and Batches control-plane JSON before strict UTF-8 and object parsing.
- Opt-in OpenTelemetry spans, operation counts, and duration histograms for all
caller-invoked public Batch API client operations, with explicit tracer/meter
Expand All @@ -35,6 +36,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Fixed

- Rejected non-callable standalone and tenant lifecycle recorders or observation reservers during client construction, before any provider operation can succeed without a usable persistence path.
- Made the tenant lifecycle migration atomic across owner-enforcement relaxation, legacy-row backfill, constraint replacement, and forced-RLS restoration so psql autocommit cannot commit an intermediate owner-bypass state.
- Bootstrap DSN and Fernet-key source selection now consults process environment
only when the corresponding explicit argument is omitted. Explicit Postgres
DSNs must be exact nonblank strings, explicit Fernet keys must be exact
Expand Down
29 changes: 29 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# CLAUDE.md

## Tenant lifecycle invariants

- Preserve the standalone client, its four-argument recorder seam, and the
explicit `standalone` database scope.
- Never derive tenant scope from provider metadata, remote identifiers, request
bodies, model output, endpoint aliases, or transport headers.
- Validate tenant scope before observation reservation, credential lookup,
provider I/O, or database I/O.
- Bind validated scope as a parameter with transaction-local `set_config`
before lifecycle table access.
- Include tenant scope in every lifecycle lookup, unique identity, conflict
target, and operational status index.
- Treat the custom setting as a trusted application boundary rather than a
credential. A database role with arbitrary SQL can call `set_config` for an
arbitrary tenant scope, so generic tenant-controlled SQL, SQL injection, and
incorrect identity mapping remain outside the RLS guarantee.
- Keep row-level security enabled and forced. Production application roles are
`NOSUPERUSER NOBYPASSRLS`.
- Keep owner-enforcement relaxation, legacy backfill, constraint migration, and
forced-RLS restoration inside one atomic PostgreSQL statement.
- Keep `pg_llm_batch/schema.sql` and
`docker/postgres/init/02_schema.sql` byte-for-byte identical.
- Keep README, operator, architecture, ADR, doctoring, and CHANGELOG contracts
synchronized with every tenant security or migration change.
- Maintain 100% production statement, branch, and public-docstring coverage.
Add realistic migration, rollback, compatibility, security, and
tenant-isolation tests before implementation changes.
65 changes: 65 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,10 @@ relicensed to **Apache-2.0** (see [`NOTICE`](NOTICE) for provenance).
through a no-echo prompt or bounded standard input, never as process arguments.
- **Disk-free assembly.** JSONL payloads are stored as `JSONB` and reconstructed
by JOIN, never written to disk.
- **Standalone or tenant-scoped lifecycle state.** `DurableBatchAPIClient`
preserves the standalone contract, while `TenantDurableBatchAPIClient` binds
shared-table lifecycle state to a trusted host-selected `tenant_scope` with
forced PostgreSQL row-level security.

## Architecture

Expand All @@ -41,6 +45,8 @@ llm_requests ──▶ PostgresBatchOrchestrator.prepare_batches()
| Token counting + accumulation | `pg_llm_batch/token_counter.py` |
| Batch assembly + persistence | `pg_llm_batch/orchestrator.py` |
| Submit / poll / wait / retrieve | `pg_llm_batch/batch_api_client.py` |
| Durable standalone and tenant lifecycle clients | `pg_llm_batch/durable_client.py` |
| Tenant-qualified lifecycle persistence and reads | `pg_llm_batch/db.py` |
| Opt-in OpenTelemetry operations | `pg_llm_batch/observability.py` |
| KV config + encrypted secrets | `pg_llm_batch/config.py` |
| DDL subset | `pg_llm_batch/schema.sql` |
Expand All @@ -52,6 +58,8 @@ llm_requests ──▶ PostgresBatchOrchestrator.prepare_batches()
- PostgreSQL with `pg_tiktoken`, `pg_cron`, and `http` (pgsql-http). The bundled
image (`docker/postgres/Dockerfile`) builds all three.
- Python 3.10+ with `psycopg[binary]` and `aiohttp` (installed via `pip install .`).
- Tenant-scoped lifecycle deployments require an application database role with
`NOSUPERUSER NOBYPASSRLS` and a trusted host authorization boundary.

---

Expand Down Expand Up @@ -135,6 +143,58 @@ same `pg_llm_batch_health_check()` SQL function.

---

## Durable lifecycle modes

Apply the canonical schema before using package-owned durable lifecycle state:

```python
from pg_llm_batch import db

db.apply_schema(dsn)
```

`DurableBatchAPIClient` keeps the original single-tenant facade and records under
the exact `standalone` scope. Shared-table hosts use
`TenantDurableBatchAPIClient` with a trusted tenant identity selected by the
host's authenticated authorization context:

```python
from pg_llm_batch import TenantDurableBatchAPIClient, get_tenant_remote_batch_state

async with TenantDurableBatchAPIClient(
dsn,
credentials_provider,
tenant_scope="customer-42",
) as client:
created = await client.create_batch_job(
input_file_id="file-provider-id",
endpoint_alias="default",
endpoint="/v1/responses",
)

state = get_tenant_remote_batch_state(
dsn,
"customer-42",
"default",
created["id"],
)
```

The durable identity is `(tenant_scope, endpoint_alias, remote_batch_id)`.
Package helpers bind tenant scope with parameterized transaction-local PostgreSQL
context and the schema enables and forces default-deny RLS. Provider metadata,
resource identifiers, payloads, and headers never select `tenant_scope`.

The custom PostgreSQL setting is **not** a tenant credential. A database role
that can execute arbitrary SQL can set arbitrary session state, so production
application roles must be `NOSUPERUSER NOBYPASSRLS`, must not be exposed through
a generic SQL surface, and still require normal authentication, authorization,
and SQL-injection controls. Direct SQL consumers that do not establish an
authorized tenant scope see no lifecycle rows after RLS is enabled.

See [`docs/remote-batch-lifecycle.md`](docs/remote-batch-lifecycle.md) for the
migration, rollback, pooling, recovery, custom-recorder, and assurance contract.

## Embed as a git submodule

```bash
Expand Down Expand Up @@ -233,6 +293,11 @@ PG_LLM_BATCH_TEST_DSN=postgresql://pgllm:pgllm@localhost:5432/pgllm \

## Docs

- [`docs/remote-batch-lifecycle.md`](docs/remote-batch-lifecycle.md)
— standalone and tenant-scoped durable lifecycle operation, RLS trust boundary,
migration, rollback, pooling, and recovery.
- [`docs/doctoring/tenant-scoped-lifecycle.md`](docs/doctoring/tenant-scoped-lifecycle.md)
— tenant identity, RLS authority, compatibility, and APA 7 references.
- [`docs/doctoring/cli-secret-input.md`](docs/doctoring/cli-secret-input.md)
— no-echo interactive secret entry, bounded stdin automation, fail-closed
validation, verification, and security references.
Expand Down
Loading
Loading