Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 55 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,44 +1,81 @@
# ContextEngine

> A multi-tenant context delivery engine: connect your team's knowledge sources upstream, deliver **authorized, evidence-backed, budget-bounded** ContextPackages to agents and IM bots downstream.
> A multi-tenant context delivery engine: connect your team's knowledge sources
> upstream, deliver **authorized, evidence-backed, budget-bounded**
> ContextPackages to agents and IM bots downstream.

多租户上下文交付引擎——上游连接团队知识源(飞书 / Slack / Google Docs / 企业微信),下游把「经过授权、带证据、有预算」的 ContextPackage 交付给 agent 应用与 IM bot(飞书群聊问答优先)。
多租户上下文交付引擎——上游连接团队知识源(飞书 / Slack / Google Docs /
企业微信),下游把「经过授权、带证据、有预算」的 ContextPackage 交付给 agent
应用与 IM bot(飞书群聊问答优先)。

**当前状态**:D0 设计闭环阶段(pre-M0),尚无可运行代码。整体计划见 [PLAN.md](./PLAN.md)。
**当前状态**:D0 设计闭环阶段(pre-M0),尚无可运行代码。整体计划见
[PLAN.md](./PLAN.md)。

本次公开候选 bundle 包含实现权威、ADR、安全契约、PRD、Tech Spec 与四个公开参考仓的证据基线;经维护者批准并提交后,它们将与实现一同版本化。公开 prior art 仅限 Dify、RAGFlow、MaxKB、Onyx 的固定版本;ContextEngine 的安全协议依据自身需求与威胁模型独立设计,零代码复制。
本次公开候选 bundle 包含实现权威、ADR、安全契约、PRD、Tech Spec
与四个公开参考仓的证据基线;经维护者批准并提交后,它们将与实现一同
版本化。公开 prior art 仅限 Dify、RAGFlow、MaxKB、Onyx 的固定版本;
ContextEngine 的安全协议依据自身需求与威胁模型独立设计,零代码复制。

## 文档入口

- [Domain glossary](./CONTEXT.md):身份、安全、内容与生命周期术语的仓库权威。
- [Implementation Design v1.2](./docs/design/2026-07-18-context-engine-implementation-design.md):集成后的实现权威与里程碑边界。
- [四个公开参考仓证据基线](./docs/research/2026-07-19-four-public-repositories-evidence.md):四仓优势、局限、clean-room 拆解与证据缺口。
- [Threat Model](./docs/security/context-engine-threat-model.md):自有资产、信任边界、威胁与 hard oracles。
- [Program PRD](./docs/agents/prd-contextengine-implementation.md) 与 [Implementation Epic Tech Spec](./docs/specs/2026-07-19-context-engine-implementation-epic.md):需求、100 条 user stories、contract shapes 与 work packages。
- [D0 Baseline Candidate](./DESIGN-BASELINE.md):当前候选状态与尚未关闭的 evidence gates。

当前只有固定 commit 的四仓静态证据与仓库内设计拆解;PostgreSQL RLS、filtered ANN 和飞书 capability 的 disposable evidence spikes 尚未完成,因此不声称已有动态可行性或产品能力验证。
- [Domain glossary](./CONTEXT.md):身份、安全、内容与生命周期术语的仓库
权威。
- [Architecture Decision Record index](./docs/decisions/README.md):实现
边界、依赖方向、禁止捷径与重访触发器。
- [Implementation Design v1.2](./docs/design/2026-07-18-context-engine-implementation-design.md):
集成后的实现权威与里程碑边界。
- [四个公开参考仓证据基线](./docs/research/2026-07-19-four-public-repositories-evidence.md):
四仓优势、局限、clean-room 拆解与证据缺口。
- [Threat Model](./docs/security/context-engine-threat-model.md):自有资产、
信任边界、威胁与 hard oracles。
- [Program PRD](./docs/agents/prd-contextengine-implementation.md) 与
[Implementation Epic Tech Spec](./docs/specs/2026-07-19-context-engine-implementation-epic.md):
需求、100 条 user stories、contract shapes 与 work packages。
- [D0 Baseline Candidate](./DESIGN-BASELINE.md):当前候选状态与尚未关闭的
evidence gates。

当前只有固定 commit 的四仓静态证据与仓库内设计拆解;PostgreSQL RLS、
filtered ANN 和飞书 capability 的 disposable evidence spikes 尚未完成,
因此不声称已有动态可行性或产品能力验证。

## 为什么做这个

现有知识库产品回答的是「怎么存、怎么搜」;RAG 工具链回答的是「怎么找到最近的 chunk」。都没有回答两个更难的问题:
现有知识库产品回答的是「怎么存、怎么搜」;RAG 工具链回答的是「怎么找到最近的
chunk」。都没有回答两个更难的问题:

1. **这个 audience 此刻有权知道什么?** —— 索引只产生 `CandidateRef`;sealed `ContextRuntime.resolve` 必须经 `AuthorizationKernel` 执行 exact authorization 和字段投影,得到 `AuthorizedProjection` 后,才能进入 Runtime 内的水合、精排、相关性模型和装箱。BotDelivery 的生成模型只接收由当前 audience-bound ContextPackage 派生的 `AuthorizedModelInput`。Live/Mirrored/Weak 三类 SourceAclEvidence 各有明确语义,Weak 绝不是强 ACL 故障时的 fallback。
2. **知识库由谁来组织?** —— Agent 承担可自动化的组织工作(语义去重、过期标记、术语沉淀),用户负责 audit;所有 AI 产物先提案、经确认、再以独立的不可变 `CurationSnapshot` 原子发布,绝不修改已发布的内容 Revision。
1. **这个 audience 此刻有权知道什么?** —— 索引只产生
`CandidateRef`;sealed `ContextRuntime.resolve` 必须经
`AuthorizationKernel` 执行 exact authorization 和字段投影,得到
`AuthorizedProjection` 后,才能进入 Runtime 内的水合、精排、相关性模型和
装箱。BotDelivery 的生成模型只接收由当前 audience-bound ContextPackage
派生的 `AuthorizedModelInput`。Live/Mirrored/Weak 三类
SourceAclEvidence 各有明确语义,Weak 绝不是强 ACL 故障时的 fallback。
2. **知识库由谁来组织?** —— Agent 承担可自动化的组织工作(语义去重、过期
标记、术语沉淀),用户负责 audit;所有 AI 产物先提案、经确认、再以独立的
不可变 `CurationSnapshot` 原子发布,绝不修改已发布的内容 Revision。

## 核心在线契约

`ContextRuntime.resolve(AuthenticatedInvocation, TrustedDeliveryContext, Acquire | Continue | OpenCitation)` 是 Runtime 唯一公开能力,HTTP 是 V1 服务端 ingress,TypeScript SDK 是 generated HTTP client;MCP 只在真实 caller 出现后激活。Continue 的 token 绑定 principal、one-shot 且累计预算;CitationOpenRef 本身不授权,每次打开都重新认证与授权。
`ContextRuntime.resolve(AuthenticatedInvocation, TrustedDeliveryContext,
Acquire | Continue | OpenCitation)` 是 Runtime 唯一公开能力,HTTP 是 V1 服务端
ingress,TypeScript SDK 是 generated HTTP client;MCP 只在真实 caller 出现后
激活。Continue 的 token 绑定 principal、one-shot 且累计预算;CitationOpenRef
本身不授权,每次打开都重新认证与授权。

IM 交付由受信 `BotDelivery` 深模块完成。它不在 wire body 自报 trusted audience,而是通过认证 metadata 传递 opaque `DeliveryEvidenceRef`,由 ingress 兑换 `TrustedDeliveryContext` / `AudienceSnapshot`;群公开和提问者私有内容分别 resolve,外部效果均通过 `ActionPlane.prepare` + `perform`。
IM 交付由受信 `BotDelivery` 深模块完成。它不在 wire body 自报 trusted
audience,而是通过认证 metadata 传递 opaque `DeliveryEvidenceRef`,由 ingress
兑换 `TrustedDeliveryContext` / `AudienceSnapshot`;群公开和提问者私有内容分别
resolve,外部效果均通过 `ActionPlane.prepare` + `perform`。

## 三条硬底线(release veto,不是分数)

- 无授权证据泄漏 = 0(Unauthorized Evidence = 0)
- 跨租户影响 = 0(wrong-Organization effect = 0)
- 缺失租户上下文一律 fail closed

任何功能收益不能抵消其中任何一条的失败。每次发布按版本化 catalog 报告 `PASS / FAIL / NOT_ACTIVE / NOT_APPLICABLE`,并把 capability coverage 单独列出;未激活能力不能冒充通过。
任何功能收益不能抵消其中任何一条的失败。每次发布按版本化 catalog 报告
`PASS / FAIL / NOT_ACTIVE / NOT_APPLICABLE`,并把 capability coverage
单独列出;未激活能力不能冒充通过。

## License

Expand Down
3 changes: 2 additions & 1 deletion docs/agents/domain.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@ Before exploring or changing the system, read:
- root `CONTEXT.md` for the domain glossary;
- `docs/design/2026-07-18-context-engine-implementation-design.md` for the
current implementation authority;
- relevant ADRs under `docs/decisions/` for accepted architectural decisions;
- `docs/decisions/README.md` for the implementation boundary ADR baseline and
relevant ADRs under `docs/decisions/` for accepted refinements;
- `docs/research/2026-07-19-four-public-repositories-evidence.md` for the
allowlisted public prior-art evidence;
- `PLAN.md` for the public implementation roadmap.
Expand Down
11 changes: 6 additions & 5 deletions docs/decisions/0001-adopt-doc-steward-standard.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@ description: >

## Context

ContextEngine is a new multi-process context-delivery system. Agent-facing documentation (the files an
AI coding agent loads before seeing a task) needs a consistent house standard from
day one so facts land at the right altitude, guardrails are always-resident, and
volatile values are never copied.
ContextEngine is a new multi-process context-delivery system. Agent-facing
documentation (the files an AI coding agent loads before seeing a task) needs a
consistent house standard from day one so facts land at the right altitude,
guardrails are always-resident, and volatile values are never copied.

## Decision

Expand All @@ -40,4 +40,5 @@ We will adopt the **doc-steward** standard:
Docs can be audited with doc-steward's read-only EVALUATE
(`doc_lint --target .`) and low-risk fixes applied via `/doc-steward-apply`.

~~TODO_DECISION: exact frontend/backend/database stack~~ — resolved 2026-07-18 by ADR-0005 (Python 3.13 + FastAPI + SQLAlchemy + PostgreSQL 17 + pgvector).
~~TODO_DECISION: exact frontend/backend/database stack~~ — resolved 2026-07-18
by ADR-0005 (Python 3.13 + FastAPI + SQLAlchemy + PostgreSQL 17 + pgvector).
27 changes: 22 additions & 5 deletions docs/decisions/0002-bot-gateway-outside-engine.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: adr-0002-bot-delivery-outside-engine
version: "1.2.0"
version: "1.2.1"
description: >
Record the decision that IM bot answering lives in a separate trusted Bot
application containing the BotDelivery Module, not inside the engine and not
Expand Down Expand Up @@ -38,17 +38,34 @@ It asks a trusted identity Adapter for a per-resolve opaque
and consumes the resulting audience-bound ContextPackage. Public-group and
asker-private deliveries use separate references and separate resolve calls.

## Rationale

Keeping generation and IM protocol handling in a caller preserves
`ContextPackage` as the engine's complete online boundary. It also keeps model
and channel-specific failures outside the Runtime authorization kernel while
still making their cleartext and effect boundaries explicit members of the
delivery trusted computing base.

## Considered Alternatives

- Generation inside the engine — rejected: re-draws every contract (Package → Answer),
mixes hallucination concerns into the security kernel's responsibility surface.
- Generation inside the engine — rejected: re-draws every contract
(Package → Answer) and mixes hallucination concerns into the security
kernel's responsibility surface.
- IM as a fourth transport — rejected: triggers the full new-transport security
revisit (canonical parity, closed wire schema, denied/not-found equivalence) for a
surface that cannot carry the full Package contract anyway.
revisit (canonical parity, closed wire schema, denied/not-found equivalence)
for a surface that cannot carry the full Package contract anyway.

## Consequences

BotDelivery becomes the engine's first real caller, exercising the Package
contract (citations, TTL, gaps, model egress, and write effects) before any
external caller exists. ADR-0013 owns the trusted delivery evidence, egress, and
capability taxonomy; ADR-0003 owns group intersection semantics.

## Revisit trigger

Reopen this boundary only if an implemented caller demonstrates that an
audience-bound ContextPackage cannot support a required product surface, and a
new threat-model review shows how moving generation or delivery responsibility
would preserve the same authorization, egress, and effect oracles. Latency or
deployment convenience alone is not sufficient.
21 changes: 17 additions & 4 deletions docs/decisions/0006-engine-delivers-context-not-answers.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: adr-0006-engine-delivers-context-not-answers
version: "1.1.0"
version: "1.1.1"
description: >
Record the engine boundary: the only online deliverable is the ContextPackage;
generation, planning, and write tools live in upper Agent Runtime applications.
Expand Down Expand Up @@ -31,9 +31,22 @@ upper-layer Agent Runtime applications (first instance: BotDelivery, ADR-0002).
Refunds, order changes, message sending stay in the separate Action Plane
(ADR-0011).

## Rationale

Context selection and authorization have deterministic security outcomes;
answer generation and external effects do not. Ending at ContextPackage makes
the engine independently testable and prevents model or action concerns from
creating a second delivery path around Runtime policy, budget, provenance, and
audit.

## Consequences

The engine can be evaluated with deterministic retrieval metrics and security
oracles; generation quality is the caller's concern. Revisit trigger: a caller
demonstrates that Package-level delivery structurally cannot serve a required
product surface.
oracles; generation quality is the caller's concern.

## Revisit trigger

Reopen only when an implemented caller demonstrates, with a concrete contract
counterexample, that Package-level delivery structurally cannot serve a
required product surface. Any replacement decision must retain a single
authorized online deliverable and keep write authority outside context reads.
24 changes: 20 additions & 4 deletions docs/decisions/0008-modular-monolith-plus-worker.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: adr-0008-modular-monolith-plus-worker
version: "1.2.0"
version: "1.2.1"
description: >
Record the process topology: a modular-monolith engine API plus an independent
worker, and one justified Bot delivery application process from M2; no
Expand All @@ -25,9 +25,10 @@ closed at both the API and worker boundaries.
## Decision

Before M2 the engine has two processes: the API service and an independent
worker, sharing one domain package. Async work flows through a transactional outbox + job table
(SKIP LOCKED) with a server-minted signed WorkerLease (WORKER-LEASE-007) and
org-scoped idempotency keys. The lease binds Organization, job, operation,
worker, sharing one domain package. Async work flows through a transactional
outbox plus job table (`SKIP LOCKED`) with a server-minted signed WorkerLease
(WORKER-LEASE-007) and org-scoped idempotency keys. The lease binds
Organization, job, operation,
source, optional resource/revision, ServiceActor/workload, policy epoch,
optional audience, idempotency key, lease generation, issued-at, expiry, and
nonce; redemption checks every claim against the durable job row.
Expand All @@ -40,10 +41,25 @@ service extraction occurs until a measured bottleneck or isolation requirement
triggers a revisit; seams are re-examined when the second Adapter of a kind
appears.

## Rationale

The API and worker need different scheduling and failure behavior but not
different domain implementations. Sharing one domain package avoids premature
distributed identity and policy propagation while the independent worker still
contains long-running Supply work behind an exact job capability.

## Consequences

Trusted context propagates through explicit request evidence or the job/lease
protocol; the worker's Organization and ServiceActor context is
transaction-local and never ambient or borrowed from a triggering user. Engine
and Bot applications may ship from one repository/release while remaining
separate process and import boundaries.

## Revisit trigger

Reopen the process boundary only with measured isolation, scaling, deployment,
or reliability evidence that the API-plus-worker topology cannot satisfy. A
second real Adapter of a kind triggers seam review, not automatic service
extraction. Any split must specify authenticated transport, Organization and
ActorContext propagation, retries, idempotency, and credential ownership first.
17 changes: 16 additions & 1 deletion docs/decisions/0011-read-write-plane-separation.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: adr-0011-read-write-plane-separation
version: "1.2.0"
version: "1.2.1"
description: >
Record the separation of context reads from action writes: ContextAccessTicket
and ActionTicket share an identity chain but different audiences and are never
Expand Down Expand Up @@ -28,6 +28,13 @@ different audience claim, never interchangeable (ACTION-SEPARATION-014). Low-ris
actions (e.g. plain-text group replies) may use a pre-approved policy tier:
the approval flow lightens, ticket separation and audit never do.

## Rationale

Read authority proves only that a caller may receive declared context. It says
nothing about whether that caller may mutate an external destination. Distinct
ticket audiences make this boundary cryptographically checkable and prevent a
read-capable component from becoming a confused deputy for writes.

## Consequences

The engine's attack surface stays read-only; every side effect has its own
Expand All @@ -41,3 +48,11 @@ use distinct tickets and idempotency keys linked only by DeliveryAttemptRef;
successful replay returns the stored receipt, while an ambiguous provider
attempt is reconciled under the original id rather than retried with a new
ticket.

## Revisit trigger

This separation may be refined when a real external effect requires a new
capability class or approval tier. It must not be reopened merely to reuse a
read ticket: any proposal still has to prove different read and effect
audiences, one-effect payload/destination binding, and zero business effect for
wrong-audience use.
18 changes: 17 additions & 1 deletion docs/decisions/0012-sealed-authorization-projection-pipeline.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: adr-0012-sealed-authorization-projection-pipeline
version: "1.0.1"
version: "1.0.2"
description: >
Seal Runtime ordering so content-free CandidateRef values become
AuthorizedProjection values before content-bearing relevance, assembly, or
Expand Down Expand Up @@ -79,10 +79,26 @@ Source ACL evidence is a closed union:
SourcePolicy fixes the mode at SourceVersion activation. A live or mirrored
failure never downgrades to weak.

## Rationale

Security ordering is a domain invariant, not application wiring. Keeping
policy, audit, budget, provenance, and exact projection inside one sealed
Kernel makes bypass structurally unavailable while leaving infrastructure and
provider variation behind narrow seams.

## Consequences

The Interface has high depth: all callers get the same security order and cannot
rearrange it. Parent and neighbor expansion costs more authorization work.
Provider contract tests must prove content-free discovery and bounded
projection. Live outages may produce a typed partial/empty result instead of an
availability-oriented downgrade.

## Revisit trigger

Reopen the kernel-versus-seam allocation only when a second production
implementation demonstrates a variation that cannot satisfy the current deep
Interface, or a security finding proves the sealed order insufficient. A
replacement must preserve `CandidateRef -> AuthorizationKernel ->
AuthorizedProjection`, mandatory policy/audit/budget/provenance behavior, and
the absence of production disable flags or no-op dependencies.
Loading