feat: publish current-main external lineage contract

CHANGELOG.d/external-lineage-contract.md

@@ -0,0 +1,12 @@
+# External email/project lineage contract
+
+- Add a strict, versioned external analysis contract for future Naruon and separately governed consumer use.
+- Export immutable request/result types, strict parsing, canonical serialization, deterministic digests, stable errors, and the store-agnostic `analyze_external_lineage` package entry point.
+- Accept only bounded caller-authorized opaque evidence references; no provider credentials, mailbox access, persistence, provider mutation, or direct application-database integration is introduced.
+- Preserve caller-observed RFC/provider/manual parent relations separately from inferred reconstructed continuation.
+- Exclude caller-observed children from alternative inferred-parent scoring, optional model disclosure, and inferred-pair budget while retaining them as candidate history for later records.
+- Enforce available-time knowledge cutoffs and disclose excluded evidence without substituting later facts.
+- Reject explicit-parent cycles and candidate-pair work above the caller-approved limit before optional LLM/provider activity.
+- Expose exact active channel scores, weights, contributions, LLM availability state, proposed project groupings, and deterministic result digests.
+- Require a provenance-bearing fast-mlsirm channel-weight estimate for inferred edges; without it, return observed edges plus an explicit unavailable limitation.
+- Add JSON Schema Draft 2020-12, union-free ADR 0239, APA 7th doctoring, and focused TDD coverage.

backend/app/analysis_run_start.py

@@ -24,7 +24,6 @@
     AnalysisRunCreateError,
     fetch_visible_analysis_run,
 )
-from backend.app.post_eligibility import SOURCE_POST_ELIGIBILITY_SQL
 from backend.app.analysis_run_outbox import (
     latest_outbox_delivery_is_claimed,
     latest_outbox_delivery_is_delivered,
@@ -34,7 +33,11 @@
     load_estimated_channel_weights,
     records_from_source_posts,
 )
-from lineageweave.adjudication_client import AdjudicationClient
+from backend.app.post_eligibility import SOURCE_POST_ELIGIBILITY_SQL
+from lineageweave.adjudication_client import (
+    AdjudicationClient,
+    AdjudicationClientError,
+)
 from lineageweave.http_client import HttpClientError, post_json
 from lineageweave.lineage_persistence import lineage_edge_specs
 from lineageweave.models import Edge
@@ -83,7 +86,13 @@ def judge(self, candidate_label: str, record_label: str) -> float:
         """Score one candidate pair, typing provider failures as such."""
         try:
             return self._inner.judge(candidate_label, record_label)
-        except (HttpClientError, OSError, ValueError, TypeError) as exc:
+        except (
+            AdjudicationClientError,
+            HttpClientError,
+            OSError,
+            ValueError,
+            TypeError,
+        ) as exc:
             raise _AdjudicationProviderError(str(exc)) from exc
 
 

backend/app/main.py

@@ -165,6 +165,7 @@
 )
 from backend.app.source_post_revision import fetch_known_at_revision, parse_as_of_clock
 from lineageweave.adjudication_client import (
+    AdjudicationClientError,
     ContextualOrchestratorAdjudicationClient,
     NullAdjudicationClient,
 )
@@ -1311,7 +1312,7 @@ async def rebuild_lineage_graph(
             "Channel weights are not estimated yet. Run "
             "scripts/estimate_channel_weights.py, then rebuild again.",
         ) from exc
-    except (HttpClientError, OSError) as exc:
+    except (AdjudicationClientError, HttpClientError, OSError) as exc:
         # This can issue up to MAXIMUM_LIVE_LLM_PAIR_EVALUATIONS sequential
         # adjudication calls across the whole corpus (lineage_ingestion.py);
         # a transient orchestrator hiccup on any one of them must not

docs/adr/0239-external-email-project-lineage-contract.md

@@ -0,0 +1,59 @@
+# ADR 0239: Publish a bounded external email/project lineage contract
+
+- Status: Accepted
+- Date: 2026-08-21
+
+## Context
+
+Naruon owns customer mail/calendar/file access, canonical message/thread identities, projects, tasks, commitments, provider credentials, authorization, and provider mutations. LineageWeave owns evidence-fused lineage reconstruction and the provenance explaining that reconstruction. Future integration must not give either product direct SQL access to the other's application database, duplicate source authority, or depend on a mutable branch/submodule.
+
+Email thread facts also have different truth semantics from reconstructed semantic continuation. RFC `Message-ID`, `References`, and `In-Reply-To` evidence may establish a caller-observed reply relation, while LineageWeave text/temporal/project signals produce an inferred relation. Flattening both into one unexplained score would make buyer correction and audit impossible.
+
+## Decision
+
+LineageWeave publishes contract version `1.0.0` through:
+
+- `lineageweave.external_lineage_contract` for strict immutable request/result shapes, canonical serialization, bounds, and deterministic digests;
+- `lineageweave.external_lineage_analysis` for adapting caller-authorized evidence to the existing reconstruction kernel.
+
+The initial implementation is a store-agnostic Python package boundary. It performs no database, mailbox, provider, or network operation. A later service or Naruon plugin adapter must preserve the same JSON Schema and truth boundaries.
+
+Inferred edges additionally require a provenance-bearing
+`ChannelWeightEstimate` produced by the repository's fast-mlsirm measurement
+boundary. The estimate is an injected execution dependency, not caller JSON:
+the external evidence contract cannot assert its own fusion weights. When no
+estimate is available, the adapter still returns caller-observed edges and an
+explicit `channel_weights_unavailable` limitation, but produces no inferred
+edge. The LLM channel is active only when the estimate explicitly includes an
+`llm` item; an available model without such measurement remains unavailable
+for this run.
+
+The caller supplies opaque evidence references, bounded text labels, occurrence and availability clocks, an optional secondary key, an optional project reference, and an optional caller-observed parent relation. Explicit observed parent relations replace an inferred parent for the same child and must form an acyclic graph. Reconstructed continuation remains `inferred`. Project groupings remain `proposed`.
+
+An admitted child with an explicit observed parent is not rescored for an alternative inferred parent and consumes no optional LLM/provider call or inferred-pair budget. The record remains in temporal history and may still be an eligible candidate parent for a later record. This preserves observed authority without weakening downstream lineage reconstruction.
+
+The caller also supplies `maximum_pair_evaluations` in the bounded policy. The package computes the exact inferred candidate-parent pair count after knowledge-cutoff filtering, excluding children whose parent is already caller-observed, and rejects work above the declared budget before any optional LLM/provider call. Contract v1 caps the declared budget at 5,000 pairs.
+
+Historical requests include evidence only when:
+
+```text
+available_at <= knowledge_cutoff
+```
+
+Evidence becoming available after the cutoff is excluded even when it describes an earlier occurrence.
+
+## Consequences
+
+- Naruon can eventually consume a released artifact without exposing credentials or application tables.
+- RFC reply/thread evidence stays distinguishable from semantic lineage.
+- Caller-observed children are never disclosed to an optional model merely to calculate an inferred edge that would be discarded.
+- The optional LLM channel is explicit as `not_requested`, `unavailable`, or `completed`; missing output is never zero.
+- Missing or malformed psychometric weight provenance yields no inferred edge; no default, equal, or caller-authored weight is substituted.
+- Canonical serialization and SHA-256 digesting are deterministic for a given request or result. Repeatability of model-backed scores additionally requires a pinned LineageWeave release, adjudicator implementation, provider/model revision, and model-side determinism policy.
+- Explicit parent cycles and analysis work above the caller-approved pair budget fail closed before inference.
+- Project evidence can inform Naruon without mutating authoritative project/task/provider state.
+- The single generic secondary key reflects the current core kernel. Multiple independent typed secondary-key channels remain a future contract revision rather than being silently flattened.
+
+## References
+
+See `docs/doctoring/EXTERNAL_LINEAGE_CONTRACT_REFERENCES.md`.

docs/contracts/README.md

@@ -0,0 +1,13 @@
+# Integration contracts
+
+LineageWeave publishes strict, versioned contracts for separately governed consumers. These contracts do not grant source access and do not replace each consumer's authorization, persistence, provider, or audit authority.
+
+## External lineage analysis v1
+
+- JSON Schema: `external-lineage-analysis-v1.schema.json`
+- Synthetic request: `external-lineage-analysis-v1.example.json`
+- Python parser and immutable types: `lineageweave.external_lineage_contract`
+- Store-agnostic execution adapter: `lineageweave.external_lineage_analysis`
+- Decision record: `docs/adr/0239-external-email-project-lineage-contract.md`
+
+A consumer must submit only bounded evidence it is already authorized to disclose. Outputs retain opaque caller references and explicit `observed`, `inferred`, or `proposed` truth boundaries. The contract performs no source-system access or provider mutation.

docs/contracts/external-lineage-analysis-v1.authorization.md

@@ -0,0 +1,5 @@
+# External lineage analysis v1 authorization contract
+
+LineageWeave does not infer authorization from an opaque reference, source kind, group, project, or caller identity. The caller must authorize evidence before projection and must reauthorize any source drill-through after receiving a result.
+
+The package does not accept provider bearer tokens, browser cookies, mailbox credentials, database DSNs, or caller SQL. A future remote service must use its own audience-scoped service credential and may not forward an end-user token to model providers.

docs/contracts/external-lineage-analysis-v1.consumer-checklist.md

@@ -0,0 +1,11 @@
+# External lineage analysis v1 consumer checklist
+
+- Validate the published JSON Schema before sending or accepting payloads.
+- Submit only evidence the calling principal is authorized to disclose for the declared purpose.
+- Use opaque caller-owned references; never send provider credentials or database locators.
+- Bind historical work to a knowledge cutoff and preserve each record's availability time.
+- Keep RFC/provider thread observations separate from inferred semantic/project lineage.
+- Treat project projections as proposals until the caller's own policy or reviewer accepts them.
+- Preserve the returned artifact digest, LineageWeave version, limitations, and channel evidence.
+- Fail closed on incompatible contract versions.
+- Keep normal caller operation available when LineageWeave is unavailable.

docs/contracts/external-lineage-analysis-v1.data-minimization.md

@@ -0,0 +1,12 @@
+# External lineage analysis v1 data minimization
+
+Consumers should prefer the minimum evidence needed for a declared analysis scope:
+
+- opaque evidence and grouping references;
+- offset-aware occurrence and availability times;
+- RFC/provider relation evidence when present;
+- bounded subject/title labels or caller-computed text features;
+- optional project or secondary-key references;
+- optional participant, body, or attachment evidence only when the caller's purpose and policy explicitly permit it.
+
+The contract does not require a mailbox dump, full thread body, recipient list, provider URL, or attachment bytes. Omitted evidence is unavailable and cannot appear in output.

docs/contracts/external-lineage-analysis-v1.example.json

@@ -0,0 +1,41 @@
+{
+  "contract_version": "1.0.0",
+  "analysis_id": "analysis:synthetic-email-lineage-001",
+  "analysis_scope_code": "email_lineage",
+  "knowledge_cutoff": "2026-08-20T09:30:00Z",
+  "policy": {
+    "candidate_window": 50,
+    "maximum_pair_evaluations": 1000,
+    "minimum_fused_score": 0.3,
+    "allow_llm": false
+  },
+  "records": [
+    {
+      "evidence_ref": "email:synthetic-001",
+      "group_ref": "workspace:synthetic",
+      "source_kind_code": "email",
+      "truth_status_code": "observed",
+      "label": "Synthetic proposal review",
+      "occurred_at": "2026-08-20T09:00:00Z",
+      "available_at": "2026-08-20T09:01:00Z",
+      "secondary_key": "provider-thread:synthetic",
+      "project_ref": "project:synthetic",
+      "explicit_parent": null
+    },
+    {
+      "evidence_ref": "email:synthetic-002",
+      "group_ref": "workspace:synthetic",
+      "source_kind_code": "email",
+      "truth_status_code": "observed",
+      "label": "Re: Synthetic proposal review",
+      "occurred_at": "2026-08-20T09:05:00Z",
+      "available_at": "2026-08-20T09:06:00Z",
+      "secondary_key": "provider-thread:synthetic",
+      "project_ref": "project:synthetic",
+      "explicit_parent": {
+        "evidence_ref": "email:synthetic-001",
+        "relation_code": "rfc_reply"
+      }
+    }
+  ]
+}

docs/contracts/external-lineage-analysis-v1.limitations.md

@@ -0,0 +1,11 @@
+# External lineage analysis v1 limitations
+
+- The contract does not read IMAP, JMAP, CalDAV, Naruon, or other provider systems.
+- It does not authenticate users, authorize tenant access, persist jobs, or retry remote work.
+- It does not make semantic lineage equivalent to RFC reply/thread identity.
+- It does not turn project groupings, responsibility context, or reconstructed edges into authoritative caller facts.
+- It does not infer unavailable evidence as a zero-valued channel.
+- It does not guarantee causal relations; reconstructed continuation is an evidence-weighted related-history hypothesis.
+- Canonical request/result serialization and digests are deterministic, but an optional remote adjudication channel is not automatically repeatable unless the consumer pins the LineageWeave artifact, adjudicator, provider/model revision, and determinism policy.
+- Contract v1 does not carry a remote provider/model receipt inside the result; production wrappers must retain that provenance alongside the result digest before model-backed integration is enabled.
+- It does not replace Naruon's canonical email identity, project/task/commitment state, provider mutation, or reconciliation authority.

docs/contracts/external-lineage-analysis-v1.operability.md

@@ -0,0 +1,5 @@
+# External lineage analysis v1 operability boundary
+
+The pure package entry point is synchronous and bounded. Remote or model-backed production use must wrap it in a separately reviewed service or plugin lifecycle with durable idempotency, cancellation, timeout, retry classification, rate limiting, resource budgets, artifact retention, OpenTelemetry signals, and user-visible degraded states.
+
+A consumer must not call optional model-backed pair adjudication directly on an unbounded web request path. LineageWeave #289 tracks the durable asynchronous reconstruction requirement for product persistence, and Naruon #1437 requires an equivalent consumer-side job receipt before integration is enabled.