Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions docs/adr/0064-lineage-evidence-and-tree-assembly.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

- Status: Accepted
- Date: 2026-08-19
- Amended by: [ADR 0200](0200-channel-weight-reconciliation.md) and
[ADR 0205](0205-tepp-lineage-anchor.md)

## Context

Expand All @@ -16,9 +18,10 @@ or promoting an inferred relation to fact.
- Treat every input record as a fallible mention and every accepted edge as a
lineage instance supported by evidence, not as a proven business fact.
- Fuse independent temporal, secondary-key, text/embedding, and optional LLM
channels through the RankWeave weighted convex fusion contract. A missing
channel is dropped and weights are renormalized; it is never replaced with a
fabricated negative or score.
channels through the RankWeave weighted convex fusion contract. Resolve a
missing channel before loading the exact calibrated active-channel vector;
never repair or renormalize a vector estimated for another channel set, and
never replace a missing channel with a fabricated negative or score.
- Keep the channel-score breakdown and provenance on every candidate decision.
Candidates below the minimum fused-score floor remain roots rather than being
force-attached.
Expand Down
20 changes: 12 additions & 8 deletions docs/adr/0172-event-lineage-channel-evidence.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
**Status:** Accepted
**Date:** 2026-08-21
**Issue:** [#274](https://github.com/ContextualWisdomLab/LineageWeave/issues/274)
**Amended by:** [ADR 0200](0200-channel-weight-reconciliation.md) and
[ADR 0205](0205-tepp-lineage-anchor.md)

## Context

Expand Down Expand Up @@ -33,8 +35,9 @@ authoritative; PROV-O/RDF export is a projection.
`lineage_signal_temporal`, `lineage_signal_secondary_key`,
`lineage_signal_text`, `lineage_signal_llm`. The LLM row is omitted
when the adjudication client is unavailable; it is never fabricated.
3. Weights are the normalized active weights actually used
(`reconstruct.active_weights`). Contribution is `weight * score` and
3. Weights are the exact calibrated active-channel vector validated by
`reconstruct.active_weights`; no different channel set is renormalized into
it. Contribution is `weight * score` and
must reconcile with `fused_score` within
`CHANNEL_EVIDENCE_TOLERANCE` (`1e-6`).
4. Live Event Lineage is replaced atomically. A singleton
Expand All @@ -46,10 +49,10 @@ authoritative; PROV-O/RDF export is a projection.
The administrator-triggered live rebuild and PostgreSQL import pass the
configured contextual-orchestrator adjudication client through the same
reconstruction boundary only when the exact candidate-pair count is at
most 5,000. Larger snapshots drop the LLM channel before any provider call
and renormalize the remaining weights. This is an operational work bound,
not a model-quality or provider-ranking heuristic. One rebuild never mixes
LLM and non-LLM weight profiles across edges.
most 5,000. Larger snapshots drop the LLM channel before loading the
separately calibrated three-channel profile. This is an operational work
bound, not a model-quality or provider-ranking heuristic. One rebuild never
mixes LLM and non-LLM weight profiles across edges.
5. `GET /api/lineage` returns an additive `channel_evidence` collection
on each visible edge (`signal_code`, `signal_label`, `score`,
`weight`, `contribution`, `rank`) ordered by contribution, then
Expand All @@ -67,8 +70,9 @@ authoritative; PROV-O/RDF export is a projection.
inference from source evidence.
- A later rebuild rewrites live Event Lineage as a whole; historic
meaning is not silently mutated in place.
- Completeness is lower when the LLM channel is unavailable, matching
ADR 0064: missing channels are dropped and weights renormalize.
- Completeness is lower when the LLM channel is unavailable; the exact
three-channel profile is loaded instead of renormalizing a four-channel
estimate.

## References

Expand Down
3 changes: 2 additions & 1 deletion docs/product-requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,8 @@ invented facts, scores, weights, or identities.
- Reconstruct bounded post-to-post parentage from independent channels.
- Keep Event Lineage separate from ontology/KG edges.
- Persist reconstruction profile and participating channel evidence.
- Drop unavailable channels and fail closed on malformed calibrated weights.
- Resolve unavailable channels before loading the exact calibrated active-set
vector; never renormalize a vector calibrated for a different channel set.
- Never label an inferred lineage edge causal or authoritative without
separate evidence.

Expand Down
28 changes: 15 additions & 13 deletions docs/product-technical-gap-baseline.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Product & Technical Gap Baseline

> Dashboard delivery snapshot: 2026-08-25 23:05 KST. Protected `main` was
> Dashboard delivery snapshot: 2026-08-25 23:33 KST. Protected `main` was
> `04e6b610655d0db91d5f7ba9486bdda1440e0b19`. This local branch is not
> protected-main release evidence.

Expand Down Expand Up @@ -60,16 +60,17 @@ only aggregate, non-identifying evidence to this repository.

### Exact open-PR boundary

At this snapshot there were 5 open PRs and 10 open issues. Exact observed heads
are listed below. PRs #579, #629, #631, #632, and #636 all target
protected `main`; each had zero unresolved review threads. Four were
`REVIEW_REQUIRED`; #629 had no formal review decision. All had queued required
workflows and remain blocked on hosted gates and independent review. These
At this snapshot there were 6 open PRs and 10 open issues. Exact observed heads
are listed below. PRs #579, #629, #631, #632, and #636 target protected
`main`; stacked PR #638 targets #636's branch. The five protected-main PRs had
zero unresolved review threads. PRs #579, #631, and #632 were
`REVIEW_REQUIRED`; #629, #636, and new #638 had no formal review decision.
Hosted gates and independent review remain required. These
observations are not merge readiness. Re-fetch exact heads,
unresolved threads, checks, approvals, rulesets, and merge SHA before any
lifecycle claim.

> Audit snapshot: 2026-08-25 23:21 KST (refreshed by the autonomous merge
> Audit snapshot: 2026-08-25 23:33 KST (refreshed by the autonomous merge
> loop). This repository records synthetic fixtures and aggregate,
> non-identifying runtime evidence only. Open PRs and local checks are not
> protected-default-branch release evidence. Identifying post identifiers,
Expand All @@ -79,17 +80,18 @@ lifecycle claim.
## 1. Exact-head and governance evidence

The protected default branch was `04e6b610655d0db91d5f7ba9486bdda1440e0b19`
when this baseline was refreshed. The live queue contained 5 open PRs and 10
when this baseline was refreshed. The live queue contained 6 open PRs and 10
open issues. The exact-head inventory below supersedes older per-PR snapshots
elsewhere in this document; those older rows remain useful historical delivery
context only.

| PR | Exact observed head | Merge/check state at this snapshot |
| ---: | --- | --- |
| #636 | `78d87a18` | publishes the calibrated external lineage contract and fails closed when calibrated weights do not exactly match active channels; zero unresolved threads; queued checks and independent review remain required |
| #638 | `51c2be9d` | stacked on #636; makes the shared reconstruction boundary reject mismatched active-channel profiles instead of dropping and renormalizing calibrated weights; hosted checks and independent review remain required |
| #636 | `cc6d64a0` | publishes the calibrated external lineage contract and fails closed when calibrated weights do not exactly match active channels; zero unresolved threads; queued checks and independent review remain required |
| #632 | `bb680329` | preserves graph-fact source provenance; zero unresolved threads; queued checks and independent review remain required |
| #631 | `c0022c97` | refreshes product-gap and CI lifecycle governance; zero unresolved threads; queued checks and independent review remain required |
| #629 | `883d1ff7` | releases provider work and bounds landing reads; zero unresolved threads, no formal review decision, and queued checks remain |
| #629 | `ac38c652` | releases provider work and bounds landing reads; zero unresolved threads, no formal review decision, and queued checks remain |
| #579 | `689a21b6` | persists leftover interaction-map coordinates and owns ADR 0211; zero unresolved threads; queued checks and independent review remain required |

No row above is merge evidence. Immediately before any lifecycle action,
Expand Down Expand Up @@ -362,7 +364,7 @@ this file per §3.5 of the prior snapshot).

| Gap | Current evidence | Acceptance requirement |
| --- | --- | --- |
| Protected release | 5 open PRs at the 2026-08-25 exact-head snapshot: #579, #629, #631, #632, and #636. All target `main`; each retains hosted checks and independent-review gates | Terminal exact-head checks, no unresolved threads, independent exact-head approvals, protected squash-merge SHA |
| Protected release | 6 open PRs at the 2026-08-25 exact-head snapshot: #579, #629, #631, #632, #636, and stacked #638. The first five target `main`; each retains hosted checks and independent-review gates | Terminal exact-head checks, no unresolved threads, independent exact-head approvals, protected squash-merge SHA |
| Evidence-grounded operations workspace | Protected-main #614 delivers governed semantic Ask, live Similar VOC, disjoint pending/failed analysis metrics, full Storybook state inventory, and current desktop/mobile screenshot evidence. Authorized-corpus backfill acceptance remains unavailable | Perform authenticated authorized-corpus acceptance with aggregate evidence and retain fail-closed no-match behavior |
| Shared frontend gate | The ADR 0109 login repair is on protected `main`; eight older branches carried the defect and received the same verified repair this loop (#521–#560) | Keep every future branch cut from post-repair bases; re-verify with frontend lint/test/build before push |
| Identifying baseline regression | `main` gap file listed real post identifiers; separately, closed #506 and pre-existing public history contain a private runtime source-table identifier, while current `main` and #507 trees are clean | Land this non-identifying rewrite, then coordinate ADR 0001 history remediation with security/privacy owners; do not reproduce the value, force-push, or delete evidence ad hoc |
Expand All @@ -374,10 +376,10 @@ this file per §3.5 of the prior snapshot).
| Knowledge Graph readability | The black evidence-node root cause is an undefined-token fallback; the design-token repair and long-label/evidence-table coverage remain only on closed, unmerged #490, not protected `main` | Recreate the token repair on a current base and deliver it through protected `main`, then verify light/dark contrast, keyboard graph navigation, full labels, and evidence tables in the authenticated rendered surface |
| Source-code lookup UX | Source state/detail codes remain evidence-bearing machine values and current detail presentation is dense | Catalog-backed display labels with raw-code provenance, compact 5W1H/source-detail hierarchy, keyboard access, and no unsupported customer/project binding |
| Calendar / Naruon | #355 delivered the projection contract; v2.17.0 wires operator consumption without forwarding the end-user token. Naruon producer, provider/consumer fixtures, and protected merge remain open (#336) | Verify observed events against the published schema without invented events; keep commitments available when the channel is unwired |
| External email/project lineage | #636 recreates the historical non-main #343 work on current `main` as a bounded store-agnostic contract. It separates observed reply facts, inferred continuation, and proposed project projection; requires calibrated host weights that exactly match active channels without silent renormalization; and leaves the Naruon consumer disabled pending immutable release | Land #636 through protected gates, publish an immutable LineageWeave artifact, then add Naruon conformance and failure-path fixtures without transferring source credentials or stale stack evidence |
| External email/project lineage | #636 recreates the historical non-main #343 work on current `main` as a bounded store-agnostic contract. It separates observed reply facts, inferred continuation, and proposed project projection; requires calibrated host weights that exactly match active channels without silent renormalization; and leaves the Naruon consumer disabled pending immutable release. Stacked #638 applies the same exact-set rule to the shared reconstruction boundary | Land #638 into #636, then land #636 through protected gates, publish an immutable LineageWeave artifact, and add Naruon conformance and failure-path fixtures without transferring source credentials or stale stack evidence |
| SKOS organization aliases | Catalog binding and chip caption live on #480 / #482 | One catalog row per corroborated org; companion caption is hint-only until bound |
| Event Lineage evidence | Channel evidence and Allen relations live on #387 / #484 | Persist channel scores, explain them in the popup, never invent a fused score |
| Scientific measurement | Durable accepted TEPP receipts and LineageWeave #614's exact accepted snapshot/cutoff/run/pair-count consumer are protected; TEPP #237 remains open, so no registered producer artifact exists yet. #387 removes inferred/default persistence weights, but several older reconstruction tests still pass hand-authored numeric dictionaries that are not estimator evidence | Land TEPP #237 through its protected gate, then replace remaining reconstruction-test constants with provenance-bearing fast-mlsirm estimates over synthetic fixtures. Retain true-parameter RMSE recovery as the acceptance bar |
| Scientific measurement | Durable accepted TEPP receipts and LineageWeave #614's exact accepted snapshot/cutoff/run/pair-count consumer are protected; TEPP #237 remains open, so no registered producer artifact exists yet. Stacked #638 removes shared-runtime dropping and renormalization of a vector calibrated for a different active channel set. Several older reconstruction tests still pass hand-authored numeric dictionaries that are not estimator evidence | Land TEPP #237 and #638 through their protected dependency gates, then replace remaining reconstruction-test constants with provenance-bearing fast-mlsirm estimates over synthetic fixtures. Retain true-parameter RMSE recovery as the acceptance bar |
| Asynchronous authorization | Protected `main` rebuilds Global Ask worker scope after the bearer token leaves the request; #468 now persists exact Keyverse organization/process-unit scope in 3NF child tables and intersects it with current affiliations | Land #468 through the protected gate; prove a second affiliation and a revoked process unit cannot widen delayed-job evidence |
| Planned-facility intent | Planned-facility relationship intent remains only on closed, unmerged #490; earlier stack-only merges were not protected delivery | Recreate the evidence-backed slice on a current base and land through protected `main` before a release claim |
| Accessibility and responsive UX | #602 delivered base post-detail modal semantics; #605 adds selected-post refocus, collapsed/hidden/inert/CSS-invisible focus exclusion across both modal types, readable evidence separators, focused tests, and desktop/mobile Storybook screenshots | Land #605 through the protected gate, then complete screen-reader and authenticated Playwright acceptance on the exact release head |
Expand Down
4 changes: 2 additions & 2 deletions lineageweave/embedding_client.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
"""Pluggable embedding channel.

The default :class:`NullEmbeddingClient` makes the channel unavailable
rather than faking a score -- ``reconstruct.active_weights`` drops and
renormalizes around any channel whose client reports ``available = False``.
rather than faking a vector; callers must use the calibrated profile for the
channels they actually execute.
:class:`ContextualOrchestratorEmbeddingClient` calls the authenticated
contextual-orchestrator ``/v1/batch/embeddings`` boundary once a credential is
set. No client in this repository calls a provider embedding endpoint directly.
Expand Down
4 changes: 2 additions & 2 deletions lineageweave/lineage_persistence.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,8 +65,8 @@ def lineage_edge_specs(

``llm`` defaults to ``None``, which ``reconstruct()`` treats as the
unavailable :class:`~lineageweave.adjudication_client.NullAdjudicationClient`
(the llm channel is then dropped and the rest renormalized, not
faked) -- callers that want the highest-weighted reasoning channel
(the supplied calibrated vector must omit the llm channel) -- callers that
want the reasoning channel
actually contributing to real reconstructions must pass a real one.

``weights`` is required and always a psychometric estimate (ADR
Expand Down
38 changes: 26 additions & 12 deletions lineageweave/reconstruct.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@

from __future__ import annotations

import math
from collections import defaultdict

import rankweave as rw
Expand All @@ -25,8 +26,10 @@
# today, TEPP when integrated) -- product paths load the persisted
# corpus estimate, and the library demo estimates from its declared
# generative design (channel_weight_estimation.estimate_fixture_channel_weights).
# Every reconstruct() caller passes weights explicitly; the llm entry
# renormalizes away when no client is configured (see active_weights()).
# Every reconstruct() caller passes weights for the exact channel set it can
# execute; a vector estimated for another set is never repaired or renormalized.

_CORE_WEIGHT_CHANNELS = frozenset({"temporal", "secondary_key", "text"})

# ponytail: only the most recent WINDOW prior records in a group are
# considered as candidate parents, bounding per-group cost to O(n*window)
Expand All @@ -47,12 +50,23 @@
def active_weights(
llm: AdjudicationClient, weights: dict[str, float]
) -> dict[str, float]:
"""Drop and renormalize the llm channel's weight when no client is configured."""
active = dict(weights)
if not getattr(llm, "available", False):
active.pop("llm", None)
total = sum(active.values())
return {channel: weight / total for channel, weight in active.items()}
"""Validate and return the calibrated vector for the exact active channels."""

expected = set(_CORE_WEIGHT_CHANNELS)
if getattr(llm, "available", False):
expected.add("llm")
if set(weights) != expected:
raise ValueError("weights must exactly match the active lineage channels")
Comment on lines +58 to +59

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve unavailable-channel renormalization

When llm is absent or reports available = False, callers supplying a vector that includes llm now receive ValueError instead of reconstructing with that unavailable channel removed. Restore dropping llm and renormalizing the remaining weights, as required for NullAdjudicationClient; otherwise the documented missing-signal fallback is broken.

AGENTS.md reference: AGENTS.md:L189-L195

Useful? React with 👍 / 👎.

Comment on lines +55 to +59

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Exact-match change is consistent across all callers

Switching from drop-and-renormalize to exact-match-or-raise could break a caller passing a four-channel vector with an unavailable llm. Every caller derives its channel set from the same available predicate active_weights uses: estimated_weight_channels, analysis_run_start (active_channels gated on _ProviderBoundaryAdjudication.available at backend/app/analysis_run_start.py:77), the demo server, and seed scripts. None passes a set that mismatches its own llm availability.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

values = tuple(weights.values())
if any(
isinstance(value, bool)
or not isinstance(value, (int, float))
or not math.isfinite(float(value))
or float(value) <= 0.0
for value in values
) or not math.isclose(sum(values), 1.0, abs_tol=1e-9):
raise ValueError("active lineage weights must be finite, positive, and sum to one")
Comment on lines +67 to +68

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Sum-to-one tolerance matches the weight loader

The new abs_tol=1e-9 sum check could in principle fail-close persisted weights, but load_estimated_channel_weights (backend/app/lineage_ingestion.py:302) already enforces the identical abs_tol=1e-9 sum check before reconstruction, and estimate_channel_weights normalizes by dividing by the total. Loaded and estimated vectors sum to 1 within float epsilon, so the stricter gate adds no new failure path.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

return dict(weights)
Comment on lines +55 to +69

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Code contradicts the mandatory pluggable-channel convention

active_weights now rejects a mismatched channel set with ValueError instead of dropping and renormalizing the unavailable channel. The AGENTS.md "Pluggable channels" section still mandates the drop-and-renormalize behavior and names this exact function, and it was not updated to match.

Prompt for agents
The mandatory convention file AGENTS.md (section "Pluggable channels: never fake a missing signal") states that Null channel clients must "make their channel dropped + renormalized (reconstruct.active_weights)". This PR intentionally changes reconstruct.active_weights to require an exact calibrated channel vector and raise ValueError instead of dropping/renormalizing, per the ADR 0200/0205 amendments. Update AGENTS.md to reflect the new fail-closed exact-vector contract so the mandatory convention file no longer describes the removed drop-and-renormalize behavior and no longer references reconstruct.active_weights as the renormalization mechanism.
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.



def _group_by(records: list[Record]) -> dict[str, list[Record]]:
Expand Down Expand Up @@ -159,10 +173,10 @@ def reconstruct(
records: every record across every group; grouping happens here.
llm: adjudication channel client; defaults to
:class:`~lineageweave.adjudication_client.NullAdjudicationClient`
(the llm channel is then dropped, not faked).
weights: per-channel fusion weights before llm-availability
renormalization. Required, and always a psychometric
estimate (ADR 0145, second amendment): the persisted
(the supplied vector must therefore omit the llm channel).
weights: fusion weights calibrated for exactly the channels this call
executes. Required, and always a psychometric estimate (ADR 0145,
second amendment): the persisted
fast-mlsirm corpus estimate on product paths, or
:func:`~lineageweave.channel_weight_estimation.estimate_fixture_channel_weights`
for the library demo. No hand-picked default exists.
Expand Down
Loading