Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions docs/adr/0064-lineage-evidence-and-tree-assembly.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

- Status: Accepted
- Date: 2026-08-19
- Amended by: [ADR 0200](0200-channel-weight-reconciliation.md) and
[ADR 0205](0205-tepp-lineage-anchor.md)

## Context

Expand All @@ -16,9 +18,10 @@ or promoting an inferred relation to fact.
- Treat every input record as a fallible mention and every accepted edge as a
lineage instance supported by evidence, not as a proven business fact.
- Fuse independent temporal, secondary-key, text/embedding, and optional LLM
channels through the RankWeave weighted convex fusion contract. A missing
channel is dropped and weights are renormalized; it is never replaced with a
fabricated negative or score.
channels through the RankWeave weighted convex fusion contract. Resolve a
missing channel before loading the exact calibrated active-channel vector;
never repair or renormalize a vector estimated for another channel set, and
never replace a missing channel with a fabricated negative or score.
- Keep the channel-score breakdown and provenance on every candidate decision.
Candidates below the minimum fused-score floor remain roots rather than being
force-attached.
Expand Down
20 changes: 12 additions & 8 deletions docs/adr/0172-event-lineage-channel-evidence.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
**Status:** Accepted
**Date:** 2026-08-21
**Issue:** [#274](https://github.com/ContextualWisdomLab/LineageWeave/issues/274)
**Amended by:** [ADR 0200](0200-channel-weight-reconciliation.md) and
[ADR 0205](0205-tepp-lineage-anchor.md)

## Context

Expand Down Expand Up @@ -33,8 +35,9 @@ authoritative; PROV-O/RDF export is a projection.
`lineage_signal_temporal`, `lineage_signal_secondary_key`,
`lineage_signal_text`, `lineage_signal_llm`. The LLM row is omitted
when the adjudication client is unavailable; it is never fabricated.
3. Weights are the normalized active weights actually used
(`reconstruct.active_weights`). Contribution is `weight * score` and
3. Weights are the exact calibrated active-channel vector validated by
`reconstruct.active_weights`; no different channel set is renormalized into
it. Contribution is `weight * score` and
must reconcile with `fused_score` within
`CHANNEL_EVIDENCE_TOLERANCE` (`1e-6`).
4. Live Event Lineage is replaced atomically. A singleton
Expand All @@ -46,10 +49,10 @@ authoritative; PROV-O/RDF export is a projection.
The administrator-triggered live rebuild and PostgreSQL import pass the
configured contextual-orchestrator adjudication client through the same
reconstruction boundary only when the exact candidate-pair count is at
most 5,000. Larger snapshots drop the LLM channel before any provider call
and renormalize the remaining weights. This is an operational work bound,
not a model-quality or provider-ranking heuristic. One rebuild never mixes
LLM and non-LLM weight profiles across edges.
most 5,000. Larger snapshots drop the LLM channel before loading the
separately calibrated three-channel profile. This is an operational work
bound, not a model-quality or provider-ranking heuristic. One rebuild never
mixes LLM and non-LLM weight profiles across edges.
5. `GET /api/lineage` returns an additive `channel_evidence` collection
on each visible edge (`signal_code`, `signal_label`, `score`,
`weight`, `contribution`, `rank`) ordered by contribution, then
Expand All @@ -67,8 +70,9 @@ authoritative; PROV-O/RDF export is a projection.
inference from source evidence.
- A later rebuild rewrites live Event Lineage as a whole; historic
meaning is not silently mutated in place.
- Completeness is lower when the LLM channel is unavailable, matching
ADR 0064: missing channels are dropped and weights renormalize.
- Completeness is lower when the LLM channel is unavailable; the exact
three-channel profile is loaded instead of renormalizing a four-channel
estimate.

## References

Expand Down
3 changes: 2 additions & 1 deletion docs/product-requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,8 @@ invented facts, scores, weights, or identities.
- Reconstruct bounded post-to-post parentage from independent channels.
- Keep Event Lineage separate from ontology/KG edges.
- Persist reconstruction profile and participating channel evidence.
- Drop unavailable channels and fail closed on malformed calibrated weights.
- Resolve unavailable channels before loading the exact calibrated active-set
vector; never renormalize a vector calibrated for a different channel set.
- Never label an inferred lineage edge causal or authoritative without
separate evidence.

Expand Down
4 changes: 2 additions & 2 deletions lineageweave/embedding_client.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
"""Pluggable embedding channel.

The default :class:`NullEmbeddingClient` makes the channel unavailable
rather than faking a score -- ``reconstruct.active_weights`` drops and
renormalizes around any channel whose client reports ``available = False``.
rather than faking a vector; callers must use the calibrated profile for the
channels they actually execute.
:class:`ContextualOrchestratorEmbeddingClient` calls the authenticated
contextual-orchestrator ``/v1/batch/embeddings`` boundary once a credential is
set. No client in this repository calls a provider embedding endpoint directly.
Expand Down
4 changes: 2 additions & 2 deletions lineageweave/lineage_persistence.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,8 +65,8 @@ def lineage_edge_specs(

``llm`` defaults to ``None``, which ``reconstruct()`` treats as the
unavailable :class:`~lineageweave.adjudication_client.NullAdjudicationClient`
(the llm channel is then dropped and the rest renormalized, not
faked) -- callers that want the highest-weighted reasoning channel
(the supplied calibrated vector must omit the llm channel) -- callers that
want the reasoning channel
actually contributing to real reconstructions must pass a real one.

``weights`` is required and always a psychometric estimate (ADR
Expand Down
38 changes: 26 additions & 12 deletions lineageweave/reconstruct.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@

from __future__ import annotations

import math
from collections import defaultdict

import rankweave as rw
Expand All @@ -25,8 +26,10 @@
# today, TEPP when integrated) -- product paths load the persisted
# corpus estimate, and the library demo estimates from its declared
# generative design (channel_weight_estimation.estimate_fixture_channel_weights).
# Every reconstruct() caller passes weights explicitly; the llm entry
# renormalizes away when no client is configured (see active_weights()).
# Every reconstruct() caller passes weights for the exact channel set it can
# execute; a vector estimated for another set is never repaired or renormalized.

_CORE_WEIGHT_CHANNELS = frozenset({"temporal", "secondary_key", "text"})

# ponytail: only the most recent WINDOW prior records in a group are
# considered as candidate parents, bounding per-group cost to O(n*window)
Expand All @@ -47,12 +50,23 @@
def active_weights(
llm: AdjudicationClient, weights: dict[str, float]
) -> dict[str, float]:
"""Drop and renormalize the llm channel's weight when no client is configured."""
active = dict(weights)
if not getattr(llm, "available", False):
active.pop("llm", None)
total = sum(active.values())
return {channel: weight / total for channel, weight in active.items()}
"""Validate and return the calibrated vector for the exact active channels."""

expected = set(_CORE_WEIGHT_CHANNELS)
if getattr(llm, "available", False):
expected.add("llm")
if set(weights) != expected:
raise ValueError("weights must exactly match the active lineage channels")
Comment on lines +58 to +59

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve unavailable-channel renormalization

When llm is absent or reports available = False, callers supplying a vector that includes llm now receive ValueError instead of reconstructing with that unavailable channel removed. Restore dropping llm and renormalizing the remaining weights, as required for NullAdjudicationClient; otherwise the documented missing-signal fallback is broken.

AGENTS.md reference: AGENTS.md:L189-L195

Useful? React with 👍 / 👎.

Comment on lines +55 to +59

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Exact-match change is consistent across all callers

Switching from drop-and-renormalize to exact-match-or-raise could break a caller passing a four-channel vector with an unavailable llm. Every caller derives its channel set from the same available predicate active_weights uses: estimated_weight_channels, analysis_run_start (active_channels gated on _ProviderBoundaryAdjudication.available at backend/app/analysis_run_start.py:77), the demo server, and seed scripts. None passes a set that mismatches its own llm availability.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

values = tuple(weights.values())
if any(
isinstance(value, bool)
or not isinstance(value, (int, float))
or not math.isfinite(float(value))
or float(value) <= 0.0
for value in values
) or not math.isclose(sum(values), 1.0, abs_tol=1e-9):
raise ValueError("active lineage weights must be finite, positive, and sum to one")
Comment on lines +67 to +68

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Sum-to-one tolerance matches the weight loader

The new abs_tol=1e-9 sum check could in principle fail-close persisted weights, but load_estimated_channel_weights (backend/app/lineage_ingestion.py:302) already enforces the identical abs_tol=1e-9 sum check before reconstruction, and estimate_channel_weights normalizes by dividing by the total. Loaded and estimated vectors sum to 1 within float epsilon, so the stricter gate adds no new failure path.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

return dict(weights)
Comment on lines +55 to +69

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Code contradicts the mandatory pluggable-channel convention

active_weights now rejects a mismatched channel set with ValueError instead of dropping and renormalizing the unavailable channel. The AGENTS.md "Pluggable channels" section still mandates the drop-and-renormalize behavior and names this exact function, and it was not updated to match.

Prompt for agents
The mandatory convention file AGENTS.md (section "Pluggable channels: never fake a missing signal") states that Null channel clients must "make their channel dropped + renormalized (reconstruct.active_weights)". This PR intentionally changes reconstruct.active_weights to require an exact calibrated channel vector and raise ValueError instead of dropping/renormalizing, per the ADR 0200/0205 amendments. Update AGENTS.md to reflect the new fail-closed exact-vector contract so the mandatory convention file no longer describes the removed drop-and-renormalize behavior and no longer references reconstruct.active_weights as the renormalization mechanism.
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.



def _group_by(records: list[Record]) -> dict[str, list[Record]]:
Expand Down Expand Up @@ -159,10 +173,10 @@ def reconstruct(
records: every record across every group; grouping happens here.
llm: adjudication channel client; defaults to
:class:`~lineageweave.adjudication_client.NullAdjudicationClient`
(the llm channel is then dropped, not faked).
weights: per-channel fusion weights before llm-availability
renormalization. Required, and always a psychometric
estimate (ADR 0145, second amendment): the persisted
(the supplied vector must therefore omit the llm channel).
weights: fusion weights calibrated for exactly the channels this call
executes. Required, and always a psychometric estimate (ADR 0145,
second amendment): the persisted
fast-mlsirm corpus estimate on product paths, or
:func:`~lineageweave.channel_weight_estimation.estimate_fixture_channel_weights`
for the library demo. No hand-picked default exists.
Expand Down
18 changes: 18 additions & 0 deletions tests/test_reconstruct.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

from datetime import datetime

import pytest

from lineageweave import Record, reconstruct
from lineageweave.fixtures import sample_records

Expand Down Expand Up @@ -80,6 +82,22 @@ def test_llm_channel_is_used_and_scored_when_a_client_is_supplied() -> None:
assert all("llm" in edge.channel_scores for edge in tree_a.edges)


def test_llm_weight_is_rejected_when_the_channel_is_unavailable() -> None:
"""A four-channel estimate is not repaired into a three-channel vector."""

with pytest.raises(ValueError, match="exactly match"):
reconstruct(sample_records(), weights=_SYNTHETIC_WEIGHTS_WITH_LLM)


def test_missing_llm_weight_is_rejected_when_the_channel_is_available() -> None:
"""An active channel requires its own exact calibrated profile."""

with pytest.raises(ValueError, match="exactly match"):
reconstruct(
sample_records(), llm=_StubAdjudicationClient(), weights=_SYNTHETIC_WEIGHTS
)


def test_candidate_window_bounds_which_priors_are_considered() -> None:
records = [
Record(f"r{i}", "G", f"record {i}", datetime(2026, 1, 1, i), "") for i in range(5)
Expand Down
Loading