fix: measure leftover-map distances on two Gabriel axes (v2.12.13)

AGENTS.md

@@ -186,10 +186,11 @@ in the same spirit) -- never against real data, per the hard rule above.
 against a live local stack (`make up`) and self-skip without one -- see
 [README.md](README.md#local-product-stack-docker-compose).
 
-Period leftover pairs (ADR 0017 / 0018) are computed in
+Period leftover pairs (ADR 0048 / 0049 / 0119) are computed in
 `lineageweave/leftover_pairs.py` from the residual after a real
-GRM/GPCM score, never invented. Missing cells stay out of the
-Gabriel factorization. Closest and farthest post–criterion pairs
+GRM/GPCM score, never invented. Distances are Euclidean on the
+two-dimensional Gabriel leftover map; missing cells stay out of
+the factorization. Closest and farthest post–criterion pairs
 persist to `report_leftover_pair` and sit above the member list so
 a click opens that post.
 

ARCHITECTURE.md

@@ -588,7 +588,7 @@ on those same fixed parameters (Kim, 2006 FIPC). After scoring,
 `information_polytomous` ranks the shared-bank items by Fisher
 information at the group's mean θ (Lord, 1980 max-info CAT). Rankings
 persist to `report_item_information`. After those IRT main effects,
-residual SVD leftover pairs (Jeon et al., 2021; ADR 0017) persist to
+residual SVD leftover pairs on two Gabriel axes (Jeon et al., 2021; ADR 0048 / 0119) persist to
 `report_leftover_pair`. Results persist to
 `report_period_score` / `report_member_score`.
 `GET /api/reports/{grouping}` lists the trend;

CHANGELOG.d/2.12.13-leftover-map-two-dimensional-distance.md

@@ -0,0 +1,8 @@
+## 2.12.13 — Two-dimensional leftover-map distances
+
+- Measure closest and farthest leftover post–criterion distances on
+  the two-axis Jeon / Gabriel interaction map (ADR 0119). Hidden SVD
+  axes after the second no longer change `leftover_distance`. Rank-0
+  and rank-1 maps still pad unused axes with zero. After `make seed`,
+  leftover pairs above the member list still open that post. Never
+  invent a leftover score or a theta.

CHANGELOG.md

@@ -12,6 +12,17 @@ All notable changes to this project are documented here. Format follows
   environment, so local OIDC and synthetic-data workflows resolve the same
   pinned dependencies as CI.
 
+## [2.12.13] - 2026-08-24
+
+### Fixed
+
+- Closest and farthest leftover post–criterion distances are now
+  Euclidean on the two-axis Jeon / Gabriel interaction map (ADR 0119).
+  Hidden SVD axes after the second no longer change `leftover_distance`.
+  Rank-0 and rank-1 maps still pad unused axes with zero, so seed
+  leftover pairs above the member list still open that post. Never
+  invent a leftover score or a theta.
+
 ## [2.12.6] - 2026-08-20
 
 ### Added

docs/adr/0048-persist-lsirm-leftover-pairs.md

@@ -2,6 +2,7 @@
 
 **Decision status:** Accepted
 **Date:** 2026-08-17
+**Amended by:** [ADR 0119](0119-leftover-map-two-dimensional-distance.md) (two leftover-map axes)
 
 ## Context
 
@@ -30,7 +31,8 @@ and one `farthest` observed cell per period report in
 `report_leftover_pair` (3NF, two-or-more-word `snake_case`).
 
 The biplot lives in `lineageweave/leftover_pairs.py` so leftover
-tests do not import `period_report` or `fast_mlsirm`.
+tests do not import `period_report` or `fast_mlsirm`. Distances are
+Euclidean on the two leftover-map axes (ADR 0119).
 
 Cascade the rows with `report_period_score`. A leftover post must
 also be a `report_member_score` row, and the leftover criterion

docs/adr/0119-leftover-map-two-dimensional-distance.md

@@ -0,0 +1,53 @@
+# ADR 0119 — Measure leftover-map distances on two Gabriel axes
+
+**Decision status:** Accepted
+**Date:** 2026-08-24
+
+Amends [ADR 0048](0048-persist-lsirm-leftover-pairs.md).
+
+## Context
+
+ADR 0048 persists closest and farthest leftover post–criterion pairs
+after IRT main effects. Jeon et al. (2021, eq. 3) place the leftover
+interaction `−γ‖ξ_p − ζ_i‖` on a two-dimensional person–item map.
+Gabriel (1971) supplies those coordinates from a residual biplot.
+
+`leftover_pairs.py` previously measured Euclidean distance on every
+kept SVD axis. A rank-3 residual therefore reported a leftover
+distance that a buyer cannot read on the 2D interaction map, and
+that would disagree with stored two-axis coordinates if those later
+persist. Rank-1 seed fixtures still passed because unused axes were
+absent, not because the estimator was two-dimensional.
+
+## Decision
+
+Closest and farthest leftover distances are Euclidean on **exactly
+two** leftover-map axes. `_leftover_map_positions` may still return
+the full Gabriel factorization; `_pad_map_axes` pads a rank-0 or
+rank-1 map with zeros and truncates hidden axes after the second.
+Missing cells stay out of the factorization. Rank-0 residuals still
+emit a stable pair with distance zero. Do not invent a leftover
+score. Do not invent a theta.
+
+This slice does not persist map coordinates or change leftover UI.
+Those remain ADR 0048 / 0049, and persist-map tables stay on their
+own PR stack.
+
+## Consequences
+
+`leftover_distance` on `report_leftover_pair` matches the 2D Jeon
+map. After `make seed`, closest and farthest pairs above the member
+list still open that post. A rank-3 synthetic residual proves the
+stored distance equals the two-axis hypot and is not the full-rank
+norm.
+
+## References
+
+Gabriel, K. R. (1971). The biplot graphic display of matrices with
+application to principal component analysis. *Biometrika, 58*(3),
+453–467. https://doi.org/10.1093/biomet/58.3.453
+
+Jeon, M., Jin, I. H., Schweinberger, M., & Baugh, S. (2021). Mapping
+unobserved item–respondent interactions: A latent space item response
+model with interaction map. *Psychometrika, 86*(2), 378–403.
+https://doi.org/10.1007/s11336-021-09762-5

frontend/package.json

@@ -1,7 +1,7 @@
 {
   "name": "frontend",
   "private": true,
-  "version": "2.12.6",
+  "version": "2.12.13",
   "type": "module",
   "scripts": {
     "dev": "vite",

lineageweave/leftover_pairs.py

@@ -1,9 +1,12 @@
-"""Jeon leftover post–criterion pairs after a main-effect IRT (ADR 0017).
+"""Jeon leftover post–criterion pairs after a main-effect IRT (ADR 0048 / 0119).
 
 Does not import ``fast_mlsirm`` or ``period_report``. A Gabriel biplot
 of the residual ``R = Y − E[Y|θ, item]`` supplies person and item
 positions. Missing response cells are excluded from the factorization;
-they are never treated as zero residuals.
+they are never treated as zero residuals. Pair distances are Euclidean
+on the two leftover-map axes (Jeon et al., 2021); unused axes pad with
+zero rather than inventing a second component, and hidden SVD axes
+after the second are dropped.
 """
 
 from __future__ import annotations
@@ -15,6 +18,7 @@
 PAIR_KIND_CLOSEST = "closest"
 PAIR_KIND_FARTHEST = "farthest"
 _LEFTOVER_SINGULAR_FLOOR = 1e-12
+_LEFTOVER_MAP_AXES = 2
 
 
 @dataclass(frozen=True)
@@ -38,9 +42,10 @@ def leftover_pairs_from_residual(
 
     Jeon et al. (2021) leftover interaction is ``−γ‖ξ_p − ζ_i‖``. This
     estimator places persons and items from the residual after IRT main
-    effects (Gabriel, 1971). Only observed cells become pairs. A rank-0
-    residual still emits a stable closest/farthest pair so seed is not
-    empty; it does not invent a leftover score.
+    effects (Gabriel, 1971). Only observed cells become pairs. Distances
+    use the two leftover-map axes; a rank-0 residual still emits a
+    stable closest/farthest pair so seed is not empty and does not
+    invent a leftover score.
     """
     if matrix.shape != (len(post_ids), len(item_codes)):
         raise ValueError(
@@ -72,13 +77,15 @@ def leftover_pairs_from_residual(
     if person_pos is not None and item_pos is not None:
         person_index = np.flatnonzero(keep_person)
         item_index = np.flatnonzero(keep_item)
+        person_xy = _pad_map_axes(person_pos)
+        item_xy = _pad_map_axes(item_pos)
         local_person = {int(person): local for local, person in enumerate(person_index)}
         local_item = {int(item): local for local, item in enumerate(item_index)}
         for person, item in observed:
             if person not in local_person or item not in local_item:
                 continue
             distance = float(
-                np.linalg.norm(person_pos[local_person[person]] - item_pos[local_item[item]])
+                np.linalg.norm(person_xy[local_person[person]] - item_xy[local_item[item]])
             )
             if not np.isfinite(distance):
                 continue
@@ -148,7 +155,12 @@ def _complete_case_positions(
 
 
 def _leftover_map_positions(filled: np.ndarray) -> tuple[np.ndarray, np.ndarray]:
-    """Gabriel biplot coordinates; rank-0 residuals collapse to the origin."""
+    """Gabriel coordinates ordered by descending singular value.
+
+    NumPy's SVD contract returns singular values largest-first, so filtering
+    by the numerical floor preserves a prefix and the first two columns remain
+    the two leading leftover-map axes. Rank-0 residuals collapse to the origin.
+    """
     n_persons, n_items = filled.shape
     if n_persons == 0 or n_items == 0 or not np.any(np.abs(filled) > _LEFTOVER_SINGULAR_FLOOR):
         return (
@@ -157,12 +169,15 @@ def _leftover_map_positions(filled: np.ndarray) -> tuple[np.ndarray, np.ndarray]
         )
     left, singular, right = np.linalg.svd(filled, full_matrices=False)
     keep = singular > _LEFTOVER_SINGULAR_FLOOR
-    if not np.any(keep):
-        return (
-            np.zeros((n_persons, 1), dtype=np.float64),
-            np.zeros((n_items, 1), dtype=np.float64),
-        )
     scale = np.sqrt(singular[keep])
     person_pos = left[:, keep] * scale
     item_pos = right[keep, :].T * scale
     return person_pos, item_pos
+
+
+def _pad_map_axes(positions: np.ndarray) -> np.ndarray:
+    """Pad or truncate Gabriel coordinates to two leftover-map axes."""
+    padded = np.zeros((positions.shape[0], _LEFTOVER_MAP_AXES), dtype=np.float64)
+    width = min(_LEFTOVER_MAP_AXES, positions.shape[1])
+    padded[:, :width] = positions[:, :width]
+    return padded

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "lineageweave"
-version = "2.12.6"
+version = "2.12.13"
 description = "Reconstructs git-branch-style lineage DAGs from scattered short records using multi-channel score fusion and LLM adjudication."
 readme = "README.md"
 license = { text = "MIT" }

tests/test_leftover_pairs.py

@@ -1,4 +1,4 @@
-"""Leftover post–criterion pairs after the main-effect IRT (ADR 0017).
+"""Leftover post–criterion pairs after the main-effect IRT (ADR 0048 / 0119).
 
 Uses a constructed residual matrix so the closest and farthest pair
 are known without calling ``fit_polytomous``. Loads
@@ -17,9 +17,12 @@
 import pytest
 
 _LEFTOVER_PATH = Path(__file__).resolve().parents[1] / "lineageweave" / "leftover_pairs.py"
+_LEFTOVER_SINGULAR_FLOOR = 1e-12
+_LEFTOVER_MAP_AXES = 2
 
 
 def _load_leftover():
+    """Load only the dependency-light leftover module under test."""
     source = _LEFTOVER_PATH.read_text(encoding="utf-8")
     imported = []
     for node in ast.parse(source).body:
@@ -43,6 +46,22 @@ def _load_leftover():
 leftover_pairs_from_residual = leftover.leftover_pairs_from_residual
 
 
+def _gabriel_positions(filled: np.ndarray) -> tuple[np.ndarray, np.ndarray]:
+    """Independent Gabriel coordinates used to prove leftover_distance axes."""
+    left, singular, right = np.linalg.svd(filled, full_matrices=False)
+    keep = singular > _LEFTOVER_SINGULAR_FLOOR
+    scale = np.sqrt(singular[keep])
+    return left[:, keep] * scale, right[keep, :].T * scale
+
+
+def _pad_map_axes(positions: np.ndarray) -> np.ndarray:
+    """Independently pad or truncate coordinates to two map axes."""
+    padded = np.zeros((positions.shape[0], _LEFTOVER_MAP_AXES), dtype=np.float64)
+    width = min(_LEFTOVER_MAP_AXES, positions.shape[1])
+    padded[:, :width] = positions[:, :width]
+    return padded
+
+
 def test_leftover_residual_biplot_separates_aligned_and_opposed_cells() -> None:
     """A rank-1 leftover spike puts the aligned cell closest and the opposed cell farthest."""
     post_ids = ["post-a", "post-b", "post-c"]
@@ -70,6 +89,7 @@ def test_leftover_residual_biplot_separates_aligned_and_opposed_cells() -> None:
 
 
 def test_zero_residual_still_emits_stable_leftover_pairs() -> None:
+    """A rank-zero map retains deterministic closest and farthest rows."""
     post_ids = ["alpha-post", "beta-post"]
     item_codes = ("item_one", "item_two")
     matrix = np.ones((2, 2), dtype=np.float64)
@@ -111,8 +131,120 @@ def test_partial_observation_does_not_treat_missing_as_zero_residual() -> None:
 
 
 def test_leftover_is_empty_without_observed_cells() -> None:
+    """An entirely missing response matrix yields no invented pair."""
     post_ids = ["post-empty"]
     item_codes = ("item_one",)
     matrix = np.array([[np.nan]], dtype=np.float64)
     expected = np.array([[0.0]], dtype=np.float64)
     assert leftover_pairs_from_residual(post_ids, item_codes, matrix, expected) == ()
+
+
+def test_rank_three_pair_distances_match_two_dimensional_gabriel_coords() -> None:
+    """Jeon leftover_distance is Euclidean on the 2D map, not the full SVD rank."""
+    post_ids = ["post-a", "post-b", "post-c", "post-d"]
+    item_codes = ("item-a", "item-b", "item-c", "item-d")
+    matrix = np.array(
+        [
+            [4.0, 1.0, 0.0, -1.0],
+            [0.0, 3.0, 1.0, -2.0],
+            [-2.0, 0.0, 2.0, 1.0],
+            [1.0, -1.0, 0.0, 4.0],
+        ],
+        dtype=np.float64,
+    )
+    expected = np.zeros_like(matrix)
+    filled = matrix - float(np.mean(matrix))
+    person_full, item_full = _gabriel_positions(filled)
+    assert person_full.shape[1] >= 3
+    person_map = _pad_map_axes(person_full)
+    item_map = _pad_map_axes(item_full)
+    full_distances = np.linalg.norm(person_full[:, None, :] - item_full[None, :, :], axis=2)
+    map_distances = np.linalg.norm(person_map[:, None, :] - item_map[None, :, :], axis=2)
+    assert float(np.max(np.abs(full_distances - map_distances))) > 1e-6
+
+    pairs = leftover_pairs_from_residual(post_ids, item_codes, matrix, expected)
+    assert [pair.pair_kind for pair in pairs] == [PAIR_KIND_CLOSEST, PAIR_KIND_FARTHEST]
+    post_index = {post_id: index for index, post_id in enumerate(post_ids)}
+    item_index = {code: index for index, code in enumerate(item_codes)}
+    for pair in pairs:
+        person = post_index[pair.post_id]
+        item = item_index[pair.criterion_code]
+        assert pair.leftover_distance == pytest.approx(float(map_distances[person, item]))
+        assert pair.leftover_distance != pytest.approx(
+            float(full_distances[person, item]), abs=1e-9
+        )
+
+    farthest_map = np.unravel_index(int(np.argmax(map_distances)), map_distances.shape)
+    farthest = pairs[1]
+    assert (post_index[farthest.post_id], item_index[farthest.criterion_code]) == farthest_map
+
+
+def test_rejects_response_and_expectation_shape_mismatches() -> None:
+    """Scientific inputs must match their declared post and criterion axes."""
+    with pytest.raises(ValueError, match="matrix shape"):
+        leftover_pairs_from_residual(
+            ["post-a"],
+            ("item-a",),
+            np.zeros((2, 1), dtype=np.float64),
+            np.zeros((2, 1), dtype=np.float64),
+        )
+    with pytest.raises(ValueError, match="expected shape"):
+        leftover_pairs_from_residual(
+            ["post-a"],
+            ("item-a",),
+            np.zeros((1, 1), dtype=np.float64),
+            np.zeros((1, 2), dtype=np.float64),
+        )
+
+
+def test_sparse_residual_uses_only_observed_cells_for_fallback_distance() -> None:
+    """No complete rectangle still yields finite observed-cell distances."""
+    matrix = np.array([[1.0, np.nan], [np.nan, -1.0]], dtype=np.float64)
+    pairs = leftover_pairs_from_residual(
+        ["post-a", "post-b"],
+        ("item-a", "item-b"),
+        matrix,
+        np.zeros_like(matrix),
+    )
+    assert [(pair.post_id, pair.criterion_code) for pair in pairs] == [
+        ("post-a", "item-a"),
+        ("post-b", "item-b"),
+    ]
+    assert [pair.leftover_distance for pair in pairs] == pytest.approx([1.0, 1.0])
+
+
+def test_nonfinite_map_distance_falls_back_to_centered_residual(
+    monkeypatch: pytest.MonkeyPatch,
+) -> None:
+    """An unusable factorization coordinate cannot become persisted distance."""
+    monkeypatch.setattr(
+        leftover,
+        "_complete_case_positions",
+        lambda *_args: (
+            np.array([[np.inf]], dtype=np.float64),
+            np.array([[-np.inf]], dtype=np.float64),
+        ),
+    )
+    pairs = leftover_pairs_from_residual(
+        ["post-a"],
+        ("item-a",),
+        np.array([[1.0]], dtype=np.float64),
+        np.array([[0.0]], dtype=np.float64),
+    )
+    assert [pair.leftover_distance for pair in pairs] == [0.0, 0.0]
+
+
+def test_empty_observation_mask_has_no_complete_case_axes() -> None:
+    """The complete-case helpers preserve an empty scientific boundary."""
+    observed = np.zeros((1, 1), dtype=bool)
+    keep_person, keep_item = leftover._complete_case_masks(observed)
+    assert not keep_person.any()
+    assert not keep_item.any()
+    person_pos, item_pos = leftover._complete_case_positions(
+        np.zeros((1, 1), dtype=np.float64),
+        0.0,
+        keep_person,
+        keep_item,
+    )
+    assert person_pos is None
+    assert item_pos is None