feat: persist leftover interaction-map coordinates (v2.12.19)

ARCHITECTURE.md

@@ -606,15 +606,20 @@ on those same fixed parameters (Kim, 2006 FIPC). After scoring,
 information at the group's mean θ (Lord, 1980 max-info CAT). Rankings
 persist to `report_item_information`. After those IRT main effects,
 residual SVD leftover pairs on two Gabriel axes (Jeon et al., 2021;
-ADR 0017 / 0048 / 0049 / 0119 / 0148 / 0158 / 0162 / 0163 / 0164 / 0168 /
-0182 / 0185 / 0201) persist to `report_leftover_pair` with signed residual `R`,
-observed `Y`, expected `E[Y|θ, item]`, full leftover-map rank, unexplained
-leftover, ADR 0201 reconstruction evidence, and ADR 0185 cross-share evidence.
-Those ADRs are the normative mathematical and storage contracts. Leftover-map axis share
-(Gabriel inertia of residual SVD axes 1 and 2; ADR 0148) persists to
+ADR 0017 / 0048 / 0049 / 0119 / 0121 / 0126 / 0148 / 0158 / 0162 / 0163 /
+0164 / 0168 / 0182 / 0185 / 0201) persist to `report_leftover_pair` with signed
+residual `R`, observed `Y`, expected `E[Y|θ, item]`, full leftover-map rank,
+unexplained leftover `U = R − R̂`, and leftover-map cross share
+`x = 2 R̂ U / R²`; signed reconstruction `R̂` also persists so the raw-residual
+identity remains auditable. Complete-case coordinates persist to
+`report_leftover_map_person` / `report_leftover_map_item` (ADR 0121).
+Leftover-map axis share (Gabriel inertia of residual SVD axes 1 and 2;
+ADR 0148) persists to
 `report_leftover_map_axis`. Complete-case leftover-map coverage (ADR
 0168) persists to `report_leftover_map_coverage` so readers see how
-many scored posts entered the factorization. Results persist to
+many scored posts entered the factorization -- without a complete-case
+rectangle neither a pair nor a coordinate is persisted either. Results
+persist to
 `report_period_score` / `report_member_score`.
 `GET /api/reports/{grouping}` lists the trend;
 `GET /api/reports/{grouping}/{period}` is ABAC-filtered;
@@ -629,12 +634,10 @@ open ticket title, status lookup label, and due date when one exists. The home p
 the actual mean θ, the FIPC delta, the CAT-selected item, leftover
 closest/farthest pairs (signed residual `R`, observed `Y`, expected
 `E`, full rank, and two-axis leftover-map distance `d` after IRT main
-effects) above the member list, leftover-map axis share for residual
-SVD axes 1 and 2, and complete-case coverage captions (map used N of M
-scored posts), plus the
-
-closest/farthest pairs above the member list, leftover pairs on the
-grouping comparison strip, and the
+effects) above the member list and on the grouping comparison strip,
+the leftover interaction map, leftover-map axis share for residual SVD
+axes 1 and 2, complete-case coverage captions (map used N of M scored
+posts), plus the
 PU / corp / thread comparison -- never a placeholder. TEPP is unchanged.
 
 ## Phase 6b: Knowledge Graph as a real Ontology + Semantic Layer

CHANGELOG.d/2.12.19-leftover-interaction-map.md

@@ -0,0 +1,11 @@
+## 2.12.19 — Persist leftover interaction-map coordinates
+
+- After IRT main effects, persist complete-case leftover-map person ξ
+  and item ζ coordinates (`report_leftover_map_person` /
+  `report_leftover_map_item`) and render the 2D Jeon / Gabriel map
+  above leftover pairs (ADR 0121 / 0126). Click a post node or a
+  leftover-pair criterion to open that post. Rank-0 and rank-1 maps
+  pad unused axes with zero. The Rust-backed `fast-mlsirm` contract owns
+  every numerical result; LineageWeave only maps identifiers, persists,
+  and presents it. Hidden posts stay hidden. Never invent a leftover score
+  or a theta.

CHANGELOG.md

@@ -103,6 +103,10 @@ All notable changes to this project are documented here. Format follows
 
 ### Changed
 
+- Residual interaction-map arithmetic now comes from the Rust-backed
+  `fast-mlsirm` contract. LineageWeave maps returned indices to product
+  identifiers and persists the evidence; it no longer computes residuals,
+  factorization, coordinates, distances, reconstruction, or cross-share.
 - ADRs 0011 and 0065 now include APA 7th References for the dated W3C
   PROV-O and PROV-DM Recommendations (30 April 2013). Decisions are
   unchanged.
@@ -222,6 +226,16 @@ All notable changes to this project are documented here. Format follows
   omits the badge rather than inventing a leftover score. Two-axis
   reconstruction `R̂` stays internal and is not persisted.
 
+## [2.12.20] - 2026-08-24
+
+### Added
+
+- Period reports now persist leftover interaction-map coordinates ξ / ζ
+  after IRT main effects and render a 2D Jeon / Gabriel map above leftover
+  pairs (ADR 0121 / 0126). Click a post node or a leftover-pair criterion
+  node to open that post. Rank-0 and rank-1 maps pad unused axes with
+  zero. Hidden posts stay hidden. Never invent a leftover score or a theta.
+
 ## [2.12.19] - 2026-08-24
 
 ### Added

backend/app/main.py

@@ -2741,6 +2741,12 @@ async def read_period_reports(
             if _can_see_post(account, pair)
             and not _is_synthetic_demo_member(pair, demo_entity_ids)
         ]
+        leftover_map_persons = [
+            person
+            for person in report.get("leftover_map_persons", [])
+            if _can_see_post(account, person)
+            and not _is_synthetic_demo_member(person, demo_entity_ids)
+        ]
         members = [
             {
                 key: value
@@ -2769,12 +2775,29 @@ async def read_period_reports(
             }
             for pair in leftover_pairs
         ]
+        leftover_map_persons = [
+            {
+                key: value
+                for key, value in person.items()
+                if key
+                not in {
+                    "has_real_source_context",
+                    "visibility_code",
+                    "corporate_entity_id",
+                    "process_unit_id",
+                }
+            }
+            for person in leftover_map_persons
+        ]
+        leftover_map_items = list(report.get("leftover_map_items", []))
         leftover_map_axes = list(report.get("leftover_map_axes", []))
         visible.append(
             {
                 **report,
                 "members": members,
                 "leftover_pairs": leftover_pairs,
+                "leftover_map_persons": leftover_map_persons,
+                "leftover_map_items": leftover_map_items,
                 "leftover_map_axes": leftover_map_axes,
                 "post_count": len(members),
             }

backend/app/report_ingestion.py

@@ -351,7 +351,7 @@ async def persist_period_report(
     period_code: str,
     report: PeriodReport,
 ) -> None:
-    """Replace the stored report, member scores, leftover pairs, leftover-map axes, leftover coverage, and item bank."""
+    """Replace the stored report, member scores, leftover pairs, leftover map, leftover-map axes, leftover coverage, and item bank."""
     await conn.execute(
         """
         delete from report_period_score
@@ -465,6 +465,38 @@ async def persist_period_report(
             pair.leftover_map_cross_share,
             pair.leftover_map_reconstruction,
         )
+    for person in report.leftover_map_persons:
+        await conn.execute(
+            """
+            insert into report_leftover_map_person (
+                grouping_kind, grouping_key, period_code, rubric_version,
+                post_id, axis_one, axis_two
+            ) values ($1,$2,$3,$4,$5,$6,$7)
+            """,
+            grouping_kind,
+            grouping_key,
+            period_code,
+            RUBRIC_VERSION,
+            person.post_id,
+            person.axis_one,
+            person.axis_two,
+        )
+    for item in report.leftover_map_items:
+        await conn.execute(
+            """
+            insert into report_leftover_map_item (
+                grouping_kind, grouping_key, period_code, rubric_version,
+                criterion_code, axis_one, axis_two
+            ) values ($1,$2,$3,$4,$5,$6,$7)
+            """,
+            grouping_kind,
+            grouping_key,
+            period_code,
+            RUBRIC_VERSION,
+            item.criterion_code,
+            item.axis_one,
+            item.axis_two,
+        )
     for axis in report.leftover_map_axes:
         await conn.execute(
             """
@@ -664,6 +696,32 @@ async def fetch_period_reports(
         period_code,
         RUBRIC_VERSION,
     )
+    # Safe SQL: the source-context expression is an immutable schema fragment; report keys are bound.
+    leftover_map_persons = await conn.fetch(  # nosemgrep: python.lang.security.audit.sqli.asyncpg-sqli.asyncpg-sqli
+        f"""
+        select lp.grouping_key, lp.post_id, lp.axis_one, lp.axis_two, p.post_title,
+               p.visibility_code, p.corporate_entity_id, p.process_unit_id,
+               ({_SOURCE_CONTEXT_PRESENT_SQL}) as has_real_source_context
+        from report_leftover_map_person lp
+        join source_post p on p.post_id = lp.post_id
+        where lp.grouping_kind = $1 and lp.period_code = $2 and lp.rubric_version = $3
+        order by lp.grouping_key, p.post_title
+        """,
+        grouping_kind,
+        period_code,
+        RUBRIC_VERSION,
+    )
+    leftover_map_items = await conn.fetch(
+        """
+        select grouping_key, criterion_code, axis_one, axis_two
+        from report_leftover_map_item
+        where grouping_kind = $1 and period_code = $2 and rubric_version = $3
+        order by grouping_key, criterion_code
+        """,
+        grouping_kind,
+        period_code,
+        RUBRIC_VERSION,
+    )
     leftover_axes = await conn.fetch(
         """
         select grouping_key, axis_index, leftover_singular_value, leftover_share
@@ -700,6 +758,12 @@ async def fetch_period_reports(
     leftover_by_group: dict[str, list[asyncpg.Record]] = defaultdict(list)
     for row in leftover:
         leftover_by_group[row["grouping_key"]].append(row)
+    leftover_persons_by_group: dict[str, list[asyncpg.Record]] = defaultdict(list)
+    for row in leftover_map_persons:
+        leftover_persons_by_group[row["grouping_key"]].append(row)
+    leftover_items_by_group: dict[str, list[asyncpg.Record]] = defaultdict(list)
+    for row in leftover_map_items:
+        leftover_items_by_group[row["grouping_key"]].append(row)
     leftover_axes_by_group: dict[str, list[asyncpg.Record]] = defaultdict(list)
     for row in leftover_axes:
         leftover_axes_by_group[row["grouping_key"]].append(row)
@@ -816,6 +880,29 @@ async def fetch_period_reports(
                     }
                     for row in leftover_by_group.get(header["grouping_key"], [])
                 ],
+                "leftover_map_persons": [
+                    {
+                        "post_id": str(row["post_id"]),
+                        "post_title": row["post_title"],
+                        "axis_one": float(row["axis_one"]),
+                        "axis_two": float(row["axis_two"]),
+                        "visibility_code": row["visibility_code"],
+                        "corporate_entity_id": str(row["corporate_entity_id"]),
+                        "process_unit_id": (
+                            None if row["process_unit_id"] is None else str(row["process_unit_id"])
+                        ),
+                        "has_real_source_context": bool(row["has_real_source_context"]),
+                    }
+                    for row in leftover_persons_by_group.get(header["grouping_key"], [])
+                ],
+                "leftover_map_items": [
+                    {
+                        "criterion_code": str(row["criterion_code"]),
+                        "axis_one": float(row["axis_one"]),
+                        "axis_two": float(row["axis_two"]),
+                    }
+                    for row in leftover_items_by_group.get(header["grouping_key"], [])
+                ],
                 "leftover_map_axes": [
                     {
                         "axis_index": int(row["axis_index"]),

backend/tests/test_api.py

@@ -196,6 +196,11 @@
     / "migrations"
     / "0169_report_leftover_map_axis.sql"
 )
+_LEFTOVER_MAP_MIGRATION = (
+    Path(__file__).resolve().parents[2]
+    / "migrations"
+    / "0172_report_leftover_interaction_map.sql"
+)
 _CHANNEL_EVIDENCE_MIGRATION = (
     Path(__file__).resolve().parents[2]
     / "migrations"
@@ -367,6 +372,7 @@ def seeded_db(demo_analyst_token):
             cur.execute(_GLOBAL_ASK_SCOPE_MIGRATION.read_text())
             cur.execute(_EVENT_OCCURRED_AT_MIGRATION.read_text())
             cur.execute(_LEFTOVER_MAP_AXIS_MIGRATION.read_text())
+            cur.execute(_LEFTOVER_MAP_MIGRATION.read_text())
             cur.execute(_CHANNEL_EVIDENCE_MIGRATION.read_text())
             cur.execute(_LEFTOVER_MAP_UNEXPLAINED_MIGRATION.read_text())
             cur.execute(_LEFTOVER_MAP_CROSS_SHARE_MIGRATION.read_text())
@@ -5543,6 +5549,22 @@ def test_seed_period_report_surfaces_on_get_reports(client, demo_analyst_token,
     assert leftover_kinds <= {"closest", "farthest"}
     assert all(pair["post_title"] for pair in high_report.get("leftover_pairs", []))
     assert all(pair["leftover_distance"] >= 0 for pair in high_report.get("leftover_pairs", []))
+    leftover_map_persons = high_report.get("leftover_map_persons", [])
+    leftover_map_items = high_report.get("leftover_map_items", [])
+    member_ids = {member["post_id"] for member in high_report["members"]}
+    item_codes = {item["item_code"] for item in high_report.get("selected_items", [])}
+    assert leftover_map_persons
+    assert leftover_map_items
+    assert all(person["post_title"] for person in leftover_map_persons)
+    assert {person["post_id"] for person in leftover_map_persons} <= member_ids
+    assert all(
+        {"visibility_code", "corporate_entity_id", "process_unit_id"}.isdisjoint(person)
+        for person in leftover_map_persons
+    )
+    assert {item["criterion_code"] for item in leftover_map_items} <= item_codes
+    for point in leftover_map_persons + leftover_map_items:
+        assert isinstance(point["axis_one"], (int, float))
+        assert isinstance(point["axis_two"], (int, float))
     assert all(
         {"visibility_code", "corporate_entity_id", "process_unit_id"}.isdisjoint(pair)
         for pair in high_report.get("leftover_pairs", [])

docs/adr/0003-fast-mlsirm-report-integration.md

@@ -102,8 +102,9 @@ than one large PR:
    reports panel. Do not reimplement an information function here.
 7. **Leftover-pair slice** (shipped in 0.71.2; ADR 0017 / 0018 / 0048 /
    0049): after IRT main effects, persist closest and farthest
-   post–criterion pairs from the residual leftover map. Do not fork LSIRM or
-   invent a leftover-pair API inside `fast-mlsirm` in this slice.
+   post–criterion pairs from the residual leftover map. Consume fast-mlsirm's
+   Rust-backed residual interaction-map contract and keep only identifier
+   mapping, authorization, persistence, and pair selection here (ADR 0211).
    Category probabilities and expected responses must use `fast-mlsirm`'s
    public Rust-backed prediction API (upstream PR #1279); LineageWeave must
    not reproduce GRM/GPCM parameter conventions locally.

docs/adr/0049-leftover-pair-report-ui.md

@@ -66,6 +66,5 @@ next action, not only the distance.
 Depends on [ADR 0048](0048-persist-lsirm-leftover-pairs.md) and
 [ADR 0003](0003-fast-mlsirm-report-integration.md). Complete-case
 coverage of the leftover map is [ADR 0168](0168-leftover-map-complete-case-coverage.md).
-
-[ADR 0003](0003-fast-mlsirm-report-integration.md). The grouping
+The grouping
 comparison strip reuses this leftover store ([ADR 0149](0149-leftover-pairs-on-comparison-strip.md)).

docs/adr/0121-persist-leftover-interaction-map.md

@@ -0,0 +1,78 @@
+# ADR 0121 — Persist leftover interaction-map coordinates
+
+**Decision status:** Accepted
+**Date:** 2026-08-24
+
+## Context
+
+ADR 0048 persists the closest and farthest leftover post–criterion
+pairs after IRT main effects. Those pairs are two cells on the Jeon
+et al. (2021, eq. 3) leftover interaction map `−γ‖ξ_p − ζ_i‖`. The
+Gabriel (1971) biplot that produces the pairs already computes person
+positions `ξ` and item positions `ζ`, then discards them. A reader
+who sees only two named pairs cannot see *why* those cells sat
+closest or farthest, or where the other complete-case posts and
+criteria sit on the same leftover map.
+
+fast-mlsirm now exposes the Rust-backed residual interaction-map contract
+adopted by ADR 0211. LineageWeave must not fork that calculation, invent a
+second IRT fit, or treat a missing residual cell as a zero residual.
+
+## Decision
+
+After a real GRM/GPCM score, keep the complete-case Gabriel
+coordinates that leftover pairs already use. Persist every complete-
+case post as `report_leftover_map_person` (`axis_one`, `axis_two`)
+and every complete-case criterion as `report_leftover_map_item`.
+Pad unused axes with zero when residual rank is below two. Do not
+invent a second component. Closest/farthest selection and persisted
+distance use those same two reader-visible axes; unpersisted higher
+components never silently change a highlighted map pair. Incomplete
+rows and columns stay out of the factorization.
+
+Cascade the rows with `report_period_score`. A leftover-map post
+must also be a `report_member_score` row. A leftover-map criterion
+must be a `report_item_information` item on that same report. Do not
+store a second theta. A rank-0 residual still emits origin
+coordinates so `make seed` is not empty; those zeros are not a
+fabricated interaction.
+
+Closest and farthest pairs remain ADR 0048 / ADR 0049. The map sits
+**above** that pair list on the period-report group. Clicking a
+person node opens that post with the same handler as a leftover
+pair. Pair-member criterion nodes open that leftover-pair post
+([ADR 0126](0126-leftover-map-criterion-node.md)). Hidden posts stay hidden: leftover-map persons join
+`source_post` and use the same ABAC gate as members and leftover
+pairs. Missing map rows render nothing.
+
+The biplot calculation lives in fast-mlsirm. `lineageweave/leftover_pairs.py`
+maps returned indices and values to authorized product identifiers and selects
+the closest/farthest cells; it contains no factorization or cross-term formula.
+
+## Consequences
+
+Rebuild and seed write leftover-map coordinates in the same
+transaction as leftover pairs. `GET /api/reports/{grouping}/{period}`
+returns `leftover_map_persons` (with post title) and
+`leftover_map_items`. Migration
+`0172_report_leftover_interaction_map.sql` upgrades volumes that
+already applied `0001` / `0012`. `migrate.sh` already replays every
+four-digit `NNNN_*.sql` file (ADR 0166), so 0172 lands on existing
+volumes without a new allowlist entry.
+
+## Related
+
+Depends on [ADR 0048](0048-persist-lsirm-leftover-pairs.md),
+[ADR 0049](0049-leftover-pair-report-ui.md), and
+[ADR 0003](0003-fast-mlsirm-report-integration.md).
+
+## References
+
+Gabriel, K. R. (1971). The biplot graphic display of matrices with
+application to principal component analysis. *Biometrika, 58*(3),
+453–467. https://doi.org/10.1093/biomet/58.3.453
+
+Jeon, M., Jin, I. H., Schweinberger, M., & Baugh, S. (2021). Mapping
+unobserved item–respondent interactions: A latent space item response
+model with interaction map. *Psychometrika, 86*(2), 378–403.
+https://doi.org/10.1007/s11336-021-09762-5