feat: persist leftover interaction-map coordinates (v2.12.7)

AGENTS.md

@@ -192,15 +192,17 @@ in the same spirit) -- never against real data, per the hard rule above.
 against a live local stack (`make up`) and self-skip without one -- see
 [README.md](README.md#local-product-stack-docker-compose).
 
-Period leftover pairs (ADR 0048 / 0049 / 0119 / 0162 / 0163 / 0164) are computed in
-`lineageweave/leftover_pairs.py` from the residual after a real
+Period leftover pairs (ADR 0048 / 0049 / 0119 / 0126 / 0162 / 0163 / 0164) are
+computed in `lineageweave/leftover_pairs.py` from the residual after a real
 GRM/GPCM score, never invented. Distances are Euclidean on the
 two-dimensional Gabriel leftover map; missing cells stay out of
 the factorization. Closest and farthest post–criterion pairs
 persist to `report_leftover_pair` with signed residual `R`, observed
 `Y`, and expected `E[Y|θ, item]` so `R = Y − E` remains auditable,
 plus leftover-map rank so rank 0 is not read as structure. They sit
-above the member list so a click opens that post.
+above the member list so a click opens that post. Pair-member
+leftover-map criterion nodes (ADR 0126) also open that leftover-pair
+post; non-pair criteria stay non-interactive.
 
 `frontend/` has its own toolchain (Node pinned via `frontend/mise.toml`,
 pnpm via Corepack -- do not add a second Node package manager or a

ARCHITECTURE.md

@@ -596,7 +596,9 @@ persist to `report_item_information`. After those IRT main effects,
 residual SVD leftover pairs on two Gabriel axes (Jeon et al., 2021;
 ADR 0048 / 0119 / 0162 / 0163 / 0164) persist to `report_leftover_pair` with
 signed residual `R`, observed `Y`, expected `E[Y|θ, item]`, and full
-leftover-map rank. Results persist to
+leftover-map rank. Complete-case leftover-map coordinates ξ / ζ
+(ADR 0121) persist to `report_leftover_map_person` and
+`report_leftover_map_item`. Results persist to
 `report_period_score` / `report_member_score`.
 `GET /api/reports/{grouping}` lists the trend;
 `GET /api/reports/{grouping}/{period}` is ABAC-filtered;
@@ -610,7 +612,9 @@ open ticket title, status lookup label, and due date when one exists. The home p
 the actual mean θ, the FIPC delta, the CAT-selected item, leftover
 closest/farthest pairs (signed residual `R`, observed `Y`, expected
 `E`, full rank, and two-axis leftover-map distance `d` after IRT main
-effects) above the member list, and the
+effects) and the leftover interaction map above the member list
+(click a leftover-map person or pair-member criterion node to open
+that post), and the
 PU / corp / thread comparison -- never a placeholder. TEPP is unchanged.
 
 ## Phase 6b: Knowledge Graph as a real Ontology + Semantic Layer

CHANGELOG.d/2.12.10-leftover-map-criterion-node.md

@@ -0,0 +1,8 @@
+## 2.12.10 — Leftover-map criterion nodes
+
+- Pair-member leftover-map criterion nodes open the leftover-pair post
+  (closest preferred, then farthest). Non-pair criteria stay
+  non-interactive diamonds so the map does not invent a next action
+  (ADR 0126).
+- Keyboard Enter/Space matches leftover-map person nodes. Click does
+  not land leftover focus on Post quality.

CHANGELOG.d/2.12.7-leftover-interaction-map.md

@@ -0,0 +1,12 @@
+## 2.12.7 — Leftover interaction map
+
+- Persist complete-case leftover-map coordinates ξ / ζ after IRT main
+  effects (Jeon et al., 2021; Gabriel, 1971) in
+  `report_leftover_map_person` and `report_leftover_map_item` (ADR 0121).
+- Render a 2D leftover interaction map above the leftover pair list.
+  Click a post node to open that post. Closest and farthest pairs stay
+  highlighted. Hidden posts stay hidden.
+- Rank-0 and rank-1 maps pad the unused axis with zero. Missing residual
+  cells never enter the factorization as zero.
+- Replay `0103_tenant_settings.sql` and
+  `0172_report_leftover_interaction_map.sql` on existing volumes.

CHANGELOG.md

@@ -98,6 +98,25 @@ All notable changes to this project are documented here. Format follows
   leftover pairs above the member list still open that post. Never
   invent a leftover score or a theta.
 
+## [2.12.10] - 2026-08-23
+
+### Added
+
+- Pair-member leftover-map criterion nodes now open the leftover-pair
+  post (closest preferred, then farthest). Non-pair criteria stay
+  non-interactive. Click does not set leftover focus on Post quality
+  (ADR 0126).
+
+## [2.12.7] - 2026-08-23
+
+### Added
+
+- Period reports now persist leftover interaction-map coordinates
+  (Jeon et al. 2021 ξ / ζ after IRT main effects) and render a 2D
+  biplot above the leftover pair list. Click a post node to open that
+  post. Rank-0 and rank-1 maps pad the unused axis with zero; missing
+  cells stay out of the factorization (ADR 0121).
+
 ## [2.12.6] - 2026-08-20
 
 ### Added

backend/app/main.py

@@ -2371,6 +2371,12 @@ async def read_period_reports(
             if _can_see_post(account, pair)
             and not _is_synthetic_demo_member(pair, demo_entity_ids)
         ]
+        leftover_map_persons = [
+            person
+            for person in report.get("leftover_map_persons", [])
+            if _can_see_post(account, person)
+            and not _is_synthetic_demo_member(person, demo_entity_ids)
+        ]
         members = [
             {key: value for key, value in member.items() if key != "has_real_source_context"}
             for member in members
@@ -2379,8 +2385,20 @@ async def read_period_reports(
             {key: value for key, value in pair.items() if key != "has_real_source_context"}
             for pair in leftover_pairs
         ]
+        leftover_map_persons = [
+            {key: value for key, value in person.items() if key != "has_real_source_context"}
+            for person in leftover_map_persons
+        ]
+        leftover_map_items = list(report.get("leftover_map_items", []))
         visible.append(
-            {**report, "members": members, "leftover_pairs": leftover_pairs, "post_count": len(members)}
+            {
+                **report,
+                "members": members,
+                "leftover_pairs": leftover_pairs,
+                "leftover_map_persons": leftover_map_persons,
+                "leftover_map_items": leftover_map_items,
+                "post_count": len(members),
+            }
         )
     return {"grouping_kind": grouping_kind, "period_code": period_code, "reports": visible}
 

backend/app/report_ingestion.py

@@ -351,7 +351,7 @@ async def persist_period_report(
     period_code: str,
     report: PeriodReport,
 ) -> None:
-    """Replace the stored report, member scores, leftover pairs, and item bank."""
+    """Replace the stored report, member scores, leftover pairs, leftover map, and item bank."""
     await conn.execute(
         """
         delete from report_period_score
@@ -460,6 +460,38 @@ async def persist_period_report(
             pair.expected_response,
             pair.leftover_map_rank,
         )
+    for person in report.leftover_map_persons:
+        await conn.execute(
+            """
+            insert into report_leftover_map_person (
+                grouping_kind, grouping_key, period_code, rubric_version,
+                post_id, axis_one, axis_two
+            ) values ($1,$2,$3,$4,$5,$6,$7)
+            """,
+            grouping_kind,
+            grouping_key,
+            period_code,
+            RUBRIC_VERSION,
+            person.post_id,
+            person.axis_one,
+            person.axis_two,
+        )
+    for item in report.leftover_map_items:
+        await conn.execute(
+            """
+            insert into report_leftover_map_item (
+                grouping_kind, grouping_key, period_code, rubric_version,
+                criterion_code, axis_one, axis_two
+            ) values ($1,$2,$3,$4,$5,$6,$7)
+            """,
+            grouping_kind,
+            grouping_key,
+            period_code,
+            RUBRIC_VERSION,
+            item.criterion_code,
+            item.axis_one,
+            item.axis_two,
+        )
 
 
 def _groups_from_rows(
@@ -620,6 +652,32 @@ async def fetch_period_reports(
         period_code,
         RUBRIC_VERSION,
     )
+    # Safe SQL: the source-context expression is an immutable schema fragment; report keys are bound.
+    leftover_map_persons = await conn.fetch(  # nosemgrep: python.lang.security.audit.sqli.asyncpg-sqli.asyncpg-sqli
+        f"""
+        select lp.grouping_key, lp.post_id, lp.axis_one, lp.axis_two, p.post_title,
+               p.visibility_code, p.corporate_entity_id,
+               ({_SOURCE_CONTEXT_PRESENT_SQL}) as has_real_source_context
+        from report_leftover_map_person lp
+        join source_post p on p.post_id = lp.post_id
+        where lp.grouping_kind = $1 and lp.period_code = $2 and lp.rubric_version = $3
+        order by lp.grouping_key, p.post_title
+        """,
+        grouping_kind,
+        period_code,
+        RUBRIC_VERSION,
+    )
+    leftover_map_items = await conn.fetch(
+        """
+        select grouping_key, criterion_code, axis_one, axis_two
+        from report_leftover_map_item
+        where grouping_kind = $1 and period_code = $2 and rubric_version = $3
+        order by grouping_key, criterion_code
+        """,
+        grouping_kind,
+        period_code,
+        RUBRIC_VERSION,
+    )
     status_labels = await labels_for_codes(
         conn,
         [row["ticket_status_code"] for row in members if row["ticket_status_code"]],
@@ -633,6 +691,12 @@ async def fetch_period_reports(
     leftover_by_group: dict[str, list[asyncpg.Record]] = defaultdict(list)
     for row in leftover:
         leftover_by_group[row["grouping_key"]].append(row)
+    leftover_persons_by_group: dict[str, list[asyncpg.Record]] = defaultdict(list)
+    for row in leftover_map_persons:
+        leftover_persons_by_group[row["grouping_key"]].append(row)
+    leftover_items_by_group: dict[str, list[asyncpg.Record]] = defaultdict(list)
+    for row in leftover_map_items:
+        leftover_items_by_group[row["grouping_key"]].append(row)
     payload: list[dict[str, Any]] = []
     for header in headers:
         grouping_key = header["grouping_key"]
@@ -724,6 +788,26 @@ async def fetch_period_reports(
                     }
                     for row in leftover_by_group.get(header["grouping_key"], [])
                 ],
+                "leftover_map_persons": [
+                    {
+                        "post_id": str(row["post_id"]),
+                        "post_title": row["post_title"],
+                        "axis_one": float(row["axis_one"]),
+                        "axis_two": float(row["axis_two"]),
+                        "visibility_code": row["visibility_code"],
+                        "corporate_entity_id": str(row["corporate_entity_id"]),
+                        "has_real_source_context": bool(row["has_real_source_context"]),
+                    }
+                    for row in leftover_persons_by_group.get(header["grouping_key"], [])
+                ],
+                "leftover_map_items": [
+                    {
+                        "criterion_code": str(row["criterion_code"]),
+                        "axis_one": float(row["axis_one"]),
+                        "axis_two": float(row["axis_two"]),
+                    }
+                    for row in leftover_items_by_group.get(header["grouping_key"], [])
+                ],
             }
         )
     return payload

backend/tests/test_api.py

@@ -132,6 +132,11 @@
     / "migrations"
     / "0164_report_leftover_map_rank.sql"
 )
+_LEFTOVER_MAP_MIGRATION = (
+    Path(__file__).resolve().parents[2]
+    / "migrations"
+    / "0172_report_leftover_interaction_map.sql"
+)
 
 
 def _postgres_available() -> bool:
@@ -249,6 +254,7 @@ def seeded_db(demo_analyst_token):
             cur.execute(_CHANNEL_WEIGHT_MIGRATION.read_text())
             cur.execute(_LEFTOVER_OBSERVED_EXPECTED_MIGRATION.read_text())
             cur.execute(_LEFTOVER_MAP_RANK_MIGRATION.read_text())
+            cur.execute(_LEFTOVER_MAP_MIGRATION.read_text())
             cur.execute(
                 "insert into common_lookup_value (lookup_category, lookup_code, lookup_label) values "
                 "('corporate_entity_level', 'group', 'Group'), "
@@ -4732,6 +4738,18 @@ def test_seed_period_report_surfaces_on_get_reports(client, demo_analyst_token,
     assert leftover_kinds <= {"closest", "farthest"}
     assert all(pair["post_title"] for pair in high_report.get("leftover_pairs", []))
     assert all(pair["leftover_distance"] >= 0 for pair in high_report.get("leftover_pairs", []))
+    leftover_map_persons = high_report.get("leftover_map_persons", [])
+    leftover_map_items = high_report.get("leftover_map_items", [])
+    member_ids = {member["post_id"] for member in high_report["members"]}
+    item_codes = {item["item_code"] for item in high_report.get("selected_items", [])}
+    assert leftover_map_persons
+    assert leftover_map_items
+    assert all(person["post_title"] for person in leftover_map_persons)
+    assert {person["post_id"] for person in leftover_map_persons} <= member_ids
+    assert {item["criterion_code"] for item in leftover_map_items} <= item_codes
+    for point in leftover_map_persons + leftover_map_items:
+        assert isinstance(point["axis_one"], (int, float))
+        assert isinstance(point["axis_two"], (int, float))
     for pair in high_report.get("leftover_pairs", []):
         assert pair["leftover_map_rank"] >= 0
         observed = pair.get("observed_response")

docs/adr/0003-fast-mlsirm-report-integration.md

@@ -100,7 +100,7 @@ than one large PR:
    `information_polytomous` (Lord, 1980 max-info). Persist the ranking
    (`report_item_information`) and show the rank-1 item on the Period
    reports panel. Do not reimplement an information function here.
-7. **Leftover-pair slice** (shipped in 0.71.2; ADR 0017 / 0018): after
+7. **Leftover-pair slice** (shipped in 0.71.2; ADR 0048 / 0049): after
    IRT main effects, persist closest and farthest post–criterion pairs
    from the residual leftover map. Do not fork LSIRM; do not invent a
    leftover-pair API inside `fast-mlsirm` in this slice.

docs/adr/0121-persist-leftover-interaction-map.md

@@ -0,0 +1,76 @@
+# ADR 0121 — Persist leftover interaction-map coordinates
+
+**Decision status:** Accepted
+**Date:** 2026-08-23
+
+## Context
+
+ADR 0048 persists the closest and farthest leftover post–criterion
+pairs after IRT main effects. Those pairs are two cells on the Jeon
+et al. (2021, eq. 3) leftover interaction map `−γ‖ξ_p − ζ_i‖`. The
+Gabriel (1971) biplot that produces the pairs already computes person
+positions `ξ` and item positions `ζ`, then discards them. A buyer
+who sees only two named pairs cannot see *why* those cells sat
+closest or farthest, or where the other complete-case posts and
+criteria sit on the same leftover map.
+
+`fast-mlsirm` still exposes no leftover-map API. LineageWeave must
+not fork LSIRM, invent a second IRT fit, or treat a missing residual
+cell as a zero residual.
+
+## Decision
+
+After a real GRM/GPCM score, keep the complete-case Gabriel
+coordinates that leftover pairs already use. Persist every complete-
+case post as `report_leftover_map_person` (`axis_one`, `axis_two`)
+and every complete-case criterion as `report_leftover_map_item`.
+Pad unused axes with zero when residual rank is below two. Do not
+invent a second component. Closest/farthest selection and persisted
+distance use those same two buyer-visible axes; unpersisted higher
+components never silently change a highlighted map pair. Incomplete
+rows and columns stay out of the factorization.
+
+Cascade the rows with `report_period_score`. A leftover-map post
+must also be a `report_member_score` row. A leftover-map criterion
+must be a `report_item_information` item on that same report. Do not
+store a second theta. A rank-0 residual still emits origin
+coordinates so `make seed` is not empty; those zeros are not a
+fabricated interaction.
+
+Closest and farthest pairs remain ADR 0048 / ADR 0049. The map sits
+**above** that pair list on the period-report group. Clicking a
+person node opens that post with the same handler as a leftover
+pair. Pair-member criterion nodes open that leftover-pair post
+([ADR 0126](0126-leftover-map-criterion-node.md)). Hidden posts stay hidden: leftover-map persons join
+`source_post` and use the same ABAC gate as members and leftover
+pairs. Missing map rows render nothing.
+
+The biplot remains in `lineageweave/leftover_pairs.py` so leftover
+tests do not import `period_report` or `fast_mlsirm`.
+
+## Consequences
+
+Rebuild and seed write leftover-map coordinates in the same
+transaction as leftover pairs. `GET /api/reports/{grouping}/{period}`
+returns `leftover_map_persons` (with post title) and
+`leftover_map_items`. Migration
+`0172_report_leftover_interaction_map.sql` upgrades volumes that
+already applied `0001` / `0012`. `migrate.sh` also replays `0103`
+(`tenant_settings`) so existing volumes pick up that table.
+
+## Related
+
+Depends on [ADR 0048](0048-persist-lsirm-leftover-pairs.md),
+[ADR 0049](0049-leftover-pair-report-ui.md), and
+[ADR 0003](0003-fast-mlsirm-report-integration.md).
+
+## References
+
+Gabriel, K. R. (1971). The biplot graphic display of matrices with
+application to principal component analysis. *Biometrika, 58*(3),
+453–467. https://doi.org/10.1093/biomet/58.3.453
+
+Jeon, M., Jin, I. H., Schweinberger, M., & Baugh, S. (2021). Mapping
+unobserved item–respondent interactions: A latent space item response
+model with interaction map. *Psychometrika, 86*(2), 378–403.
+https://doi.org/10.1007/s11336-021-09762-5