Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -186,12 +186,13 @@ in the same spirit) -- never against real data, per the hard rule above.
against a live local stack (`make up`) and self-skip without one -- see
[README.md](README.md#local-product-stack-docker-compose).

Period leftover pairs (ADR 0017 / 0018) are computed in
Period leftover pairs (ADR 0048 / 0049 / 0181) are computed in
`lineageweave/leftover_pairs.py` from the residual after a real
GRM/GPCM score, never invented. Missing cells stay out of the
Gabriel factorization. Closest and farthest post–criterion pairs
persist to `report_leftover_pair` and sit above the member list so
a click opens that post.
persist to `report_leftover_pair` with leftover-map lengths `‖ξ‖`
and `‖ζ‖` when a complete-case leftover map exists, and sit above
the member list so a click opens that post.

`frontend/` has its own toolchain (Node pinned via `frontend/mise.toml`,
pnpm via Corepack -- do not add a second Node package manager or a
Expand Down
5 changes: 3 additions & 2 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -588,8 +588,9 @@ on those same fixed parameters (Kim, 2006 FIPC). After scoring,
`information_polytomous` ranks the shared-bank items by Fisher
information at the group's mean θ (Lord, 1980 max-info CAT). Rankings
persist to `report_item_information`. After those IRT main effects,
residual SVD leftover pairs (Jeon et al., 2021; ADR 0017) persist to
`report_leftover_pair`. Results persist to
residual SVD leftover pairs (Jeon et al., 2021; ADR 0048 / 0181) persist to
`report_leftover_pair` with leftover-map lengths `‖ξ‖` and `‖ζ‖` when
the complete-case leftover map exists. Results persist to
`report_period_score` / `report_member_score`.
`GET /api/reports/{grouping}` lists the trend;
`GET /api/reports/{grouping}/{period}` is ABAC-filtered;
Expand Down
8 changes: 8 additions & 0 deletions CHANGELOG.d/2.12.25-leftover-map-length.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
## 2.12.25 — Leftover-map length

- Persist leftover-map lengths `‖ξ‖` and `‖ζ‖` on leftover
post–criterion pairs (ADR 0181). After `make seed`, closest and
farthest leftover pairs sit above the member list with `‖ξ‖` and
`‖ζ‖` next to leftover-map distance `d`; click opens that post.
Omit the badges when lengths are missing. Never invent a leftover
score.
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,17 @@ All notable changes to this project are documented here. Format follows
environment, so local OIDC and synthetic-data workflows resolve the same
pinned dependencies as CI.

## [2.12.25] - 2026-08-24

### Added

- Period leftover pairs now name leftover-map lengths `‖ξ‖` and `‖ζ‖`
next to leftover-map distance `d`. After `make seed`, closest and
farthest leftover pairs sit above the member list with named
leftover-map magnitude; click opens that post (Gabriel, 1971;
Jeon et al., 2021, eq. 3; ADR 0181). A missing length omits the
badge. Never invent a leftover score.

## [2.12.6] - 2026-08-20

### Added
Expand Down
4 changes: 3 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,9 @@ chip name contains `Corporate entity: Demo Corp` and the persisted
mean θ. The period-report panel says Demo Corp is the opened grouping
and to read its mean θ and member posts, then open a post. Those
members land immediately under that next action, ahead of Other Corp
and the week strip. Opening Public post names the next action: read
and the week strip. After `make seed`, leftover closest/farthest pairs
sit above the member list with leftover-map lengths `‖ξ‖` and `‖ζ‖`
next to leftover-map distance `d`. Opening Public post names the next action: read
Event Lineage, Keyman, and evaluation on that post. The popup Event
Lineage DAG marks that post current. After that current node, the
popup names Keyman and evaluation as the next read. After landed
Expand Down
21 changes: 18 additions & 3 deletions backend/app/report_ingestion.py
Original file line number Diff line number Diff line change
Expand Up @@ -443,8 +443,9 @@ async def persist_period_report(
"""
insert into report_leftover_pair (
grouping_kind, grouping_key, period_code, rubric_version,
pair_kind, post_id, criterion_code, leftover_distance, leftover_residual
) values ($1,$2,$3,$4,$5,$6,$7,$8,$9)
pair_kind, post_id, criterion_code, leftover_distance, leftover_residual,
leftover_map_person_length, leftover_map_item_length
) values ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11)
""",
grouping_kind,
grouping_key,
Expand All @@ -455,6 +456,8 @@ async def persist_period_report(
pair.criterion_code,
pair.leftover_distance,
pair.leftover_residual,
pair.leftover_map_person_length,
pair.leftover_map_item_length,
)


Expand Down Expand Up @@ -601,7 +604,9 @@ async def fetch_period_reports(
leftover = await conn.fetch( # nosemgrep: python.lang.security.audit.sqli.asyncpg-sqli.asyncpg-sqli
f"""
select lp.grouping_key, lp.pair_kind, lp.post_id, lp.criterion_code,
lp.leftover_distance, lp.leftover_residual, p.post_title,
lp.leftover_distance, lp.leftover_residual,
lp.leftover_map_person_length, lp.leftover_map_item_length,
p.post_title,
p.visibility_code, p.corporate_entity_id,
({_SOURCE_CONTEXT_PRESENT_SQL}) as has_real_source_context
from report_leftover_pair lp
Expand Down Expand Up @@ -698,6 +703,16 @@ async def fetch_period_reports(
"criterion_code": str(row["criterion_code"]),
"leftover_distance": float(row["leftover_distance"]),
"leftover_residual": float(row["leftover_residual"]),
"leftover_map_person_length": (
None
if row["leftover_map_person_length"] is None
else float(row["leftover_map_person_length"])
),
"leftover_map_item_length": (
None
if row["leftover_map_item_length"] is None
else float(row["leftover_map_item_length"])
),
"visibility_code": row["visibility_code"],
"corporate_entity_id": str(row["corporate_entity_id"]),
"has_real_source_context": bool(row["has_real_source_context"]),
Expand Down
8 changes: 8 additions & 0 deletions backend/tests/test_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -4518,6 +4518,14 @@ def test_seed_period_report_surfaces_on_get_reports(client, demo_analyst_token,
assert leftover_kinds <= {"closest", "farthest"}
assert all(pair["post_title"] for pair in high_report.get("leftover_pairs", []))
assert all(pair["leftover_distance"] >= 0 for pair in high_report.get("leftover_pairs", []))
assert all(
pair.get("leftover_map_person_length") is None or pair["leftover_map_person_length"] >= 0
for pair in high_report.get("leftover_pairs", [])
)
assert all(
pair.get("leftover_map_item_length") is None or pair["leftover_map_item_length"] >= 0
for pair in high_report.get("leftover_pairs", [])
)

week3 = client.get(
"/api/reports/process_unit/2026-W03",
Expand Down
2 changes: 1 addition & 1 deletion docker/postgres-init/migrate.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ for migration in /opt/lineageweave/migrations/*.sql; do
migration_name=${migration##*/}
case "$migration_name" in
0012_*|0013_*|0014_*|0015_*|0016_*|0017_*|0018_*|0019_*|0020_*|0021_*|0022_*|0023_*|0024_*|0025_*|0026_*|0027_*|0028_*|0029_*|0030_*|0031_*|0032_*|0033_*|0034_*|0035_*|0036_*|0037_*|0038_*|0039_*|0040_*|0041_*|0042_*|0043_*|0044_*|0045_*|0046_*|0047_*|0048_*|0049_*|0050_*) ;;
0060_*|0100_*|0101_*|0102_*) ;;
0060_*|0100_*|0101_*|0102_*|0181_*) ;;
*) continue ;;
esac
printf 'Applying %s\n' "$migration_name"
Expand Down
9 changes: 6 additions & 3 deletions docs/adr/0003-fast-mlsirm-report-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,10 +100,13 @@ than one large PR:
`information_polytomous` (Lord, 1980 max-info). Persist the ranking
(`report_item_information`) and show the rank-1 item on the Period
reports panel. Do not reimplement an information function here.
7. **Leftover-pair slice** (shipped in 0.71.2; ADR 0017 / 0018): after
7. **Leftover-pair slice** (shipped in 0.71.2; ADR 0048 / 0181): after
IRT main effects, persist closest and farthest post–criterion pairs
from the residual leftover map. Do not fork LSIRM; do not invent a
leftover-pair API inside `fast-mlsirm` in this slice.
from the residual leftover map, and name leftover-map lengths
`‖ξ‖` and `‖ζ‖` when a complete-case leftover map exists so
leftover-map magnitude is not read as leftover-map distance. Do not
fork LSIRM; do not invent a leftover-pair API inside `fast-mlsirm`
in this slice.

**TEPP boundary.** [ARCHITECTURE.md](../../ARCHITECTURE.md) already
assigns calibrated temporal/event measurement to
Expand Down
1 change: 1 addition & 0 deletions docs/adr/0048-persist-lsirm-leftover-pairs.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

**Decision status:** Accepted
**Date:** 2026-08-17
**Amended by:** [ADR 0181](0181-leftover-map-length.md) (leftover-map length)

## Context

Expand Down
18 changes: 13 additions & 5 deletions docs/adr/0049-leftover-pair-report-ui.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,16 +17,24 @@ second navigation surface.

On each period-report group, render leftover pairs **above** the
member list. Each pair is a button: closest or farthest label, post
title, criterion short label, leftover-map distance, and the next
action (“Open this post to read the criterion it sat closest to /
farthest from after main effects.”). Clicking the button opens that
post with the same handler as a member row.
title, criterion short label, leftover-map distance, leftover-map
lengths `‖ξ‖` and `‖ζ‖` when present, and the next action.
Length next action: leftover-map length names leftover-map magnitude
independently of leftover-map distance; open this post to read the
named criterion. When lengths are missing, keep “Open this post to
read the criterion it sat closest to / farthest from after main
effects.” Clicking the button opens that post with the same handler
as a member row.

After `make seed`, closest and farthest leftover pairs sit above the
member list. Click a pair to open that post.
member list with leftover-map length next to leftover-map distance
`d`. Click a pair to open that post.

Missing leftover rows render nothing — never a placeholder pair.
A hidden post never appears as a leftover pair.
A missing leftover-map length omits the `‖ξ‖` / `‖ζ‖` badges and
keeps the closest/farthest next action. See
[ADR 0181](0181-leftover-map-length.md).

## Consequences

Expand Down
85 changes: 85 additions & 0 deletions docs/adr/0181-leftover-map-length.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
# ADR 0181 — Name leftover-map length on period-report pair rows

**Decision status:** Accepted
**Date:** 2026-08-24

Amends [ADR 0048](0048-persist-lsirm-leftover-pairs.md) and
[ADR 0049](0049-leftover-pair-report-ui.md).

## Context

ADR 0048 already persists leftover-map distance `d = ‖ξ_p − ζ_i‖` and
leftover residual `R = Y − E[Y|θ, item]` on `report_leftover_pair`.
ADR 0049 already renders closest and farthest pairs above the member
list and opens the named post. Distance is the Jeon et al. (2021,
eq. 3) map gap. Gabriel (1971) also names leftover-map vector lengths
`‖ξ‖` and `‖ζ‖`. Hiding those lengths lets a buyer read a close
leftover-map pair as leftover-map aligned magnitude, or a distant pair
as large leftover-map displacement, without the polar magnitude.

A close leftover-map pair can sit at the origin (`‖ξ‖ = ‖ζ‖ = 0`) or
share a non-origin position (`d = 0` with positive length). Those are
different leftover-map facts.

This increment does not persist leftover-map coordinates, does not name
observed `Y` / expected `E`, does not name leftover-map cosine, does
not name leftover-map inner product, does not name leftover-map rank,
does not split leftover-map distance onto two axes, and does not land
Post quality on the leftover criterion.

The unprotected-stack reconstructions for neighbouring leftover facts
use 0121–0180. This protected-main increment uses **0181** so it does
not collide with leftover-map cosine (0180), leftover-map inner product
(0179), leftover residual disclosure (0178), leftover observed `Y` /
expected `E` (0177), leftover-map rank (0172), two-axis leftover-map
distance (0166), leftover coverage (0168), leftover-map axis share
(0148), or leftover interaction-map persistence (0121).

## Decision

Each leftover pair names `leftover_map_person_length` and
`leftover_map_item_length` — the Euclidean lengths `‖ξ‖` and `‖ζ‖` of
the leftover-map person and item coordinates that produced leftover-map
distance `d`. Migration `0181` is the single source of the columns on
every install path, fresh or existing -- shipped migrations (`0001` /
`0012`) are never edited after the fact. The columns are nullable so
older leftover rows keep distance and residual without fabricating
lengths. Fallback pairs that have no complete-case leftover map omit
the values rather than inventing them. Origin coordinates persist
length `0` because that is the measured leftover-map magnitude.

The pair button shows `‖ξ‖ {person}` and `‖ζ‖ {item}` next to
leftover-map distance `d` when both values are finite. Next action:
leftover-map length names leftover-map magnitude independently of
leftover-map distance; open this post to read the named criterion. A
missing or non-finite length omits the badges and keeps the existing
closest/farthest next action. Do not invent a leftover score. Do not
invent a theta.

## Consequences

`GET /api/reports/{grouping}/{period}` returns
`leftover_map_person_length` and `leftover_map_item_length`. After
`make seed`, closest and farthest leftover pairs sit above the member
list with named `‖ξ‖` and `‖ζ‖` next to `d`; click opens that post.
Hidden posts stay hidden.

## Related

Independent of leftover interaction-map persistence, leftover-criterion
evaluation landing, leftover residual disclosure, leftover observed
`Y` / expected `E`, leftover-map complete-case coverage, leftover-map
axis share, leftover pairs on the grouping comparison strip, two-axis
leftover-map distance, leftover-map rank, leftover-map inner product,
and leftover-map cosine.

## References

Gabriel, K. R. (1971). The biplot graphic display of matrices with
application to principal component analysis. *Biometrika, 58*(3),
453–467. https://doi.org/10.1093/biomet/58.3.453

Jeon, M., Jin, I. H., Schweinberger, M., & Baugh, S. (2021). Mapping
unobserved item–respondent interactions: A latent space item response
model with interaction map. *Psychometrika, 86*(2), 378–403.
https://doi.org/10.1007/s11336-021-09762-5
2 changes: 1 addition & 1 deletion frontend/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "frontend",
"private": true,
"version": "2.12.6",
"version": "2.12.25",
"type": "module",
"scripts": {
"dev": "vite",
Expand Down
12 changes: 10 additions & 2 deletions frontend/src/App.test.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -932,6 +932,8 @@ describe("App, authenticated", () => {
criterion_code: "sales_lead_specificity",
leftover_distance: 0.12,
leftover_residual: 0.4,
leftover_map_person_length: 0.4,
leftover_map_item_length: 0.5,
},
{
pair_kind: "farthest",
Expand All @@ -940,6 +942,8 @@ describe("App, authenticated", () => {
criterion_code: "general_sentiment_negative",
leftover_distance: 1.84,
leftover_residual: -1.1,
leftover_map_person_length: 1.2,
leftover_map_item_length: 0.9,
},
],
members: [
Expand Down Expand Up @@ -3357,13 +3361,17 @@ describe("App, authenticated", () => {
});
expect(closestPair).toHaveTextContent("Closest leftover: Public post · sales-lead");
expect(closestPair).toHaveTextContent(
"Open this post to read the criterion it sat closest to after main effects.",
"Leftover-map length ‖ξ‖ 0.40 and ‖ζ‖ 0.50 names leftover-map magnitude independently of leftover-map distance. Open this post to read sales-lead.",
);
expect(closestPair).toHaveTextContent("‖ξ‖ 0.40");
expect(closestPair).toHaveTextContent("‖ζ‖ 0.50");
expect(closestPair).toHaveTextContent("d 0.12");
expect(farthestPair).toHaveTextContent("Farthest leftover: Specification revision requested · negative");
expect(farthestPair).toHaveTextContent(
"Open this post to read the criterion it sat farthest from after main effects.",
"Leftover-map length ‖ξ‖ 1.20 and ‖ζ‖ 0.90 names leftover-map magnitude independently of leftover-map distance. Open this post to read negative.",
);
expect(farthestPair).toHaveTextContent("‖ξ‖ 1.20");
expect(farthestPair).toHaveTextContent("‖ζ‖ 0.90");
expect(farthestPair).toHaveTextContent("d 1.84");
const memberButton = screen.getByRole("button", { name: /open report post: public post/i });
expect(closestPair.compareDocumentPosition(memberButton) & Node.DOCUMENT_POSITION_FOLLOWING).toBeTruthy();
Expand Down
24 changes: 20 additions & 4 deletions frontend/src/App.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,12 @@ import {
useLocale,
} from "./i18n";
import { rememberOidcReturnUrl, returnUrlFromLocation } from "./oidcReturnUrl";
import {
formatLeftoverMapItemLength,
formatLeftoverMapLength,
formatLeftoverMapPersonLength,
LEFTOVER_MAP_LENGTH_ACTION,
} from "./leftoverMapLength";
import "./App.css";

function orchestratorUnavailableMessage(err: unknown, action: string): string {
Expand Down Expand Up @@ -3394,11 +3400,19 @@ function ReportsPanel({
{report.leftover_pairs.map((pair) => {
const kindLabel =
pair.pair_kind === "farthest" ? "Farthest leftover" : "Closest leftover";
const nextAction =
pair.pair_kind === "farthest"
? "Open this post to read the criterion it sat farthest from after main effects."
: "Open this post to read the criterion it sat closest to after main effects.";
const criterion = criterionShortLabel(pair.criterion_code);
const personLength = formatLeftoverMapPersonLength(pair.leftover_map_person_length);
const itemLength = formatLeftoverMapItemLength(pair.leftover_map_item_length);
const nextAction =
personLength === null || itemLength === null
? pair.pair_kind === "farthest"
? "Open this post to read the criterion it sat farthest from after main effects."
: "Open this post to read the criterion it sat closest to after main effects."
: tf(LEFTOVER_MAP_LENGTH_ACTION, {
person: formatLeftoverMapLength(pair.leftover_map_person_length) ?? "—",
item: formatLeftoverMapLength(pair.leftover_map_item_length) ?? "—",
criterion,
});
return (
<li
key={`${pair.pair_kind}:${pair.post_id}:${pair.criterion_code}`}
Expand All @@ -3413,6 +3427,8 @@ function ReportsPanel({
{kindLabel}: {pair.post_title} · {criterion}
</span>
<span className="post-badge">{nextAction}</span>
{personLength ? <span className="post-badge">{personLength}</span> : null}
{itemLength ? <span className="post-badge">{itemLength}</span> : null}
<span className="post-badge">d {pair.leftover_distance.toFixed(2)}</span>
</button>
</li>
Expand Down
2 changes: 2 additions & 0 deletions frontend/src/api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -759,6 +759,8 @@ export interface LeftoverPair {
criterion_code: string;
leftover_distance: number;
leftover_residual: number;
leftover_map_person_length?: number | null;
leftover_map_item_length?: number | null;
}

export interface PeriodGroupReport {
Expand Down
Loading
Loading