Skip to content
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
1bd45fc
feat: persist leftover observed Y and expected E (v2.12.20)
seonghobae Aug 23, 2026
a30bb28
Repair inherited login build boundary
seonghobae Aug 23, 2026
df18ed6
Move leftover evidence contract to ADR 0177
seonghobae Aug 23, 2026
e05d63b
feat: name leftover-map rank on leftover pairs (v2.12.21)
seonghobae Aug 23, 2026
6fcf603
Move leftover-map rank contract to ADR 0172
seonghobae Aug 23, 2026
1834e7d
feat: name leftover residual on period-report pair rows (v2.12.22)
seonghobae Aug 23, 2026
2ca0974
fix: restore frontend build on leftover residual disclosure
seonghobae Aug 23, 2026
54f3f69
fix: restore frontend build on leftover-map PR
seonghobae Aug 23, 2026
cb5b21f
Merge remote-tracking branch 'origin/feat/board-source-detail-state-f…
seonghobae Aug 23, 2026
435992d
fix: show leftover Y and E after stacking onto the workspace branch
seonghobae Aug 23, 2026
dc508ae
Merge remote-tracking branch 'origin/feat/leftover-pair-observed-expe…
seonghobae Aug 23, 2026
a988ca6
fix: keep leftover-map rank after stacking onto leftover Y/E
seonghobae Aug 23, 2026
9b0e47e
Merge remote-tracking branch 'origin/feat/leftover-map-rank-v21221' i…
seonghobae Aug 23, 2026
13d95e1
Merge remote-tracking branch 'origin/feat/oidc-return-remember-login-…
seonghobae Aug 23, 2026
de4ff43
Merge remote-tracking branch 'origin/feat/leftover-pair-observed-expe…
seonghobae Aug 23, 2026
9d4ccf3
Merge remote-tracking branch 'origin/feat/leftover-map-rank-v21221' i…
seonghobae Aug 23, 2026
e1d2442
Merge remote-tracking branch 'origin/feat/oidc-return-remember-login-…
seonghobae Aug 24, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -186,12 +186,13 @@ in the same spirit) -- never against real data, per the hard rule above.
against a live local stack (`make up`) and self-skip without one -- see
[README.md](README.md#local-product-stack-docker-compose).

Period leftover pairs (ADR 0017 / 0018) are computed in
`lineageweave/leftover_pairs.py` from the residual after a real
GRM/GPCM score, never invented. Missing cells stay out of the
Period leftover pairs (ADR 0017 / 0018 / 0048 / 0049 / 0177) are
computed in `lineageweave/leftover_pairs.py` from the residual after a
real GRM/GPCM score, never invented. Missing cells stay out of the
Gabriel factorization. Closest and farthest post–criterion pairs
persist to `report_leftover_pair` and sit above the member list so
a click opens that post.
persist to `report_leftover_pair` with observed `Y` and expected
`E[Y|θ, item]` so residual reconciles to `Y − E`, and sit above the
member list so a click opens that post.

`frontend/` has its own toolchain (Node pinned via `frontend/mise.toml`,
pnpm via Corepack -- do not add a second Node package manager or a
Expand Down
8 changes: 5 additions & 3 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -588,8 +588,9 @@ on those same fixed parameters (Kim, 2006 FIPC). After scoring,
`information_polytomous` ranks the shared-bank items by Fisher
information at the group's mean θ (Lord, 1980 max-info CAT). Rankings
persist to `report_item_information`. After those IRT main effects,
residual SVD leftover pairs (Jeon et al., 2021; ADR 0017) persist to
`report_leftover_pair`. Results persist to
residual SVD leftover pairs (Jeon et al., 2021; ADR 0017 / 0048 / 0177)
persist to `report_leftover_pair` with observed `Y` and expected
`E[Y|θ, item]`. Results persist to
`report_period_score` / `report_member_score`.
`GET /api/reports/{grouping}` lists the trend;
`GET /api/reports/{grouping}/{period}` is ABAC-filtered;
Expand All @@ -601,7 +602,8 @@ bank as the dummy high/low band rows, so comparison-strip click
through opens those DAG posts. Report members include the earliest
open ticket title, status lookup label, and due date when one exists. The home page renders
the actual mean θ, the FIPC delta, the CAT-selected item, leftover
closest/farthest pairs above the member list, and the
closest/farthest pairs (observed `Y` and expected `E` after IRT main
effects plus leftover-map distance `d`) above the member list, and the
PU / corp / thread comparison -- never a placeholder. TEPP is unchanged.

## Phase 6b: Knowledge Graph as a real Ontology + Semantic Layer
Expand Down
8 changes: 8 additions & 0 deletions CHANGELOG.d/2.12.20-leftover-observed-expected.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
## 2.12.20 — Leftover observed Y and expected E

- Persist observed `Y` and expected `E[Y|θ, item]` on leftover
post–criterion pairs (ADR 0177). Residual stays `R = Y − E`. After
`make seed`, closest and farthest leftover pairs sit above the member
list with `Y` and `E` next to leftover-map distance `d`; click opens
that post. Omit the badge when either value is missing. Never invent
a leftover score or a theta.
8 changes: 8 additions & 0 deletions CHANGELOG.d/2.12.21-leftover-map-rank.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
## 2.12.21 — Leftover-map rank

- Persist leftover-map rank on leftover post–criterion pairs (ADR 0172).
Rank is the number of Gabriel singular values above the floor. After
`make seed`, closest and farthest leftover pairs sit above the member
list with `rank {n}` next to leftover-map distance `d`; click opens
that post. Rank 0 names no leftover structure. Never invent a leftover
score or a theta.
7 changes: 7 additions & 0 deletions CHANGELOG.d/2.12.22-leftover-residual-disclosure.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
## 2.12.22 — Leftover residual disclosure

- Name signed leftover residual `R = Y − E[Y|θ, item]` on leftover
post–criterion pair rows (ADR 0178). After `make seed`, closest and
farthest leftover pairs sit above the member list with `R` next to
leftover-map distance `d`; click opens that post. A non-finite
residual is an em dash, never a fabricated leftover score.
28 changes: 28 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -203,6 +203,34 @@ All notable changes to this project are documented here. Format follows
- The static SQL review contract now counts the Customer Master evidence query
that uses closed schema fragments and bound entity ids.

## [2.12.20] - 2026-08-24

### Added

- Period leftover pair rows now name observed `Y` and expected
`E[Y|θ, item]` after IRT main effects next to leftover-map distance
`d`, then open that post (Jeon et al., 2021, eq. 3; ADR 0177). Residual
stays `R = Y − E`. Missing or non-finite `Y` / `E` omit the badge
rather than inventing a leftover score.

## [2.12.21] - 2026-08-24

### Added

- Period leftover pair rows now name leftover-map rank after IRT main
effects next to leftover-map distance `d`, then open that post
(Jeon et al., 2021, eq. 3; ADR 0172). Rank 0 names no leftover
structure rather than inventing a leftover score.

## [2.12.22] - 2026-08-24

### Added

- Period leftover pair rows now name signed leftover residual
`R = Y − E[Y|θ, item]` next to leftover-map distance `d`, then open
that post (Jeon et al., 2021, eq. 3; ADR 0178). A non-finite residual
is an em dash rather than a fabricated leftover score.

## [2.12.6] - 2026-08-20

### Added
Expand Down
4 changes: 3 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,9 @@ chip name contains `Corporate entity: Demo Corp` and the persisted
mean θ. The period-report panel says Demo Corp is the opened grouping
and to read its mean θ and member posts, then open a post. Those
members land immediately under that next action, ahead of Other Corp
and the week strip. Opening Public post names the next action: read
and the week strip. Leftover closest/farthest pairs name observed `Y`
and expected `E` after IRT main effects next to leftover-map distance
`d`; click a pair to open that post. Opening Public post names the next action: read
Event Lineage, Keyman, and evaluation on that post. The popup Event
Lineage DAG marks that post current. After that current node, the
popup names Keyman and evaluation as the next read. After landed
Expand Down
26 changes: 23 additions & 3 deletions backend/app/report_ingestion.py
Original file line number Diff line number Diff line change
Expand Up @@ -443,8 +443,9 @@ async def persist_period_report(
"""
insert into report_leftover_pair (
grouping_kind, grouping_key, period_code, rubric_version,
pair_kind, post_id, criterion_code, leftover_distance, leftover_residual
) values ($1,$2,$3,$4,$5,$6,$7,$8,$9)
pair_kind, post_id, criterion_code, leftover_distance, leftover_residual,
observed_response, expected_response, leftover_map_rank
) values ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12)
""",
grouping_kind,
grouping_key,
Expand All @@ -455,6 +456,9 @@ async def persist_period_report(
pair.criterion_code,
pair.leftover_distance,
pair.leftover_residual,
pair.observed_response,
pair.expected_response,
pair.leftover_map_rank,
)


Expand Down Expand Up @@ -602,7 +606,8 @@ async def fetch_period_reports(
leftover = await conn.fetch( # nosemgrep: python.lang.security.audit.sqli.asyncpg-sqli.asyncpg-sqli
f"""
select lp.grouping_key, lp.pair_kind, lp.post_id, lp.criterion_code,
lp.leftover_distance, lp.leftover_residual, p.post_title,
lp.leftover_distance, lp.leftover_residual,
lp.observed_response, lp.expected_response, lp.leftover_map_rank, p.post_title,
p.visibility_code, p.corporate_entity_id,
p.author_account_id, p.source_detail_state_code,
({_SOURCE_CONTEXT_PRESENT_SQL}) as has_real_source_context
Expand Down Expand Up @@ -702,6 +707,21 @@ async def fetch_period_reports(
"criterion_code": str(row["criterion_code"]),
"leftover_distance": float(row["leftover_distance"]),
"leftover_residual": float(row["leftover_residual"]),
"observed_response": (
None
if row["observed_response"] is None
else float(row["observed_response"])
),
"expected_response": (
None
if row["expected_response"] is None
else float(row["expected_response"])
),
"leftover_map_rank": (
None
if row["leftover_map_rank"] is None
else int(row["leftover_map_rank"])
),
"visibility_code": row["visibility_code"],
"corporate_entity_id": str(row["corporate_entity_id"]),
"author_account_id": str(row["author_account_id"]),
Expand Down
6 changes: 6 additions & 0 deletions backend/tests/test_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -5677,6 +5677,12 @@ def test_seed_period_report_surfaces_on_get_reports(client, demo_analyst_token,
assert leftover_kinds <= {"closest", "farthest"}
assert all(pair["post_title"] for pair in high_report.get("leftover_pairs", []))
assert all(pair["leftover_distance"] >= 0 for pair in high_report.get("leftover_pairs", []))
for pair in high_report.get("leftover_pairs", []):
observed = pair.get("observed_response")
expected = pair.get("expected_response")
if observed is None or expected is None:
continue
assert abs(pair["leftover_residual"] - (observed - expected)) < 1e-6

week3 = client.get(
"/api/reports/process_unit/2026-W03",
Expand Down
2 changes: 1 addition & 1 deletion docker/postgres-init/migrate.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ for migration in /opt/lineageweave/migrations/*.sql; do
migration_name=${migration##*/}
case "$migration_name" in
0012_*|0013_*|0014_*|0015_*|0016_*|0017_*|0018_*|0019_*|0020_*|0021_*|0022_*|0023_*|0024_*|0025_*|0026_*|0027_*|0028_*|0029_*|0030_*|0031_*|0032_*|0033_*|0034_*|0035_*|0036_*|0037_*|0038_*|0039_*|0040_*|0041_*|0042_*|0043_*|0044_*|0045_*|0046_*|0047_*|0048_*|0049_*|0050_*) ;;
0060_*|0100_*|0101_*|0102_*|0103_*|0104_*|0105_*|0106_*|0107_*|0108_*|0109_*|0110_*|0111_*|0112_*|0113_*|0114_*|0130_*|0133_*|0134_*|0136_*|0137_*|0138_*|0139_*|0176_*) ;;
0060_*|0100_*|0101_*|0102_*|0103_*|0104_*|0105_*|0106_*|0107_*|0108_*|0109_*|0110_*|0111_*|0112_*|0113_*|0114_*|0130_*|0133_*|0134_*|0136_*|0137_*|0138_*|0139_*|0172_*|0176_*|0177_*) ;;
*) continue ;;
esac
printf 'Applying %s\n' "$migration_name"
Expand Down
7 changes: 4 additions & 3 deletions docs/adr/0003-fast-mlsirm-report-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,10 +103,11 @@ than one large PR:
`information_polytomous` (Lord, 1980 max-info). Persist the ranking
(`report_item_information`) and show the rank-1 item on the Period
reports panel. Do not reimplement an information function here.
7. **Leftover-pair slice** (shipped in 0.71.2; ADR 0017 / 0018): after
7. **Leftover-pair slice** (shipped in 0.71.2; ADR 0017 / 0018 / 0048 / 0049 / 0177): after
IRT main effects, persist closest and farthest post–criterion pairs
from the residual leftover map. Do not fork LSIRM; do not invent a
leftover-pair API inside `fast-mlsirm` in this slice.
from the residual leftover map, naming observed `Y` and expected
`E[Y|θ, item]` so residual reconciles to `Y − E`. Do not fork LSIRM;
do not invent a leftover-pair API inside `fast-mlsirm` in this slice.

**TEPP boundary.** [ARCHITECTURE.md](../../ARCHITECTURE.md) already
assigns calibrated temporal/event measurement to
Expand Down
5 changes: 4 additions & 1 deletion docs/adr/0048-persist-lsirm-leftover-pairs.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

**Decision status:** Accepted
**Date:** 2026-08-17
**Amended by:** [ADR 0177](0177-leftover-observed-expected.md) (observed Y and expected E)

## Context

Expand Down Expand Up @@ -30,7 +31,9 @@ and one `farthest` observed cell per period report in
`report_leftover_pair` (3NF, two-or-more-word `snake_case`).

The biplot lives in `lineageweave/leftover_pairs.py` so leftover
tests do not import `period_report` or `fast_mlsirm`.
tests do not import `period_report` or `fast_mlsirm`. Each leftover
row also names observed `Y` and expected `E[Y|θ, item]` so residual
reconciles to `Y − E` (ADR 0177).

Cascade the rows with `report_period_score`. A leftover post must
also be a `report_member_score` row, and the leftover criterion
Expand Down
1 change: 1 addition & 0 deletions docs/adr/0049-leftover-pair-report-ui.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

**Decision status:** Accepted
**Date:** 2026-08-17
**Amended by:** [ADR 0177](0177-leftover-observed-expected.md) (observed Y and expected E)

## Context

Expand Down
71 changes: 71 additions & 0 deletions docs/adr/0172-leftover-map-rank.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# ADR 0172 — Name leftover-map rank on leftover pairs

**Decision status:** Accepted
**Date:** 2026-08-24

Amends [ADR 0048](0048-persist-lsirm-leftover-pairs.md) and
[ADR 0049](0049-leftover-pair-report-ui.md).

## Context

ADR 0048 already persists leftover-map distance and leftover residual
`R = Y − E[Y|θ, item]` on `report_leftover_pair`. A rank-0 residual
still emits a stable closest/farthest pair so `make seed` is not empty;
the stored distance is then zero, not a fabricated interaction. ADR 0049
renders those pairs above the member list. Without leftover-map rank, a
buyer cannot tell a Gabriel biplot with leftover structure (Jeon et al.,
2021, eq. 3) from an origin collapse that still shows `d 0.00`.

This increment does not persist leftover-map coordinates, does not name
observed `Y` / expected `E`, does not change leftover-map axis count, and
does not land Post quality on the leftover criterion.

The unprotected-stack ADR for the same buyer fact was 0164. This
protected-main reconstruction uses **0172** so it does not collide with
two-axis leftover-map distance (0166), leftover coverage (0168),
leftover-map axis share (0148), leftover observed Y / expected E
(0177), or analysis-run status same clock (0171).

## Decision

Each leftover pair names `leftover_map_rank`: the number of Gabriel
singular values above the leftover singular floor on the complete-case
residual rectangle. Closest and farthest pairs on one period report share
that rank. A fallback pair that is not placed on a leftover map stores
rank `0`. Migration `0172` is the single source of the column on every
install path; shipped migrations (`0001` / `0012`) are never rewritten.
It adds a nullable column so older leftover rows keep distance and residual
without fabricating a rank.

The pair button shows `rank {n}` when the value is a finite
non-negative integer. Rank `0` next action: leftover map has no leftover
structure after IRT main effects; open this post. Rank `≥ 1` next action:
read leftover map rank after IRT main effects, then open this post. Omit
the rank badge when the value is missing. Do not invent a leftover score.
Do not invent a theta.

## Consequences

`GET /api/reports/{grouping}/{period}` returns `leftover_map_rank`. After
`make seed`, closest and farthest leftover pairs sit above the member
list with leftover-map rank; click opens that post. Hidden posts stay
hidden.

## Related

Independent of leftover interaction-map persistence, leftover-criterion
evaluation landing, leftover residual UI extraction, leftover-map
complete-case coverage, leftover-map axis share, leftover pairs on the
grouping comparison strip, two-axis leftover-map distance, and leftover
observed `Y` / expected `E`.

## References

Gabriel, K. R. (1971). The biplot graphic display of matrices with
application to principal component analysis. *Biometrika, 58*(3),
453–467. https://doi.org/10.1093/biomet/58.3.453

Jeon, M., Jin, I. H., Schweinberger, M., & Baugh, S. (2021). Mapping
unobserved item–respondent interactions: A latent space item response
model with interaction map. *Psychometrika, 86*(2), 378–403.
https://doi.org/10.1007/s11336-021-09762-5
76 changes: 76 additions & 0 deletions docs/adr/0177-leftover-observed-expected.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# ADR 0177 — Persist observed Y and expected E on leftover pairs

**Decision status:** Accepted
**Date:** 2026-08-24

Amends [ADR 0048](0048-persist-lsirm-leftover-pairs.md) and
[ADR 0049](0049-leftover-pair-report-ui.md).

## Context

ADR 0048 already persists leftover-map distance and leftover residual
`R = Y − E[Y|θ, item]` on `report_leftover_pair`. ADR 0049 already
renders closest and farthest pairs above the member list and opens the
named post. Residual disclosure without naming `Y` and `E` leaves a
reader unable to tell whether a leftover cell is a high observed
response or a low expected category after IRT main effects.

Jeon et al. (2021, eq. 3) leftover interaction is `−γ‖ξ_p − ζ_i‖`.
Gabriel (1971) supplies the leftover-map coordinates from a residual
biplot of `R`. `R` is not an invented leftover score: it is the
observed category minus the already-fitted expected category. Those
two inputs must travel with the pair row.

This increment does not persist leftover-map coordinates, does not
change leftover-map axis count, does not name complete-case coverage,
and does not land Post quality on the leftover criterion.

The unprotected-stack ADR for the same leftover-pair fact was 0163. This
protected-main reconstruction uses **0177** so it does not collide
with any current open-head ADR.

## Decision

Each leftover pair names:

1. `observed_response` — the observed category `Y` for that
post–criterion cell;
2. `expected_response` — `E[Y|θ, item]` from the already-fitted
GRM/GPCM main effects;
3. `leftover_residual`, which must equal `Y − E` within `1e-6`.

Migration `0177` is the single source of both columns on every
install path, fresh or existing -- shipped migrations (`0001`/`0012`)
are never edited after the fact. It adds them as nullable so older
leftover rows keep distance and residual without fabricating `Y` or
`E`. The pair button shows
`Y {observed} · E {expected}` next to leftover-map distance `d` when
both values are finite. The next action remains ADR 0049:
`Open {post}, then read Post quality criterion {criterion}.` Omit the
`Y` / `E` badge when either value is missing or non-finite. Do not
invent a leftover score. Do not invent a theta.

## Consequences

`GET /api/reports/{grouping}/{period}` returns `observed_response`
and `expected_response`. After `make seed`, closest and farthest
leftover pairs sit above the member list with named `Y` and `E`;
click opens that post. Hidden posts stay hidden.

## Related

Independent of leftover interaction-map persistence, leftover-criterion
evaluation landing, leftover residual UI extraction, leftover-map
complete-case coverage, leftover-map axis share, leftover pairs on the
grouping comparison strip, and two-axis leftover-map distance.

## References

Gabriel, K. R. (1971). The biplot graphic display of matrices with
application to principal component analysis. *Biometrika, 58*(3),
453–467. https://doi.org/10.1093/biomet/58.3.453

Jeon, M., Jin, I. H., Schweinberger, M., & Baugh, S. (2021). Mapping
unobserved item–respondent interactions: A latent space item response
model with interaction map. *Psychometrika, 86*(2), 378–403.
https://doi.org/10.1007/s11336-021-09762-5
Loading
Loading