fix(lineage): preserve grouping provenance and fail closed weights - #506
fix(lineage): preserve grouping provenance and fail closed weights#506seonghobae wants to merge 3 commits into
Conversation
…0145)
The convex weights reconstruct() fuses its evidence channels with were
hand-picked constants justified only by a code comment. ADR 0145
replaces assertion with estimation: channels are items observing the
latent "these posts are related" trait, scored candidate pairs are
respondents nested in reconstruction groups, and fast-mlsirm's
multilevel 2PL (MLS2PLM, mmle estimator with cluster random intercepts
per Fox & Glas 2001) estimates per-channel discriminations -- the
information-optimal weights per Birnbaum (1968). Fail-closed at every
step: no fast_mlsirm, too small a sample, a degenerate channel, or a
non-finite estimate produces no weights and rebuilds keep the (now
explicitly labeled) fallback constants; persisted weights are used only
on an exact active-channel-set match, never mixed.
Also lands the grouping-evidence backfill this estimation depends on:
the zcrht811 import stuffed per-row-unique identifiers into BOTH
grouping columns (thread_group_key = own record GUID on 43,811/43,839
rows; secondary_grouping_key = own document number), so _group_by saw
~43k singleton groups and no channel ever scored a real candidate pair
-- Event Lineage was silently non-functional corpus-wide.
backfill_thread_group_keys.py clears the placeholder signature rows to
'' (migration 0002's designed "no signal" value; grouping falls back to
process-unit/corporate-entity candidate pools) and routes
source_project_code to secondary_grouping_key -- the secondary-key
channel's documented signal ("e.g. project code") -- as fused evidence,
never a hard partition that would wall related posts apart. Fails
closed when an analysis_scope_thread_group run (TEPP/report scope keys
resolve live against thread_group_key) would be orphaned.
Verified: parameter-recovery test recovers planted discrimination
ordering (strong > mid > weak) via the real fast-mlsirm fit; estimation
script fails closed against the live un-backfilled corpus (105 pairs <
200 minimum, nothing written); a100-fork rebuild regression stays
green.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015HgzNGsCxqiTaT4YuJEb5J
|
Warning Review limit reached
Next review available in: 6 minutes Limit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?Wait for the limit to reset, then comment An organization admin can change what happens after included review limits in Billing. How do review limits work?CodeRabbit enforces per-developer PR review limits within each organization. For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (18)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Superseded by #507. The clean single-commit restack preserves the validated final tree while keeping private source identifiers out of the protected-main route. No force-push was used. |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| case "$migration_name" in | ||
| 0012_*|0013_*|0014_*|0015_*|0016_*|0017_*|0018_*|0019_*|0020_*|0021_*|0022_*|0023_*|0024_*|0025_*|0026_*|0027_*|0028_*|0029_*|0030_*|0031_*|0032_*|0033_*|0034_*|0035_*|0036_*|0037_*|0038_*|0039_*|0040_*|0041_*|0042_*|0043_*|0044_*|0045_*|0046_*|0047_*|0048_*|0049_*|0050_*) ;; | ||
| 0060_*|0100_*|0101_*|0102_*) ;; | ||
| 0060_*|0100_*|0101_*|0102_*|0135_*) ;; |
There was a problem hiding this comment.
🔍 Migration 0103 absent from the replay gate
The replay gate gains 0135_* but 0103_tenant_settings.sql stays excluded. migrate.sh replays migrations on every compose up, so volumes predating 0103 never create tenant_settings, and read_tenant_settings in main.py then 500s on undefined_table. It cannot simply be added: 0103 lacks IF NOT EXISTS and would fail replay where the table already exists. Pre-existing, but this line is edited here.
Was this helpful? React with 👍 or 👎 to provide feedback.
1 participant
Summary
Provenance
The only conflict resolutions were base-specific: register migration 0135 without importing absent unrelated migration families, and adapt the importer insert from 30 to 32 parameters rather than the docs-branch 36-parameter schema.
Verification
No real source records or identifying examples are included.