fix(clustering): rename only genuine title outliers, not near-ties - #76
Merged
Conversation
The margin-based rule shipped in #75 over-fired. A dry run of the backfill renamed 52 of 200 events (26%), and the samples showed why: most were same-story swaps, e.g. "Două explozii la Damasc, lângă hotelul unde a fost cazat Macron" replaced by "Explozii la Damasc, lângă hotelul în care este cazat Macron". That is churn on a published headline, not a repair. Calibrating against 26 production clusters showed the discriminator is the incumbent's absolute centrality, not its gap to the winner: Zelensky -> Moscow incumbent 0.014 genuine divergence Oana Țoiu -> Nicușor Dan incumbent 0.014 genuine divergence Damen Mangalia shipyard incumbent 0.163 same story Air alert incumbent 0.158 same story Helicopter / cable car incumbent 0.086 same story A title is now treated as the outlier only when it shares essentially nothing with its cluster (< 0.05), the replacement is itself representative (>= 0.05), and it dominates by 3x rather than edging ahead. Both real divergent events still get renamed; the same-story swaps no longer do. Adds the Damascus cluster as a churn regression test. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012MpkuigW6T4N1exXip6bRp
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #75. The margin-based rename rule over-fired and would have churned published headlines. This recalibrates it against production data.
What the dry run showed
Running the backfill in dry-run mode renamed 52 of 200 events (26%) — far above the 4–6% divergence rate actually observed. The samples explained why:
Same story, reworded. Swapping one for the other on a published, indexed headline is churn, not repair.
The actual discriminator
Calibrating against 26 production clusters showed the rule was wrong in kind, not degree. The gap between winner and incumbent doesn't separate the cases — the incumbent's absolute centrality does:
The two genuinely broken events share essentially nothing with their clusters. The false positives are perfectly representative titles that merely lost to a marginally more central phrasing.
Change
A title is renamed only when all three hold:
Both real divergent events still get renamed; the same-story swaps no longer do.
Testing
339 passed, 4 skipped. Adds the Damascus cluster as an explicit churn regression test, alongside the existing Moscow cluster test that must still fire.
🤖 Generated with Claude Code
Generated by Claude Code