fix: handle multiple GLiNER labels share the same span - #238
Conversation
Greptile SummaryThis PR changes exact-span entity conflict resolution and preserves detector provenance during occurrence propagation. It also extends W&B benchmark-import identity metadata and guarded SDK configuration behavior.
Confidence Score: 4/5The PR is not yet safe to merge because mixed-source exact-span collisions can still replace detector entities with synthetic augmented entities. The propagation-specific provenance regression is addressed, but resolve_overlaps still compares real detector confidence against synthetic score-1.0 values from apply_augmented_entities, leaving the previously reported mixed-source entity replacement behavior reachable. Files Needing Attention: src/anonymizer/engine/detection/postprocess.py Important Files Changed
Reviews (4): Last reviewed commit: "fix: skip propagation copies for spans a..." | Re-trigger Greptile |
Signed-off-by: asteier2026 <asteier@nvidia.com>
…he same span resolve_overlaps previously used alphabetical label order as a tiebreaker for exact-span matches, causing higher-scoring labels like relationship (0.941) to be dropped in favour of lower-scoring ones like last_name (0.719). Adding -score before label in the sort key ensures the highest-confidence label wins. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: asteier2026 <asteier@nvidia.com>
5bb4fd6 to
06e5aaa
Compare
The score-based tiebreak (added to prefer highest-scoring GLiNER label for same-span conflicts) caused expand_entity_occurrences to emit propagation copies with score=1.0 over original detector spans with lower scores, replacing their source and score metadata in final_entities. Fix: skip creating a propagation copy when the position is already covered by an original entity span, preserving detector provenance. Adds regression test confirming source and score are retained. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
|
Changes made per Greptile suggestions |
Summary
resolve_overlapspreviously used alphabetical label order as the final tiebreaker when two GLiNER detections shared the exact same character span, causing higher-confidence labels to be silently dropped in favour of lower-confidence ones.-item.scoreto the sort key (beforeitem.label) so the highest-scoring label wins on exact-span ties.Example:
"Mum"was tagged as bothrelationship(score 0.941) andlast_name(score 0.719). Before this fix,last_namewon becausel<ralphabetically. After,relationshipcorrectly wins.Test plan
test_detection_postprocess.pyparametrized test covers the new behaviour — verify it passes withmake testrelationshipentities surface correctly🤖 Generated with Claude Code