-
Notifications
You must be signed in to change notification settings - Fork 1
fix: leave tied organization similarity unbound (v0.86.4) #159
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| An R&R organization name that matches two catalog rows at the same | ||
| similarity score stays text. Open the post after the catalog collision | ||
| is unique, then click. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,80 @@ | ||
| # ADR 0021 — Tied organization similarity stays unbound | ||
|
|
||
| **Decision status:** Accepted | ||
| **Date:** 2026-08-16 | ||
|
|
||
| ## Context | ||
|
|
||
| ADR 0019 stores the catalog id on `post_summary_role` so fetch no | ||
| longer rejoins `corporate_entity.entity_name`. Persist still calls | ||
| `resolve_corporate_entity` before that write. That function kept the | ||
| first candidate whose similarity score was strictly greater than the | ||
| running best. Two catalog rows can share a display name (different | ||
| `corporate_entity_code`, different parents). Both score 1.0. The | ||
| winner was then whichever row an unordered `select` happened to | ||
| return. | ||
|
|
||
| The buyer path is: open the post, click the R&R organization name, | ||
| walk related nodes. A first-wins bind walks the homonym. ADR 0019's | ||
| backfill already refuses to guess when two same-named mentions exist | ||
| on one post (`HAVING count(*) = 1`). Write-time resolution did not. | ||
|
|
||
| String similarity is only the candidate-generation stage of | ||
| collective entity resolution (Bhattacharya & Getoor, 2007). Fellegi | ||
| and Sunter (1969) leave an uncertain pair for clerical review rather | ||
| than forcing a link. Christen (2012) treats that hold-out as part of | ||
| the matching decision, not a defect to paper over. | ||
|
|
||
| ```mermaid | ||
| flowchart TD | ||
| mention["R&R organization name"] --> score["Score every catalog row"] | ||
| score --> unique{"One unique top score at or above the threshold?"} | ||
| unique -->|yes| bind["Store that catalog id on the role row"] | ||
| unique -->|no| unbound["Leave the name as text — no related-node button"] | ||
| bind --> walk["Click walks GET /api/corporate-entities/{id}/related"] | ||
| unbound --> later["Resolve the catalog collision, then persist again"] | ||
| ``` | ||
|
|
||
| ## Decision | ||
|
|
||
| `resolve_corporate_entity` returns a catalog id only when exactly one | ||
| candidate holds the top score and that score clears `min_similarity`. | ||
| A tied top score returns `None`. `persist_post_summary` then stores | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Persist does not call resolve. It calls |
||
| no `cataloged_corporate_entity_id` and writes no | ||
| `post_organization_mention`. The popup shows the name as text, not a | ||
| button. | ||
|
|
||
| The same function is shared with Keyman affiliation matching and | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Keyman affiliation also uses |
||
| entity-relationship counterparty resolution. Those paths already | ||
| treat `None` as "do not invent a link." | ||
|
|
||
| This does not create a `corporate_entity` row. Null inference / | ||
| verification clients stay unavailable. Do not invent a TEPP theta. | ||
|
|
||
| ## Consequences | ||
|
|
||
| - Open a post whose R&R names `Tied Energy` when two catalog rows | ||
| share that display name. The name is not a button. Resolve the | ||
| catalog collision (distinct codes, parents, or a verified unique | ||
| match), persist the summary again, then click. | ||
| - A unique exact match among other homonym neighbors still binds. | ||
| - The same catalog id listed twice in the candidate snapshot is one | ||
| winner, not a tie. | ||
|
|
||
| ## References | ||
|
|
||
| Bhattacharya, I., & Getoor, L. (2007). Collective entity resolution | ||
| in relational data. *ACM Transactions on Knowledge Discovery from | ||
| Data, 1*(1), Article 5. https://doi.org/10.1145/1217299.1217304 | ||
|
|
||
| Christen, P. (2012). *Data matching: Concepts and techniques for | ||
| record linkage, entity resolution, and duplicate detection*. | ||
| Springer. https://doi.org/10.1007/978-3-642-31164-2 | ||
|
|
||
| Fellegi, I. P., & Sunter, A. B. (1969). A theory for record linkage. | ||
| *Journal of the American Statistical Association, 64*(328), | ||
| 1183–1210. https://doi.org/10.2307/2286061 | ||
|
|
||
| International Organization for Standardization. (2023). *ISO/IEC | ||
| 11179-1:2023: Information technology—Metadata registries (MDR)—Part 1: | ||
| Framework*. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -55,4 +55,4 @@ | |
| "sentence_excerpts", | ||
| ] | ||
|
|
||
| __version__ = "0.86.2" | ||
| __version__ = "0.86.4" | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -18,6 +18,8 @@ | |
| upgrade path once real usage shows single-mention similarity scoring | ||
| under- or over-resolving in practice -- it is not implemented here because | ||
| nothing yet demonstrates the need for it over this simpler, cheaper stage. | ||
| A tied top score therefore stays unbound (ADR 0021; Fellegi & Sunter, | ||
| 1969) instead of first-winning a homonym. | ||
| """ | ||
|
|
||
| from __future__ import annotations | ||
|
|
@@ -61,26 +63,36 @@ def resolve_corporate_entity( | |
| candidates: Sequence[CorporateEntityCandidate], | ||
| min_similarity: float = DEFAULT_MIN_SIMILARITY, | ||
| ) -> str | None: | ||
| """Returns the best-matching candidate's `corporate_entity_id`, or | ||
| `None` if no candidate clears `min_similarity`. | ||
| """Return the unique best-matching catalog id, or ``None``. | ||
|
|
||
| Returning `None` for a genuine non-match is the point, not a failure | ||
| case to work around: a wrong hierarchy link corrupts every downstream | ||
| Knowledge Graph traversal through it, so "no confident match" must | ||
| stay a real, distinguishable outcome from "matched entity X." | ||
| ``None`` is the honest outcome when no candidate clears | ||
| ``min_similarity`` **or** two or more candidates share the top | ||
| score. A wrong hierarchy link corrupts every downstream Knowledge | ||
| Graph walk, so a tied homonym must not become a button (ADR 0021; | ||
| Fellegi & Sunter, 1969; Bhattacharya & Getoor, 2007). Duplicate | ||
| rows for the same ``corporate_entity_id`` still count as one | ||
| candidate. | ||
| """ | ||
| normalized_mention = normalize_organization_name(mentioned_name) | ||
| if not normalized_mention: | ||
| return None | ||
|
|
||
| best_id: str | None = None | ||
| best_ids: list[str] = [] | ||
| best_score = 0.0 | ||
| for candidate in candidates: | ||
| score = SequenceMatcher( | ||
| None, normalized_mention, normalize_organization_name(candidate.entity_name) | ||
| ).ratio() | ||
| if score > best_score: | ||
| best_score = score | ||
| best_id = candidate.corporate_entity_id | ||
| best_ids = [candidate.corporate_entity_id] | ||
| elif ( | ||
| score == best_score | ||
| and score > 0.0 | ||
| and candidate.corporate_entity_id not in best_ids | ||
| ): | ||
| best_ids.append(candidate.corporate_entity_id) | ||
|
|
||
| return best_id if best_score >= min_similarity else None | ||
| if best_score < min_similarity or len(best_ids) != 1: | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This |
||
| return None | ||
| return best_ids[0] | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -542,3 +542,59 @@ def test_homonym_organization_role_binds_the_resolved_catalog_id( | |
|
|
||
| database_dsn, post_id, _summary_person_id = projection_database.split("|") | ||
| asyncio.run(_exercise_homonym_organization_role_binding(database_dsn, post_id)) | ||
|
|
||
|
|
||
| async def _exercise_tied_organization_similarity_stays_unbound( | ||
| database_dsn: str, | ||
| post_id: str, | ||
| ) -> None: | ||
| """Write-time resolution must not first-win a same-named catalog pair.""" | ||
|
|
||
| connection = await asyncpg.connect(database_dsn) | ||
| try: | ||
| await connection.execute( | ||
| """ | ||
| insert into corporate_entity | ||
| (corporate_entity_code, entity_name, entity_level_code) | ||
| values | ||
| ('HOMONYM-TIED-A', 'Tied Energy', 'company'), | ||
| ('HOMONYM-TIED-B', 'Tied Energy', 'company') | ||
| """ | ||
| ) | ||
| payload = await persist_post_summary( | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This persist uses default Null clients, so it cannot catch create-on-tie. Repeat it with available inference + corroborating verification and assert no |
||
| connection, | ||
| post_id, | ||
| PostSummary( | ||
| korean_summary="동점 조직은 버튼을 만들지 않는다.", | ||
| roles_and_responsibilities=( | ||
| RoleResponsibility( | ||
| actor_name="Tied Energy", | ||
| responsibility="납품 일정 확정", | ||
| actor_type_code=ACTOR_TYPE_ORGANIZATION, | ||
| ), | ||
| ), | ||
| ), | ||
| ) | ||
| roles = payload["roles_and_responsibilities"] | ||
| assert len(roles) == 1 | ||
| assert roles[0]["catalog_node_id"] is None | ||
| assert roles[0]["catalog_node_type_code"] is None | ||
| fetched = await fetch_persisted_summary(connection, post_id) | ||
| assert fetched is not None | ||
| assert fetched["roles_and_responsibilities"][0]["catalog_node_id"] is None | ||
| mention_count = await connection.fetchval( | ||
| "select count(*) from post_organization_mention where post_id = $1", | ||
| post_id, | ||
| ) | ||
| assert mention_count == 0 | ||
| finally: | ||
| await connection.close() | ||
|
|
||
|
|
||
| def test_tied_organization_similarity_stays_unbound( | ||
| projection_database: str, | ||
| ) -> None: | ||
| """ADR 0021: open the post — the shared name is text, not a homonym button.""" | ||
|
|
||
| database_dsn, post_id, _summary_person_id = projection_database.split("|") | ||
| asyncio.run(_exercise_tied_organization_similarity_stays_unbound(database_dsn, post_id)) | ||
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
“On a miss” now includes a tied top score. A tie is an ambiguous existing catalog match, not an unseen name. Creation must not run. Keep ADR 0010 for names that no candidate scores at or above
min_similarity.