-
Notifications
You must be signed in to change notification settings - Fork 1
feat: search verified multilingual organization labels #316
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
d557a37
05c8378
e5b3ef3
1d4e657
94c0c98
78434ce
caccb31
fb91c3f
0575de7
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -68,10 +68,12 @@ async def semantic_candidate_post_ids( | |
|
|
||
| Project mentions, responsibility/affiliation evidence, Keyman names, and | ||
| organization/team catalogs use one ``ILIKE`` predicate per indexed column. | ||
| This preserves multilingual substring lookup without wrapping indexed fields | ||
| in an expression that forces a sequential scan. Ontology lookup codes are | ||
| applied only to graph lookup-code columns. The function never returns source | ||
| text and nomination never grants access. | ||
| Search-corroborated raw/canonical organization-name pairs additionally | ||
| nominate direct organization mentions and affiliated people's posts; pending | ||
| or uncorroborated aliases do not. This preserves multilingual substring lookup | ||
| without wrapping indexed fields in an expression that forces a sequential | ||
| scan. Ontology lookup codes are applied only to graph lookup-code columns. | ||
| The function never returns source text and nomination never grants access. | ||
| """ | ||
|
|
||
| if maximum_candidates <= 0: | ||
|
|
@@ -84,6 +86,15 @@ async def semantic_candidate_post_ids( | |
| """ | ||
| with query_terms as ( | ||
| select unnest($1::text[]) as term | ||
| ), verified_organization as ( | ||
| select distinct entity.corporate_entity_id | ||
| from organization_name_resolution resolution | ||
| join corporate_entity entity | ||
| on entity.entity_name = resolution.resolved_organization_name | ||
| join query_terms term | ||
| on resolution.raw_organization_name ilike '%' || term.term || '%' | ||
| or resolution.resolved_organization_name ilike '%' || term.term || '%' | ||
| where resolution.verification_status_code = 'verify_corroborated' | ||
|
Comment on lines
+89
to
+97
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔍 verified_organization joins corporate_entity by non-unique display name
Was this helpful? React with 👍 or 👎 to provide feedback.
Comment on lines
+89
to
+97
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 📝 Info: verified_organization CTE correctly gated to corroborated rows only The Was this helpful? React with 👍 or 👎 to provide feedback. |
||
| ), candidate_post as ( | ||
| select mention.post_id, post.created_at | ||
| from post_project_mention mention | ||
|
|
@@ -126,6 +137,20 @@ async def semantic_candidate_post_ids( | |
| where entity.entity_name ilike '%' || term.term || '%' | ||
| ) | ||
| union all | ||
| select mention.post_id, post.created_at | ||
| from post_organization_mention mention | ||
| join verified_organization organization | ||
| on organization.corporate_entity_id = mention.corporate_entity_id | ||
| join source_post post on post.post_id = mention.post_id | ||
| union all | ||
| select mention.post_id, post.created_at | ||
| from post_person_mention mention | ||
| join person_affiliation affiliation | ||
| on affiliation.person_id = mention.person_id | ||
| join verified_organization organization | ||
| on organization.corporate_entity_id = affiliation.affiliated_corporate_entity_id | ||
| join source_post post on post.post_id = mention.post_id | ||
|
Comment on lines
+140
to
+152
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 📝 Info: Verified-org union branches intentionally omit a query_terms predicate The two new union branches ( Was this helpful? React with 👍 or 👎 to provide feedback. |
||
| union all | ||
| select mention.post_id, post.created_at | ||
| from post_team_mention mention | ||
| join cataloged_team team on team.team_id = mention.team_id | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -18,7 +18,7 @@ for migration in /opt/lineageweave/migrations/*.sql; do | |
| migration_name=${migration##*/} | ||
| case "$migration_name" in | ||
| 0012_*|0013_*|0014_*|0015_*|0016_*|0017_*|0018_*|0019_*|0020_*|0021_*|0022_*|0023_*|0024_*|0025_*|0026_*|0027_*|0028_*|0029_*|0030_*|0031_*|0032_*|0033_*|0034_*|0035_*|0036_*|0037_*|0038_*|0039_*|0040_*|0041_*|0042_*|0043_*|0044_*|0045_*|0046_*|0047_*|0048_*|0049_*|0050_*) ;; | ||
| 0051_*|0052_*|0053_*|0054_*) ;; | ||
| 0051_*|0052_*|0053_*|0054_*|0055_*) ;; | ||
| 0060_*|0100_*|0101_*|0102_*) ;; | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔍 Migration 0103 absent from migrate.sh replay window The case statement was extended to cover Was this helpful? React with 👍 or 👎 to provide feedback. |
||
| *) continue ;; | ||
| esac | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| begin; | ||
|
|
||
| -- ADR 0008: only search-corroborated raw/canonical pairs act as Global Ask | ||
| -- aliases. These column indexes preserve multilingual contains-search without | ||
| -- copying context-scoped labels into a second table. | ||
| create extension if not exists pg_trgm; | ||
|
|
||
| create index if not exists organization_name_resolution_raw_search_idx | ||
| on organization_name_resolution using gin (raw_organization_name gin_trgm_ops); | ||
| create index if not exists organization_name_resolution_resolved_search_idx | ||
| on organization_name_resolution using gin (resolved_organization_name gin_trgm_ops); | ||
|
|
||
| commit; | ||
|
seonghobae marked this conversation as resolved.
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| begin; | ||
|
|
||
| drop index if exists organization_name_resolution_resolved_search_idx; | ||
| drop index if exists organization_name_resolution_raw_search_idx; | ||
|
|
||
| -- pg_trgm is shared with the broader Global Ask search slice. | ||
| commit; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔍 verified_organization relies on exact entity_name string equality
The
verified_organizationCTE joinscorporate_entityto the resolution cache withentity.entity_name = resolution.resolved_organization_name(global_ask_retrieval.py). This exact string equality (case- and whitespace-sensitive) means any divergence between the corroborated canonical name and the catalog'sentity_namesilently yields no verified organization, disabling the feature for that org. This is consistent with the design assumption thatresolve_corporate_entitysubstitutes the canonical name so the catalog row already carries the exact canonical form, but it is worth confirming that assumption holds for all corroborated rows.Was this helpful? React with 👍 or 👎 to provide feedback.