fix(server): repair connections.entity_ids schema drift (query_sql)#1157
Conversation
connections.entity_ids + its GIN index exist in the squashed baseline but drifted DBs provisioned from the old pre-squash baseline (e.g. prod owletto) never got them, so query_sql's allowlist projection fails with 'column "entity_ids" does not exist'. Add an idempotent delta migration (no-op on fresh installs, repair on drifted DBs).
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
📝 WalkthroughWalkthroughAdds an idempotent database migration that repairs schema drift by creating the ChangesSchema Repair – Connections Entity IDs
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~3 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 inconclusive)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@db/migrations/20260529120000_connections_entity_ids.sql`:
- Line 16: Replace the blocking index creation with a concurrent one: change the
statement that creates idx_connections_entity_ids on public.connections to use
CREATE INDEX CONCURRENTLY IF NOT EXISTS ... USING gin (entity_ids) so the GIN
build won’t lock the table; also ensure this migration is executed outside a
transactional migration runner (CONCURRENTLY cannot run inside a transaction
block) or mark the migration as non-transactional in your migration tool.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 16297562-bb86-46e2-ad90-e4b03ffbd9de
📒 Files selected for processing (1)
db/migrations/20260529120000_connections_entity_ids.sql
| -- definition exactly so the allowlist stays consistent with the real schema. | ||
| ALTER TABLE public.connections ADD COLUMN IF NOT EXISTS entity_ids bigint[]; | ||
|
|
||
| CREATE INDEX IF NOT EXISTS idx_connections_entity_ids ON public.connections USING gin (entity_ids); |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Check PostgreSQL version to confirm CONCURRENTLY IF NOT EXISTS support (requires 9.5+)
rg -n "FROM postgres:" --type dockerfileRepository: lobu-ai/lobu
Length of output: 91
🌐 Web query:
PostgreSQL CREATE INDEX CONCURRENTLY IF NOT EXISTS minimum version
💡 Result:
Minimum PostgreSQL version: PostgreSQL 9.5. Reasoning (from sources): - PostgreSQL 9.5 release notes state: “Add IF NOT EXISTS clause to … CREATE INDEX …” [1]. - The CREATE INDEX grammar documents both optional CONCURRENTLY and IF NOT EXISTS clauses: “CREATE [ UNIQUE ] INDEX [ CONCURRENTLY ] [ [ IF NOT EXISTS ] name ] …” [2][3]. - A CREATE INDEX IF NOT EXISTS community reference also notes “Postgres 9.5 or newer” and that it “Also works for CREATE INDEX CONCURRENTLY IF NOT EXISTS.” [4]. So the combined form “CREATE INDEX CONCURRENTLY IF NOT EXISTS …” is available starting in PostgreSQL 9.5. [1][2][4]
Citations:
- 1: https://www.postgresql.org/docs/9.5/release.html
- 2: https://www.postgresql.org/docs/14/sql-createindex.html
- 3: https://www.postgresql.org/docs/current/sql-createindex.htm
- 4: https://dba.stackexchange.com/questions/35616/create-index-if-it-does-not-exist
Add CONCURRENTLY to avoid production locks during the GIN index build.
The migration’s CREATE INDEX IF NOT EXISTS ... USING gin (entity_ids); will take a table lock while the index is built on a non-empty public.connections table. CREATE INDEX CONCURRENTLY IF NOT EXISTS is supported by PostgreSQL 9.5+.
🔧 Proposed fix
-CREATE INDEX IF NOT EXISTS idx_connections_entity_ids ON public.connections USING gin (entity_ids);
+CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_connections_entity_ids ON public.connections USING gin (entity_ids);📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| CREATE INDEX IF NOT EXISTS idx_connections_entity_ids ON public.connections USING gin (entity_ids); | |
| CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_connections_entity_ids ON public.connections USING gin (entity_ids); |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@db/migrations/20260529120000_connections_entity_ids.sql` at line 16, Replace
the blocking index creation with a concurrent one: change the statement that
creates idx_connections_entity_ids on public.connections to use CREATE INDEX
CONCURRENTLY IF NOT EXISTS ... USING gin (entity_ids) so the GIN build won’t
lock the table; also ensure this migration is executed outside a transactional
migration runner (CONCURRENTLY cannot run inside a transaction block) or mark
the migration as non-transactional in your migration tool.
…1158) * feat(server): associate connections with entities (union with feeds) Add entity_ids to manage_connections create/connect/update (cross-org validated via assertEntityIdsInOrg, tri-state clear on update); list entity_names + entity_id filter become a UNION of the connection's own entity_ids and its feeds' entity_ids; parse entity_ids to number[] so the API returns a typed array. Bumps owletto pointer for the matching UI. Depends on the connections.entity_ids drift migration (#1157) for prod DBs. * chore: bump owletto pointer to merged #249 (09e361d)
2 participants
Problem
The admin
query_sqltool crashes on theconnectionstable in prod:SQL_ERROR: column "entity_ids" does not exist, even whenentity_idsisn't selected — it builds the projection from theSAFE_COLUMN_DEFSallowlist, which (correctly) listsentity_ids.Root cause
connections.entity_ids bigint[]and its GIN index are defined in the squashed baseline (db/migrations/00000000000000_baseline.sql:716,3522). The prod serving DB (owletto) was provisioned from an older pre-squash baseline plus a migration series no longer present in the repo, soconnections.entity_ids(added only in the squash) was never applied — while sibling tables (events/watchers/feeds) did get it. Classic squashed-baseline drift; the allowlist is correct, the prod table is behind.Fix
Idempotent delta migration adding the column + GIN index
IF NOT EXISTS: a no-op on fresh installs (baseline already has it) and a repair on drifted DBs. No change totable-schema.ts.Validation — red→green on real Postgres matching prod's drifted shape
connectionsshape (noentity_ids), ran the exact allowlist projectionquery_sqlbuilds → reproducedcolumn "entity_ids" does not exist(RED).SELECT slug, status FROM connections ORDER BY updated_at DESCreturned rows (GREEN). Idempotency confirmed (re-run = skip). Scratch DB dropped.tsc --noEmit(packages/server): clean.Follow-ups (not in this PR)
packages/server/src/utils/__tests__/table-schema.test.ts) only checks DB→allowlist, not allowlist→DB, and runs against the current baseline — so it could not have caught this. Add the reverse-direction assertion.connections.entity_idsis provisioned but not yet wired into the product (connection↔entity association). Tracked as a separate feature.Need help on this PR? Tag
@codesmithwith what you need. Autofix is disabled.Summary by CodeRabbit