Skip to content

fix(etl): use raw SQL for completion write to bypass neon-http enum issue #2395

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
andrew-bierman merged 10 commits into from
May 12, 2026
Merged

fix(etl): use raw SQL for completion write to bypass neon-http enum issue #2395

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
d78ac17
fix(etl): use raw SQL for completion write to bypass neon-http enum i…
andrew-bierman May 8, 2026
cb7ef0a
fix(etl): make catalog_items weight nullable + add ETL integration tests
andrew-bierman May 8, 2026
60d5be8
fix(db): regenerate migration with drizzle-kit (0037_rich_electro)
andrew-bierman May 9, 2026
f694d96
fix(etl): handle nullable weight/weightUnit when building pack items …
andrew-bierman May 12, 2026
f8671be
chore: merge main into fix/etl-completion-and-retry — resolve drizzle…
andrew-bierman May 12, 2026
6f84a52
fix(admin,etl): fix CORS Access-Control-Allow-Origin missing on prefl…
andrew-bierman May 12, 2026
a28c16c
fix(types): resolve TypeScript errors blocking CI checks job
andrew-bierman May 12, 2026
fdbc495
fix(types): annotate safe-casts and remove redundant Promise<T> casts
andrew-bierman May 12, 2026
90b4999
fix(expo/auth): fix TS errors blocking CI checks
andrew-bierman May 12, 2026
5676ae5
chore(deps): pin @packrat-ai/nativewindui to 2.0.3-2
andrew-bierman May 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions packages/api/drizzle/0037_nullable_catalog_weight.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
-- catalog_items.weight and weight_unit: drop NOT NULL to allow items without weight data.
-- The validator intentionally skips weight (clothing/footwear often omit it), but the
-- NOT NULL constraint was causing upserts to throw, which cascaded to ETL job failures.
ALTER TABLE "catalog_items" ALTER COLUMN "weight" DROP NOT NULL;--> statement-breakpoint
ALTER TABLE "catalog_items" ALTER COLUMN "weight_unit" DROP NOT NULL;
Comment on lines +1 to +5

@coderabbitai coderabbitai Bot May 9, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Remove the duplicate 0037 migration file to avoid drift/confusion.

This file duplicates the 0037_rich_electro DDL while the journal tracks 0037_rich_electro. Keeping both increases the risk of future divergence during migration maintenance.

🧹 Suggested cleanup 
--- a/packages/api/drizzle/0037_nullable_catalog_weight.sql
+++ /dev/null
@@
--- catalog_items.weight and weight_unit: drop NOT NULL to allow items without weight data.
--- The validator intentionally skips weight (clothing/footwear often omit it), but the
--- NOT NULL constraint was causing upserts to throw, which cascaded to ETL job failures.
-ALTER TABLE "catalog_items" ALTER COLUMN "weight" DROP NOT NULL;--> statement-breakpoint
-ALTER TABLE "catalog_items" ALTER COLUMN "weight_unit" DROP NOT NULL;
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
-- catalog_items.weight and weight_unit: drop NOT NULL to allow items without weight data.
-- The validator intentionally skips weight (clothing/footwear often omit it), but the
-- NOT NULL constraint was causing upserts to throw, which cascaded to ETL job failures.
ALTER TABLE "catalog_items" ALTER COLUMN "weight" DROP NOT NULL;--> statement-breakpoint
ALTER TABLE "catalog_items" ALTER COLUMN "weight_unit" DROP NOT NULL;
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/api/drizzle/0037_nullable_catalog_weight.sql` around lines 1 - 5,
Delete this duplicate migration file that applies ALTER TABLE "catalog_items"
ALTER COLUMN "weight" DROP NOT NULL and ALTER COLUMN "weight_unit" DROP NOT NULL
and keep the canonical 0037_rich_electro migration tracked by the journal;
remove the duplicate 0037 nullable catalog migration from the repository, ensure
the migration journal still references only 0037_rich_electro, and run the
migration tests (or a dry-run) to confirm no drift remains.

2 changes: 2 additions & 0 deletions packages/api/drizzle/0037_rich_electro.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
ALTER TABLE "catalog_items" ALTER COLUMN "weight" DROP NOT NULL;--> statement-breakpoint
ALTER TABLE "catalog_items" ALTER COLUMN "weight_unit" DROP NOT NULL;
Loading
Loading