fix: migrate catalog ETL to Cloudflare Workflows (audit remediation, phase 1)#2462

Merged

andrew-bierman merged 27 commits into

mainfrom

fix/etl-pipeline-workflows-migration

May 21, 2026

Commits on May 20, 2026

docs(etl): pivot remediation plan from Queues+outbox to Workflows
andrew-bierman
committed
feat(etl): U1 Workflows spike (throwaway POC)
andrew-bierman
committed
feat(etl): U1 standalone spike worker — Workflows verified GO
andrew-bierman
committed
feat(etl): U2 schema migration 0048 — Workflows-aware columns
andrew-bierman
committed
feat(etl): row-boundary-aligned R2 chunker (chunkCsvForR2)
andrew-bierman
committed
feat(etl): CatalogEtlWorkflow — durable ETL via Cloudflare Workflows
andrew-bierman
committed
feat(etl): producer cutover — default to Workflow, retain queue fallback
andrew-bierman
committed
feat(etl): U4 validator hardening — close SSRF, IDN, length, charset gaps
andrew-bierman
committed
chore(etl): remove standalone spike worker; wire ETL_WORKFLOW into env.dev
andrew-bierman
committed
fix(etl): drop `as Error` casts in CatalogEtlWorkflow
andrew-bierman
committed
refactor(etl): slim U2 to workflow_instance_id + total_embedding_failures
andrew-bierman
committed
feat(etl): U7 invalid_item_logs retention sweep
andrew-bierman
committed
feat(etl): U6 part 1 — structured logger + error propagation fixes
andrew-bierman
committed
feat(etl): U5 (minimal) — workflow-aware retry + reconcile admin endpoints
andrew-bierman
committed
docs(etl): U8 operator runbook for the Workflows pipeline
andrew-bierman
committed
fix(etl): CI failures — type errors, coverage threshold, hoisted mock
andrew-bierman
committed
feat(etl): migration 0050 — ETag fail-closed repair + supersession audit trail
andrew-bierman
committed
feat(etl): U6 part 2 — @sentry/cloudflare wiring
andrew-bierman
committed
fix(etl): logger uses @packrat/guards type predicates instead of raw typeof
andrew-bierman
committed
fix(etl): drop isBoolean import — @packrat/guards doesn't export it
andrew-bierman
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: migrate catalog ETL to Cloudflare Workflows (audit remediation, phase 1)#2462

fix: migrate catalog ETL to Cloudflare Workflows (audit remediation, phase 1)#2462
andrew-bierman merged 27 commits into
mainfrom
fix/etl-pipeline-workflows-migration

Commits on May 20, 2026

docs(etl): pivot remediation plan from Queues+outbox to Workflows

feat(etl): U1 Workflows spike (throwaway POC)

feat(etl): U1 standalone spike worker — Workflows verified GO

feat(etl): U2 schema migration 0048 — Workflows-aware columns

feat(etl): row-boundary-aligned R2 chunker (chunkCsvForR2)

feat(etl): CatalogEtlWorkflow — durable ETL via Cloudflare Workflows

feat(etl): producer cutover — default to Workflow, retain queue fallback

feat(etl): U4 validator hardening — close SSRF, IDN, length, charset gaps

chore(etl): remove standalone spike worker; wire ETL_WORKFLOW into env.dev

fix(etl): drop `as Error` casts in CatalogEtlWorkflow

refactor(etl): slim U2 to workflow_instance_id + total_embedding_failures

feat(etl): U7 invalid_item_logs retention sweep

feat(etl): U6 part 1 — structured logger + error propagation fixes

feat(etl): U5 (minimal) — workflow-aware retry + reconcile admin endpoints

docs(etl): U8 operator runbook for the Workflows pipeline

fix(etl): CI failures — type errors, coverage threshold, hoisted mock

feat(etl): migration 0050 — ETag fail-closed repair + supersession audit trail

feat(etl): U6 part 2 — @sentry/cloudflare wiring

fix(etl): logger uses @packrat/guards type predicates instead of raw typeof

fix(etl): drop isBoolean import — @packrat/guards doesn't export it

Commits on May 21, 2026

feat(etl): GET /api/admin/analytics/catalog/audit endpoint

refactor(db): drizzle.config schema path uses in-package re-export

chore(etl): consolidate ETL migrations to single drizzle-kit-generated migration

fix: address P0/P1 review findings on ETL workflow

docs: fix plan doc contradiction and stale runbook section

🐛 fix: chunk boundary byte offset and retention returning type

🐛 fix(etl): address PR review feedback — chunker guards + docs