Skip to content

fix(ingestion): ai_dev_usage placeholder missing cost_cents / PR-attribution columns — E2E red on main since Jun 4 - #1289

Closed
mitasovr wants to merge 1 commit into
constructorfabric:mainfrom
mitasovr:fix/ai-dev-usage-placeholder-drift
Closed

fix(ingestion): ai_dev_usage placeholder missing cost_cents / PR-attribution columns — E2E red on main since Jun 4#1289
mitasovr wants to merge 1 commit into
constructorfabric:mainfrom
mitasovr:fix/ai-dev-usage-placeholder-drift

Conversation

@mitasovr

@mitasovr mitasovr commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Problem

The E2E suite (E2E — Bronze to API) has been failing on main since 2026-06-04 (last 4 runs red). Root cause:

  • Migration 20260601000000_ai-claude-team-metrics.sql creates a gold view selecting c.cost_cents, c.prs_with_cc_count, c.prs_total_count from silver.class_ai_dev_usage.
  • Those columns were added to the dbt model / connector staging projections, but the bootstrap placeholder in scripts/create-bronze-placeholders.sh was not extended.
  • On a fresh cluster (and in the e2e rig, which bootstraps placeholders + migrations from scratch), ClickHouse validates the CREATE VIEW SELECT against the placeholder and aborts:
Code: 47. DB::Exception: Identifier 'c.cost_cents' cannot be resolved from table with name c.

All subsequent migrations are skipped, so the whole session setup collapses and every e2e test errors.

Fix

Add the three columns to the class_ai_dev_usage placeholder as Nullable(UInt32) — matching the connector staging models (claude_team__ai_dev_usage, claude_enterprise__ai_dev_usage, cursor__ai_dev_usage, copilot__ai_dev_usage all emit them as Nullable(UInt32)). The placeholder is dropped and replaced by dbt with the real schema on first run; minimum-viable parity with the migrations is all that's needed.

Verified the migration references nothing else that's missing: it reads only silver.class_ai_dev_usage (all 12 referenced columns now covered) and insight.people.

Note

This is another instance of the "schema updated in one place, forgotten in another" drift class — same family as cross-connector silver drift. A follow-up PR will propose a contract/parity gate for exactly this.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Chores
    • Enhanced internal data schema with three new fields to improve type-checking validation on fresh instances.

…ge placeholder

Migration 20260601000000_ai-claude-team-metrics.sql creates a gold view
selecting c.cost_cents, c.prs_with_cc_count and c.prs_total_count from
silver.class_ai_dev_usage, but the bootstrap placeholder for that table was
not extended when those columns were added to the dbt model. On a fresh
cluster (and in the e2e rig, which applies placeholders + migrations from
scratch) ClickHouse validates the CREATE VIEW SELECT and fails with
`Code: 47 UNKNOWN_IDENTIFIER: 'c.cost_cents'`, aborting all migrations —
this is why the E2E suite on main has been red since 2026-06-04.

Add the three columns as Nullable(UInt32), matching the connector staging
models (claude_team__ai_dev_usage et al.). The placeholder is dropped and
replaced by dbt with the real schema on the first run, so minimum-viable
parity with the migrations is all that is required.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: insight-ci <ci@cyberfabric.local>
@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

This PR updates the bronze placeholder schema script to extend the silver.class_ai_dev_usage table definition with three new nullable UInt32 columns (cost_cents, prs_with_cc_count, prs_total_count) so that gold-view and dbt SQL queries can resolve these fields on fresh clusters.

Changes

Placeholder Schema Extension

Layer / File(s) Summary
Schema extension for class_ai_dev_usage placeholder
src/ingestion/scripts/create-bronze-placeholders.sh
Added three nullable UInt32 columns (cost_cents, prs_with_cc_count, prs_total_count) to the silver.class_ai_dev_usage placeholder table to support type-checking of downstream gold-view and dbt SQL.

🎯 2 (Simple) | ⏱️ ~5 minutes

🐰 A schema grows with three new fields so bright,
cost_cents and PRs now type-check just right,
On fresh clusters they'll stand, no errors in sight,
Small additions that make the whole system more tight!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and specifically identifies the main fix: adding missing columns to the ai_dev_usage placeholder. It references the exact columns (cost_cents, PR-attribution) and the critical impact (E2E failures since Jun 4), making it a precise summary of the changeset.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@mitasovr

Copy link
Copy Markdown
Contributor Author

Folding this into #1287 per review — the e2e-fix PR needs it anyway to go green.

@mitasovr mitasovr closed this Jun 11, 2026
@mitasovr
mitasovr deleted the fix/ai-dev-usage-placeholder-drift branch June 11, 2026 10:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant