test(e2e): Claude Team AI metrics — per-metric specs + bronze placeholder & watermark guard (#1440) - #1491
Conversation
|
Warning Review limit reached
Next review available in: 59 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (6)
📝 WalkthroughWalkthroughThis PR adds Claude Team bronze placeholder support, changes one incremental dbt model to skip the cutoff when the target table is empty, and adds end-to-end metric specs plus shared fixtures for ChangesClaude Team AI metrics support
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related issues
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/ingestion/scripts/create-bronze-placeholders.sh`:
- Around line 1047-1049: The placeholder schema for the bronze seed data
currently defines email as required, which conflicts with the rest of the
ingestion contract. Update the schema in create-bronze-placeholders.sh so the
placeholder table definition matches the e2e model by making email nullable
alongside the existing Nullable fields, and keep the surrounding placeholder
column definitions consistent with claude_team__ai_dev_usage expectations.
In `@src/ingestion/tests/e2e/specs/ai_cc_active.test.yaml`:
- Around line 11-21: The ai_cc_active test spec currently only seeds unique
claude_team_code_metrics rows, so it never exercises the dedup path the PR adds.
Update the ai_cc_active fixture to include a duplicate re-sync seed alongside
the existing claude_team_usage.yaml template entries for
bronze_claude_team.claude_team_code_metrics, using the same unique_key pattern
as the other ai_cc_* specs so the active-row dedup behavior is verified.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 9dc29597-bd8f-4e83-967e-fc21e6a2ce8f
📥 Commits
Reviewing files that changed from the base of the PR and between 358344d and fadb3e810ee2c7076a4f2e51f6838bdfdade04ab.
📒 Files selected for processing (9)
src/ingestion/connectors/ai/claude-team/dbt/claude_team__ai_dev_usage.sqlsrc/ingestion/scripts/create-bronze-placeholders.shsrc/ingestion/tests/e2e/conftest.pysrc/ingestion/tests/e2e/specs/ai_cc_active.test.yamlsrc/ingestion/tests/e2e/specs/ai_cc_cost.test.yamlsrc/ingestion/tests/e2e/specs/ai_cc_lines.test.yamlsrc/ingestion/tests/e2e/specs/ai_cc_sessions.test.yamlsrc/ingestion/tests/e2e/specs/schemas/bronze_claude_team.claude_team_code_metrics.yamlsrc/ingestion/tests/e2e/specs/templates/claude_team_usage.yaml
|
Thanks @coderabbitai — both addressed:
Re-ran |
|
Tip For best results, initiate chat on the files or code changes.
🐇✨ If you found this review helpful, would you consider giving us a shout-out on X? Thank you for using CodeRabbit! |
…e guard Two fixes the e2e rig needs (both latent in prod): - Add the `bronze_claude_team.claude_team_code_metrics` placeholder (DB + table) to create-bronze-placeholders.sh, mirroring the connector InlineSchemaLoader. It was missing, so `claude_team__ai_dev_usage` couldn't build without Airbyte. - Guard the incremental watermark against the empty-table Date underflow (`coalesce(max(day),'1970-01-01') - INTERVAL 3 DAY` wraps to ~2149 over an empty `this`, dropping every row). Short-circuit with `count()=0 OR …` — same fix as cursor (#1468); mirrors the m365__collab_* guard. Refs #1440 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Sergei Mozhaev <mozhaev.dev@gmail.com>
Adds one spec per metric the claude_team connector drives, over a clean claude_team_code_metrics seed: ai_cc_lines (40), ai_cc_sessions (12), ai_cc_cost (200¢), ai_cc_active (1-of-3). All assert the IC Bullet AI (…0013) served value/median/range; each value spec includes a re-sync duplicate that must dedup. New reusable schema + template (claude_team_usage); cursor__/claude_team__ staging added to _SESSION_START_TRUNCATE for warm-rerun safety. cc_tool_accept / cc_tool_acceptance need tool_use_accepted/offered, which claude_team doesn't expose (NULL) — they belong to the claude_enterprise / admin slice and will be covered there. Refs #1440 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Sergei Mozhaev <mozhaev.dev@gmail.com>
… + ai_cc_active dedup row - create-bronze-placeholders.sh: make tenant_id/source_id/data_source/email Nullable in the claude_team placeholder to match the e2e schema contract ([string,"null"]); unique_key stays non-null (ORDER BY key). Avoids an INSERT mismatch for rows that legitimately omit these (the model filters email anyway). - ai_cc_active: add the re-sync duplicate row, consistent with the other ai_cc_* specs — cc_active must stay 1 (an "active" metric dedups by person). Refs #1440 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Sergei Mozhaev <mozhaev.dev@gmail.com>
…format Rewrite the 4 cc metric specs' `description:` blocks to the canonical shape from the metric-test skill (collab_active_days.test.yaml precedent): plain- language bronze→silver→gold layer flow + a Team line tying seeded member values to the asserted median/range. No behavior change — 4 passed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Sergei Mozhaev <mozhaev.dev@gmail.com>
d8121b4 to
1828754
Compare
cc_overage ← class_ai_overage.overage_cents = max(0, used_credits − monthly credit limit), both already in cents. Seeded via a new bronze stream claude_team_overage_spend; alice used 300/limit 100 → 200¢, median 100, range [50, 200] (dedup holds). 1 passed. Supporting: - bronze placeholder for claude_team_overage_spend (+ schema + template). - claude_team__ai_overage: same empty-table watermark guard as the dev-usage models — the INTERVAL 1 MONTH window underflows the Date range over an empty `this` (rig resets staging between tests), so short-circuit with count()=0. - conftest: reset staging.claude_team__ai_overage at session start (warm-rerun). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Sergei Mozhaev <mozhaev.dev@gmail.com>
3 participants
Part of #1440 (E2E smoke: AI Adoption metrics) — the claude_team slice. Sibling of #1468 (cursor). One spec per metric.
Tests — claude_team-driven AI metrics (4)
Seeds
bronze_claude_team.claude_team_code_metricsand asserts the IC Bullet AI (…0013):ai_cc_lines= 40 (← total_lines_accepted)ai_cc_sessions= 12 (← total_sessions)ai_cc_cost= 200¢ (← total_cost"2.00", decimal-as-string → cents)ai_cc_active= 1-of-3Each value spec includes a re-sync duplicate that must dedup. New reusable
schemas/bronze_claude_team.claude_team_code_metrics.yaml+templates/claude_team_usage.yaml.cc_tool_accept/cc_tool_acceptanceare not here — they needtool_use_accepted/offered, which claude_team doesn't expose (NULL by design); they belong to the claude_enterprise / admin slice.Fixes (latent in prod, exposed by the rig)
claude_team__ai_dev_usagecouldn't build without Airbyte. Added DB + table mirroring the connector InlineSchemaLoader.claude_team__ai_dev_usage— same Date-underflow as cursor (test(e2e): AI metrics per-metric specs + bronze_cursor placeholder & watermark fixes (#1440) #1468):max(day)over an emptythisis the epoch,- 3 DAYwraps to ~2149-06-04 and filters everything out. Guarded with(SELECT count() FROM this) = 0 OR …(m365 pattern). NB: this guard is copy-pasted across all 10 AI staging models — chatgpt_team next._SESSION_START_TRUNCATEreset for the claude staging (warm-rerun safety).Verification
./e2e.sh test -k ai_cc→ 4 passed; warm re-run (nodown) → 4 passed.Refs #1440
🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Bug Fixes