Skip to content

fix(claude-admin): send day-aligned exclusive ending_at for usage/cost streams - #1926

Merged
cyberdima merged 8 commits into
mainfrom
fix/1901-claude-admin-cost-report-date-range
Jul 28, 2026
Merged

fix(claude-admin): send day-aligned exclusive ending_at for usage/cost streams#1926
cyberdima merged 8 commits into
mainfrom
fix/1901-claude-admin-cost-report-date-range

Conversation

@cyberdima

@cyberdima cyberdima commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Summary

Fix claude-admin connector's messages_usage and cost_report streams to emit day-aligned ending_at timestamps.

The Anthropic cost API (/v1/organizations/cost_report) with bucket_width=1d requires day-aligned (midnight) ending_at values. Current implementation uses cursor_granularity: PT1S which produces non-aligned endings (e.g., 23:59:59Z), causing 400 errors on the cost_report endpoint.

Root cause: DatetimeBasedCursor computes ending_at = partition_start + step - cursor_granularity. With step:P1D and cursor_granularity:PT1S, the ending lands at 23:59:59Z instead of the next midnight.

Fix: Set cursor_granularity: P0D for both messages_usage and cost_report streams. This ensures ending_at = start + P1D = next midnight, producing day-aligned exclusive boundaries (00:00 → next 00:00) as required by the cost API.

Verified behavior: Live sync probe confirms cost_report now returns valid day buckets with proper alignment.

Changes

  • src/ingestion/connectors/ai/claude-admin/connector.yaml: Update cursor_granularity for messages_usage and cost_report streams to P0D

Refs #1901

Summary by CodeRabbit

  • Bug Fixes

    • Improved daily date-window handling for Claude Admin usage and cost reporting.
    • Prevented invalid date ranges and HTTP 400 errors in cost report retrieval.
    • Ensured daily windows align with UTC midnight without gaps or overlaps.
  • Documentation

    • Updated operational guidance and architecture records for date-boundary behavior.
  • Tests

    • Added coverage for incremental and full refreshes, empty responses, schema validation, and record metadata.

…t streams (#1901)

The cost_report stream returned HTTP 400 "Invalid date range: ending date
must be after starting date" on every daily slice, ingesting 0 rows.

The Admin usage/cost API treats ending_at as EXCLUSIVE ("buckets that end
before this timestamp") and snaps starting_at to the UTC day start. The CDK
computes each slice end as (start + step) - cursor_granularity; with
step=P1D, cursor_granularity=PT1S that yielded 00:00:00Z -> 23:59:59Z, which
is not before the bucket's midnight end, so zero buckets qualified.

Set cursor_granularity=P0D on claude_admin_messages_usage and
claude_admin_cost_report so each slice sends ending_at = starting_at + P1D
(the next UTC midnight), selecting exactly one daily bucket with no gap and
no double-count (verified against CDK 6.60.16 _partition_daterange).
claude_admin_code_usage is unchanged (it sends only starting_at).

Also correct the manifest version header 7.0.4 -> 6.60.9 to match the CDK
line the mock-test harness and every other nocode manifest use; the manifest
uses no 7.x-only feature and the deployed SDM 7.23.6 accepts it. This keeps
the change isolated to claude-admin (no shared-harness CDK bump, which would
regress zoom's substream-parent slicing tests).

Adds the L1 mock-server suite (asserts each slice's ending_at is the next
midnight, never 23:59:59Z) and registers claude-admin in the connector
mock-test CI component. Documents the boundary in ADR-0002 (supersedes
ADR-0001) and the connector README.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Signed-off-by: Dmitry Saukh <38005371+cyberdima@users.noreply.github.com>
@cyberdima
cyberdima requested a review from a team as a code owner July 25, 2026 23:58
@coderabbitai

coderabbitai Bot commented Jul 25, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@cyberantonz, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 32 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 0026cc4e-f8b7-4d85-8fc2-dd286a901065

📥 Commits

Reviewing files that changed from the base of the PR and between 049a356 and 450d4eb.

📒 Files selected for processing (1)
  • src/ingestion/connectors/ai/claude-admin/connector.yaml
📝 Walkthrough

Walkthrough

The Claude Admin connector now uses P0D cursor granularity for messages and cost streams, aligning exclusive ending_at values to the next UTC midnight. ADRs, documentation, test configuration, fixtures, and stream tests were added or updated.

Changes

Claude Admin cursor boundary fix

Layer / File(s) Summary
Boundary decision and connector configuration
docs/components/connectors/ai/claude-admin/specs/ADR/*, src/ingestion/connectors/ai/claude-admin/connector.yaml, src/ingestion/connectors/ai/claude-admin/README.md
ADR-0002 records the exclusive boundary behavior and applies P0D to messages and cost streams; connector configuration and operational documentation reflect the change.
Test harness and response fixtures
src/ingestion/connectors/ai/claude-admin/tests/config.py, src/ingestion/connectors/ai/claude-admin/tests/fixtures/*
Test configuration and messages/cost response fixtures are added for Claude Admin stream tests.
Cost report window validation
src/ingestion/connectors/ai/claude-admin/tests/test_claude_admin_cost_report.py
Mocked daily requests verify next-midnight boundaries, incremental and full-refresh behavior, metadata stamping, schema conformance, and empty responses.
Messages usage window validation
src/ingestion/connectors/ai/claude-admin/tests/test_claude_admin_messages_usage.py
Mocked daily requests verify next-midnight boundaries, incremental and full-refresh behavior, metadata and derived fields, schema conformance, and empty responses.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant DatetimeBasedCursor
  participant ClaudeAdminConnector
  participant ClaudeAdminAPI
  DatetimeBasedCursor->>ClaudeAdminConnector: Compute daily P0D window
  ClaudeAdminConnector->>ClaudeAdminAPI: Request usage or cost data through next UTC midnight
  ClaudeAdminAPI-->>ClaudeAdminConnector: Return daily bucket results
Loading

Possibly related issues

  • Issue 1901 — The connector change directly addresses the cost report invalid date range caused by PT1S cursor boundaries.

Suggested reviewers: mitasovr

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 10.53% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: day-aligned exclusive ending_at handling for the Claude Admin usage and cost streams.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/1901-claude-admin-cost-report-date-range

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@cyberdima cyberdima self-assigned this Jul 25, 2026
@cyberdima cyberdima changed the title fix(claude-admin): send day-aligned exclusive ending_at for usage/cost streams (#1901) fix(claude-admin): send day-aligned exclusive ending_at for usage/cost streams Jul 25, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@docs/components/connectors/ai/claude-admin/specs/ADR/0001-cursor-granularity-boundary-fix.md`:
- Around line 11-16: Update the historical statement around “all three streams
include end_time_option” in ADR-0001 to clarify that it applied only to the
older configuration, or revise it to match the supersession note. Ensure the
document no longer contradicts the current behavior that claude_admin_code_usage
does not send ending_at.

In
`@docs/components/connectors/ai/claude-admin/specs/ADR/0002-cost-report-exclusive-ending-boundary.md`:
- Line 3: Update the ADR metadata date at the document header to the actual
acceptance date, July 25, 2026, instead of the future-dated July 26, 2026 value.

In `@src/ingestion/connectors/ai/claude-admin/connector.yaml`:
- Line 1: Align the connector manifest version in connector.yaml with the
verified airbyte_cdk version referenced by the connector’s ADR and tests
package: update the version from 6.60.9 to the supported 6.60.16 range, or
consistently revise the ADR and test dependency to document 6.60.9 instead.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 0f7c493f-22ee-462d-8147-57386b67acf2

📥 Commits

Reviewing files that changed from the base of the PR and between a3e865f and 5f83d3c.

📒 Files selected for processing (11)
  • docs/components/connectors/ai/claude-admin/specs/ADR/0001-cursor-granularity-boundary-fix.md
  • docs/components/connectors/ai/claude-admin/specs/ADR/0002-cost-report-exclusive-ending-boundary.md
  • scripts/ci/components.py
  • src/ingestion/connectors/ai/claude-admin/README.md
  • src/ingestion/connectors/ai/claude-admin/connector.yaml
  • src/ingestion/connectors/ai/claude-admin/tests/config.py
  • src/ingestion/connectors/ai/claude-admin/tests/conftest.py
  • src/ingestion/connectors/ai/claude-admin/tests/fixtures/cost_result.json
  • src/ingestion/connectors/ai/claude-admin/tests/fixtures/messages_result.json
  • src/ingestion/connectors/ai/claude-admin/tests/test_claude_admin_cost_report.py
  • src/ingestion/connectors/ai/claude-admin/tests/test_claude_admin_messages_usage.py

Comment thread src/ingestion/connectors/ai/claude-admin/connector.yaml Outdated
- ADR-0001: clarify end_time_option statement is historical (code_usage
  no longer sends ending_at per ADR-0002)
- ADR-0002: fix future-dated acceptance date (2026-07-26 -> 2026-07-25)
- connector.yaml: align CDK pin with verified version (6.60.9 -> 6.60.16)
- descriptor.yaml: bump version so reconcile republishes the manifest
  (ADR-0015 gate compares descriptor version, unchanged = noop)

Signed-off-by: Dmitry Saukh <38005371+cyberdima@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Regenerate the connectors-ddl snapshot

This PR changes src/ingestion/**. If your change affects any
bronze / silver / gold schema, regenerate the committed DDL snapshot
and include it in this PR:

cd src/ingestion/scripts/bootstrap-db
set -a; source pins.env; source .env; set +a
./bootstrap-db.sh connectors-config.yaml   # fresh ClickHouse 25.7.5
./dump-ddl.sh                              # writes scripts/connectors-ddl/*.sql

Commit the resulting scripts/connectors-ddl/*.sql diff. If nothing
changed, no snapshot update is needed. (Regeneration is manual for now.)

Populate missing docstrings in config.py and test modules
(test_claude_admin_cost_report.py, test_claude_admin_messages_usage.py)
to satisfy the 80% coverage threshold.

Coverage was 10.53% due to new helper functions and test cases
lacking docstrings. Added one-line docstrings to all test helpers
(_day_params, _bucket, _next, _register_all) and test functions.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

Signed-off-by: Dmitry Saukh <38005371+cyberdima@users.noreply.github.com>
@cyberdima
cyberdima enabled auto-merge July 27, 2026 08:43
@cyberdima
cyberdima requested review from ktursunov and mitasovr July 27, 2026 08:53
@cyberdima
cyberdima force-pushed the fix/1901-claude-admin-cost-report-date-range branch from c4cb69a to d43e43a Compare July 27, 2026 09:29
@cyberdima
cyberdima requested a review from a team July 27, 2026 15:11
Comment thread src/ingestion/connectors/ai/claude-admin/tests/conftest.py
@cyberdima
cyberdima merged commit f7ba4ac into main Jul 28, 2026
41 of 42 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[High] claude-admin cost_report: invalid date range 400 on every daily slice (0 rows ingested)

3 participants