Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 32 additions & 3 deletions docs/components/connectors/ai/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,9 @@ This feeds the **planned** `class_ai_org_usage` (see below).
| `class_ai_api_usage` (existing) | ❌ N/A | ✅ feeds | ✅ feeds | ❌ tagged as `openai`, not silver-class | ❌ N/A (flat-seat) | ❌ no API | ❌ N/A | ❌ N/A | ❌ N/A |
| `class_ai_assistant_usage` (proposed, NEW) | ❌ N/A | ❌ N/A | ✅ feeds (chat/cowork/office/web_search) | ❌ N/A | ✅ feeds (planned — Compliance Logs `conversations` events) | ❌ no API | ❌ N/A | 🟡 chat surface | ❌ N/A |
| `class_ai_cost` (proposed, NEW) | ❌ | ✅ feeds | ❌ | ✅ feeds | ❌ (flat-seat, not exposed) | ❌ | ❌ | ✅ `promptCreditsUsed` | ❌ |
| `class_ai_overage` (NEW — Claude Team BUILT †) | ❌ | ❌ | ❌ | 🟡 `credits_used` vs quota | ❌ | 🟡 `credits_used` vs quota | ❌ | 🟡 `promptCreditsUsed` vs plan | ❌ |

† **Claude Team** is the first live source for `class_ai_overage` but is **not a column above** (this matrix predates the claude-team connector). Source `claude_team__ai_overage` reads `/overage_spend_limits` via the customer proxy: per-seat `used_credits` vs `monthly_credit_limit` (already cents, USD) → `overage_cents = max(0, used − limit)`. Verified live (149 seats, Owner role required). See the dedicated section below.

## Known gaps (todo backlog)

Expand All @@ -193,15 +196,16 @@ This feeds the **planned** `class_ai_org_usage` (see below).

Goal: each class is one **product domain** with a homogeneous schema. No NULL-padded columns from sources that "don't expose this kind of data". When a provider grows (Anthropic adds Compliance API, OpenAI splits ChatGPT/Codex, etc.) — rows get added to existing classes or a new class appears, but **existing class schemas don't get smeared**.

## Proposed structure (5 core + 2 optional classes)
## Proposed structure (6 core + 2 optional classes)

```
```text
silver/
├── ai/
│ ├── class_ai_dev_usage (per-person-day code activity)
│ ├── class_ai_assistant_usage (per-person-day chat/cowork/office/web)
│ ├── class_ai_api_usage (per-key-or-project-day tokens + multi-modal API)
│ ├── class_ai_cost (per-line-item-day financial)
│ ├── class_ai_cost (per-line-item-day financial — org/project total)
│ ├── class_ai_overage (per-person-month spend-over-limit — BUILT: Claude Team)
│ └── class_ai_audit_log (event-stream — admin actions, compliance)
└── _shared/
├── class_people (existing) — identity unification
Expand Down Expand Up @@ -319,6 +323,31 @@ provider, source, data_source, collected_at
| Claude Enterprise | ❌ (no cost data) |
| OpenAI | `openai__ai_cost` (existing `to_ai_cost.sql` — re-tag `silver:class_ai_cost`) |

### 4a. `class_ai_overage` (NEW — sibling of `class_ai_cost`; **BUILT for Claude Team**)

**Grain:** `(insight_tenant_id, source, account/email, billing month)` — one row per person × month.

**Why separate from `class_ai_cost`:** `class_ai_cost` answers *"how much did the org/project spend"* (financial line items, per-project/day, not attributable to a person). `class_ai_overage` answers *"how much did **each person** spend **above their own monthly limit**"* — a per-seat budget-vs-actual comparison. Overage needs a per-seat **limit** that cost line items don't carry, and is the metric product asked for ("сколько денег человек потратил свыше лимита"). Keeping it separate avoids smearing a per-person limit column across the org-grain cost class.

**Schema (19 positional cols — first contributor `claude_team__ai_overage` defines the contract; vendor extras → `overage_metrics_json`, never new columns):**
```text
insight_tenant_id, source_id, unique_key, email, account_id
period_month, tool, seat_tier, currency
credit_limit_cents, used_amount_cents, overage_cents -- minor units (cents); overage = max(0, used − limit), honest-NULL if no limit
is_over_limit, is_enabled, overage_metrics_json
source, data_source, collected_at, _version
```

**Provider feeding:**

| Provider | Source | Status |
|---|---|---|
| **Claude Team** | `claude_team__ai_overage` — `/overage_spend_limits` per-seat `used_credits` vs `monthly_credit_limit` (already cents, USD) | ✅ **BUILT** (needs proxy sessionKey with `billing:view`/Owner — verified live, 149 seats) |
| OpenAI / ChatGPT | `openai__ai_overage` (FUTURE) — `chat_activity.credits_used` + codex `credits` per user | 🟡 needs per-seat included quota + credit→$ rate (product input); `overage_cents` stays NULL until quota known |
| Windsurf | `windsurf__ai_overage` (FUTURE) — `promptCreditsUsed` per user vs plan | 🟡 possible if plan limit is known |

**Units:** `used_credits` / `monthly_credit_limit` from the Claude web API are **already cents** (USD, `decimal_places=2`) — mapped straight to `*_cents` with **no ×100** (unlike `claude_team__ai_dev_usage.cost_cents`, which ×100 a dollar-string). **Month grain:** the endpoint is a current-period snapshot with no period field → `period_month = toStartOfMonth(_airbyte_extracted_at)`, latest snapshot per (seat, month) kept (month in `unique_key`) so history accrues and the in-flight month stays idempotent.

### 5. `class_ai_audit_log` (NEW)

**Grain:** event-stream, `(event_id, event_at)`
Expand Down
18 changes: 15 additions & 3 deletions docs/components/connectors/ai/claude-team/specs/DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,9 +196,21 @@ allowed via `additionalProperties: true`.

### 4.4 Silver layer

Out of scope for MVP. `descriptor.dbt_select: ''`. When Silver lands
(Phase 6+), models will be tagged `claude-team` and selected with
`tag:claude-team+`.
Out of scope for the **Bronze MVP** — but Silver has since landed
(`descriptor.dbt_select: 'tag:claude-team+'`, not `''`). Models are
tagged `claude-team` and contribute to shared Silver classes:

- `claude_team__ai_dev_usage` → `class_ai_dev_usage` (per-user-per-day
Claude Code usage from `claude_team_code_metrics`; INSIGHT-458).
- `claude_team__ai_overage` → `class_ai_overage` (per-seat-per-month
spend over the monthly credit limit from `claude_team_overage_spend`;
descriptor 1.3.0). `overage_cents = max(0, used_credits −
monthly_credit_limit)`, units already cents (no ×100). Gold surfaces
it as the `cc_overage` AI bullet. See the cross-connector contract in
`docs/components/connectors/ai/README.md` and `src/ingestion/silver/ai/schema.yml`.

`bronze_promoted` (ADR-0002) promotes all populated Bronze streams,
including `claude_team_overage_spend`, to ReplacingMergeTree.

## 5. Operational limitations (MVP)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,9 @@

Bring up the `claude-team` declarative Airbyte source against the
customer-deployed proxy (see PRD §1.2 and DESIGN §1). Land four
streams in `bronze_claude_team.*`. No Silver, no Gold.
streams in `bronze_claude_team.*`. (Bronze MVP scope — Silver/Gold
landed later: `class_ai_dev_usage` per INSIGHT-458 and `class_ai_overage`
/ Gold `cc_overage` per descriptor 1.3.0; see DESIGN §4.4.)

## 2. Deliverables

Expand Down
6 changes: 5 additions & 1 deletion docs/components/connectors/ai/claude-team/specs/PRD.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,11 @@ Proxy source code, Dockerfile, and deployment instructions:

- Proxy implementation. Lives in `secure-enclave`. The Insight side
treats it as an opaque HTTP endpoint.
- Silver / Gold transformations. `dbt_select: ''` in descriptor.
- Silver / Gold transformations were out of scope for the Bronze MVP
but have since landed (`dbt_select: 'tag:claude-team+'`): Silver
`claude_team__ai_dev_usage` → `class_ai_dev_usage` (INSIGHT-458) and
`claude_team__ai_overage` → `class_ai_overage` (descriptor 1.3.0,
Gold bullet `cc_overage`). See DESIGN §4.4.
- Real-time / streaming sync. Daily cron only.
- Multi-org. One connector instance per claude.ai org. To serve
multiple orgs, deploy multiple proxy containers (one per org) and
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -107,8 +107,10 @@ mod tests {
// asserts every metric_key the gold view emits is classified into exactly
// one bucket — so adding a connector key without classifying it fails CI.
// =====================================================================
/// `ai_person_period` sum-branch (counters). Mirrors the multiIf in
/// 20260610000000_ai-person-period-rollup-fix.sql — keep in sync.
/// `ai_person_period` sum-branch (counters). Mirrors the multiIf in the
/// latest period-rollup migration (20260618000000_ai-claude-team-overage-gold.sql,
/// which re-set the view to add `cc_overage`; previously 20260610000000) —
/// keep in sync.
const SUM_KEYS: &[&str] = &[
"chatgpt",
"cc_lines",
Expand All @@ -123,6 +125,7 @@ mod tests {
"cc_offered",
"cc_tool_accept",
"cc_cost",
"cc_overage",
"prs_total",
"prs_with_cc",
"cursor_offered",
Expand All @@ -138,13 +141,17 @@ mod tests {
];

/// The `metric_keys` actually EMITTED into `insight.ai_bullet_rows` by the gold
/// view 20260609000000 (its ARRAY JOIN branches) — these are the only keys
/// that reach `ai_person_period` and must therefore be classified. NB this is
/// the GOLD key set, NOT the `query_ref` ARRAY JOIN: the latter also lists
/// query_ref-computed ratios (`cursor_acceptance`, `cc_tool_acceptance`,
/// `ai_loc_share2`) and the `claude_web` stub, which are never emitted to
/// `ai_bullet_rows` and so never hit the period rollup. prs_* were removed
/// (honest-NULL) so they are absent here too.
/// view (its ARRAY JOIN branches; latest = 20260618000000, branches 1–6) —
/// these are the only keys that reach `ai_person_period` and must therefore be
/// classified. NB this is the GOLD key set, NOT the `query_ref` ARRAY JOIN: the
/// latter also lists query_ref-computed ratios (`cursor_acceptance`,
/// `cc_tool_acceptance`, `ai_loc_share2`) and the `claude_web` stub, which are
/// never emitted to `ai_bullet_rows` and so never hit the period rollup. prs_*
/// were removed (honest-NULL) so they are absent here too.
///
/// ⚠️ This list is hand-maintained (mirrors the gold view's branches). A new
/// gold branch key MUST be added here AND classified in `SUM_KEYS/MAX_KEYS`, or
/// the guard gives false-green (it only checks keys present in this list).
const BULLET_ROWS_KEYS: &[&str] = &[
// branch 1 (all dev tools)
"active_ai_members",
Expand All @@ -170,6 +177,8 @@ mod tests {
// branch 5 (chatgpt chat)
"chatgpt_active",
"chatgpt",
// branch 6 (claude overage)
"cc_overage",
];

/// Every key the gold view emits must be classified into EXACTLY one of
Expand Down Expand Up @@ -199,4 +208,19 @@ mod tests {
assert!(!SUM_KEYS.contains(&"chatgpt_active"));
assert!(!MAX_KEYS.contains(&"codex_lines"));
}

/// `cc_overage` is a per-period spend counter (twin of `cc_cost`) → sum, never
/// max/avg. Avg would divide a monthly snapshot by active-day count (#1286).
#[test]
fn cc_overage_sums_like_cc_cost() {
assert!(SUM_KEYS.contains(&"cc_overage"), "cc_overage must sum");
assert!(
!MAX_KEYS.contains(&"cc_overage"),
"cc_overage is not an active flag"
);
assert!(
BULLET_ROWS_KEYS.contains(&"cc_overage"),
"cc_overage must be listed as an emitted gold key"
);
}
}
Loading
Loading