Repository navigation
feat(usage): show per-thread and per-project API spend #26474
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -39,6 +39,7 @@ import * as Orchestrator from "./Orchestrator.ts"; | |
| import * as ProjectionStore from "./ProjectionStore.ts"; | ||
| import * as ProviderAdapterRegistry from "./ProviderAdapterRegistry.ts"; | ||
| import * as ProviderReplayHarness from "./testkit/ProviderReplayHarness.ts"; | ||
| import { sessionRefsForProviderThreads } from "../usage/usageSessionSpendInput.ts"; | ||
| import type * as ProviderAdapter from "@t3tools/provider-core/server/ProviderAdapter"; | ||
|
|
||
| const instanceId = ProviderInstanceId.make("codex"); | ||
|
|
@@ -596,6 +597,63 @@ it.effect("implements a proposed plan that the command projection leaves out", ( | |
| }).pipe(Effect.provide(layerTest)), | ||
| ); | ||
|
|
||
| it.effect("resolves a thread's usage sessions from its thread id", () => | ||
| Effect.gen(function* () { | ||
| const orchestrator = yield* Orchestrator.OrchestratorV2; | ||
| const projections = yield* ProjectionStore.ProjectionStoreV2; | ||
| const threadId = ThreadId.make("thread:spend-refs"); | ||
| const now = yield* DateTime.now; | ||
| yield* orchestrator.dispatch({ | ||
| type: "thread.create", | ||
| commandId: CommandId.make("create-spend-refs"), | ||
| threadId, | ||
| projectId: ProjectId.make("project:spend-refs"), | ||
| title: "Spend refs", | ||
| modelSelection, | ||
| runtimeMode: "full-access", | ||
| interactionMode: "default", | ||
| branch: null, | ||
| worktreePath: null, | ||
| createdBy: "user", | ||
| creationSource: "web", | ||
| }); | ||
| yield* projections.apply({ | ||
| id: EventId.make("spend-refs:provider-thread"), | ||
| type: "provider-thread.updated", | ||
| threadId, | ||
| occurredAt: now, | ||
| payload: { | ||
| id: ProviderThreadId.make("provider-thread:spend-refs"), | ||
| driver: adapter.driver, | ||
| providerInstanceId: instanceId, | ||
| providerSessionId: null, | ||
| appThreadId: threadId, | ||
| ownerNodeId: null, | ||
| nativeThreadRef: { | ||
| driver: adapter.driver, | ||
| nativeId: "session:spend-refs", | ||
| strength: "strong", | ||
| }, | ||
| nativeConversationHeadRef: null, | ||
| status: "idle", | ||
| firstRunOrdinal: 1, | ||
| lastRunOrdinal: 1, | ||
| handoffIds: [], | ||
| forkedFrom: null, | ||
| createdAt: now, | ||
| updatedAt: now, | ||
| }, | ||
| }); | ||
|
|
||
| // The spend RPC reads threads by id; a read that returned no provider threads here would | ||
| // price nothing while still reporting success. | ||
| const { providerThreads } = yield* projections.getThreadRecords(threadId, ["providerThreads"]); | ||
| assert.deepStrictEqual(sessionRefsForProviderThreads(providerThreads), [ | ||
|
Comment on lines
+650
to
+651
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
| { provider: "codex", sessionId: "session:spend-refs" }, | ||
| ]); | ||
| }).pipe(Effect.provide(layerTest)), | ||
| ); | ||
|
|
||
| it.effect("settles only the stopped run's background work, once", () => | ||
| Effect.gen(function* () { | ||
| const orchestrator = yield* Orchestrator.OrchestratorV2; | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Require
AuthOrchestrationReadScopewhenthreadIdsis present.serverGetSessionSpendneeds onlyAuthDiagnosticsReadScope. The handler inapps/server/src/ws.tsusesinput.threadIdsto read thread snapshots. It returns the native providersessionIdof each matched thread. Other thread reads in this map requireAuthOrchestrationReadScope.A session with
diagnostics:readbut withoutorchestration:readcan send arbitrary thread IDs. It can then confirm that threads exist and read their provider session IDs. This bypasses the scope boundary.Add a payload-dependent rule to
requiredScopesForRpcCall. This is the same pattern asserverRetryResourceTelemetry. Confirm that a multi-scope result requires all listed scopes.Also gate the thread and project spend UI on the same scope.
ProjectSpendLabelcurrently checks onlyAuthDiagnosticsReadScope.