feat(experiments): Add avg tokens column to evaluations table

[shanaiabuggy]

Screen.Recording.2026-07-27.at.3.38.31.PM.mov

Summary by CodeRabbit

• New Features
  • Added token-based rollup metrics to the Evaluations API, including average total (input + output) tokens per test case, and enabled sorting/filtering via tokens.* (for example, tokens.mean).
  • Added an “Avg Tokens” column to Experiment Group views with sortable headers, numeric range filtering, and - when no value is available.
• Documentation
  • Updated OpenAPI specs to include the new tokens rollup fields and query/sort/filter options.
• Tests
  • Expanded rollup and metric-filter tests to validate token metric behavior end-to-end.

[coderabbitai]

📝 Walkthrough

Walkthrough

Adds token rollups from ClickHouse through the evaluations API and Studio, including filtering, sorting, response schemas, OpenAPI documentation, guarded aggregation, and tests.

Changes

Evaluation token metrics

Layer / File(s)	Summary
ClickHouse token rollup computation services/intake/src/nmp/intake/spans/evaluation_rollup_repository.py, services/intake/tests/test_experiment_rollup_repository.py	Token attributes are aggregated with presence-aware null handling, distribution statistics, hydration, and query assertions.
Evaluation API token contracts services/intake/src/nmp/intake/api/v2/experiments/schemas.py, services/intake/src/nmp/intake/api/v2/experiments/endpoints.py, openapi/.../openapi.yaml	Evaluation responses and filters expose tokens, while endpoint documentation describes token sort and filter paths.
Metric validation and response hydration services/intake/src/nmp/intake/api/v2/experiments/endpoints.py	The tokens namespace is accepted for metric paths, extracted for sorting, and hydrated into evaluation responses.
Studio token column web/packages/studio/src/components/dataViews/ExperimentGroupDataView/index.tsx	The data view supports token sorting and filtering and renders the rounded Avg Tokens value.
Token filter validation services/intake/tests/test_experiment_metric_filter.py	Regression coverage validates tokens.mean range predicates against evaluation responses.

Sequence Diagram(s)

sequenceDiagram
  participant Studio
  participant EvaluationsAPI
  participant EvaluationRollupRepository
  participant ClickHouse
  Studio->>EvaluationsAPI: Request evaluations with tokens sort/filter
  EvaluationsAPI->>EvaluationRollupRepository: Load evaluation rollups
  EvaluationRollupRepository->>ClickHouse: Query token aggregates
  ClickHouse-->>EvaluationRollupRepository: Return token distributions
  EvaluationRollupRepository-->>EvaluationsAPI: Return rollup.tokens
  EvaluationsAPI-->>Studio: Return token metrics

Loading

Possibly related PRs

• NVIDIA-NeMo/nemo-platform#612: Introduces the evaluations API and schema plumbing extended here with token rollup fields.
• NVIDIA-NeMo/nemo-platform#714: Updates the same evaluation sort parsing and validation flow extended here for tokens.<stat>.
• NVIDIA-NeMo/nemo-platform#753: Modifies the same evaluation rollup pipeline and hydration logic for additional aggregate metrics.

Suggested reviewers: briannewsom

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 16.67% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title matches the main user-facing change: adding an avg tokens column to the evaluations table.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch sbuggy/ase-668

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

services/intake/src/nmp/intake/api/v2/experiments/endpoints.py (1)

1138-1150: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Handle tokens in _evaluation_sort_value
tokens.<stat> falls through to the evaluators path, so sort=tokens.mean and filter[tokens.*] treat the metric as missing. Add a tokens branch here.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@services/intake/src/nmp/intake/api/v2/experiments/endpoints.py` around lines
1138 - 1150, Update _evaluation_sort_value to explicitly handle tokens.<stat>
paths before the evaluators fallback, using the same metric-stat resolution
behavior as the existing cost_usd and latency_ms branches. Ensure
sort=tokens.mean and filter[tokens.*] resolve the token metric instead of
treating it as missing.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@web/packages/studio/src/components/dataViews/ExperimentGroupDataView/index.tsx`:
- Around line 445-448: Alias the `run_count` property to a camelCase `runCount`
local during destructuring in the row rendering logic, then continue passing
`runCount` to `MeanValueTooltipCell`. Keep the API field name `run_count` only
at the boundary.

---

Outside diff comments:
In `@services/intake/src/nmp/intake/api/v2/experiments/endpoints.py`:
- Around line 1138-1150: Update _evaluation_sort_value to explicitly handle
tokens.<stat> paths before the evaluators fallback, using the same metric-stat
resolution behavior as the existing cost_usd and latency_ms branches. Ensure
sort=tokens.mean and filter[tokens.*] resolve the token metric instead of
treating it as missing.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 7da86dd5-1071-40c4-843b-0001b3529a03

📥 Commits

Reviewing files that changed from the base of the PR and between ee4dfd8 and 5e6a43c.

📒 Files selected for processing (8)

• openapi/ga/individual/platform.openapi.yaml
• openapi/ga/openapi.yaml
• openapi/openapi.yaml
• services/intake/src/nmp/intake/api/v2/experiments/endpoints.py
• services/intake/src/nmp/intake/api/v2/experiments/schemas.py
• services/intake/src/nmp/intake/spans/evaluation_rollup_repository.py
• services/intake/tests/test_experiment_rollup_repository.py
• web/packages/studio/src/components/dataViews/ExperimentGroupDataView/index.tsx

[github-actions]

Suite	Lines Covered	Line Rate	Branch Rate
Unit Tests	27341/35089	77.9%	62.2%
Integration Tests	16048/33801	47.5%	19.9%