refactor(experiments): Rename all child resources of ExperimentGroup entity to "Experiment" - #938
Merged
Conversation
Signed-off-by: shanaiabuggy <59746633+shanaiabuggy@users.noreply.github.com>
…aking) [ASE-550]
Rename the parent resource user-facing from ExperimentGroup to Experiment across
the intake API and RBAC. The stored entity stays experiment_group; the API layer
maps at the boundary. Non-breaking: old URLs and response shapes keep working.
* Routes: /experiment-groups -> /experiments (canonical). The old paths are kept
as hidden deprecated aliases (include_in_schema=False) pointing at the same
handlers, so existing callers keep working until they migrate.
* Schemas: ExperimentGroup{Request,Response,Filter} -> Experiment{...} (identical
JSON shape). Tag "Experiment Groups" -> "Experiments". Entity class untouched.
* RBAC: intake.experiment-groups.* -> intake.experiments.*; the new /experiments
path is authorized and the hidden /experiment-groups aliases are mapped via
x-not-in-openapi (mirrors the #612 pattern). OPA tests pass (236/236).
* Regenerated OpenAPI; updated tests + seed scripts to the canonical path; added
a back-compat test proving the legacy /experiment-groups alias still works.
Studio FE follows in a subsequent commit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: shanaiabuggy <59746633+shanaiabuggy@users.noreply.github.com>
Point Studio at the renamed /experiments API and rename the internal
ExperimentGroup* components, routes, hooks, types, route param, and user-facing
copy to Experiment*. No behavior change; the parent resource is consistently
"Experiment" now.
* SDK refs: ExperimentGroupResponse -> ExperimentResponse, useGetExperimentGroup
-> useGetExperiment, etc. (regenerated from the renamed spec).
* Components/routes/files: ExperimentGroupDataView -> ExperimentDataView,
ExperimentGroupDetailRoute -> ExperimentDetailRoute, {Create,Edit}Modal,
ParetoChart, Metrics, Card, InsightExperiments, useExperimentEvaluations.
* Route param experimentGroupName -> experimentName (URL structure unchanged),
route key experimentGroupDetail -> experimentDetail.
* User-facing copy "Experiment Group(s)" -> "Experiment(s)"; camelCase
experimentGroupId -> experimentId; localStorage keys and the CSS scoping class.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: shanaiabuggy <59746633+shanaiabuggy@users.noreply.github.com>
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughChangesExperiment API and contract migration
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
web/packages/studio/src/components/ExperimentEditModal/index.tsx (1)
69-97: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winFinish the visible terminology migration.
web/packages/studio/src/components/ExperimentEditModal/index.tsx#L69-L97: change the toast, modal title, and instruction to “experiment”.web/packages/studio/src/routes/ExperimentDetailRoute/index.tsx#L23-L26: change the load-error message to “experiment”.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/ExperimentEditModal/index.tsx` around lines 69 - 97, Finish the visible terminology migration by updating the error toast, modal title, and instruction in ExperimentEditModal to use “experiment” terminology instead of “group”; update the load-error message in ExperimentDetailRoute likewise. Apply the requested wording changes at web/packages/studio/src/components/ExperimentEditModal/index.tsx lines 69-97 and web/packages/studio/src/routes/ExperimentDetailRoute/index.tsx lines 23-26.openapi/ga/openapi.yaml (1)
3639-3647: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winStale "ExperimentGroup"/"group" wording in Patch Evaluation docs.
The PR's objective is renaming ExperimentGroup children to Experiment, but this endpoint's
description("curating an evaluation into another ExperimentGroup", "any new group must exist... belongs to >=1 group") and its400response ("A referenced ExperimentGroup does not exist") still use the old terminology, contradicting the rest of this same file (e.g.EvaluationPatchRequestdocstring at Line 10607-10612 already says "experiment").🔧 Suggested fix
description: "Partially update an evaluation: only fields present in the request\ - \ are changed.\n\nThe common case is curating an evaluation into another ExperimentGroup\ - \ \u2014 PATCH with the merged\n``experiment_ids``. Membership is replaced\ - \ (not appended), so send the full desired set; any new\ngroup must exist\ - \ and the set must be non-empty (an evaluation always belongs to >=1 group).\ + \ are changed.\n\nThe common case is curating an evaluation into another Experiment\ + \ \u2014 PATCH with the merged\n``experiment_ids``. Membership is replaced\ + \ (not appended), so send the full desired set; any new\nexperiment must exist\ + \ and the set must be non-empty (an evaluation always belongs to >=1 experiment).\ \ Omitted\nfields are left untouched (unlike the full-body PUT, which overwrites\ \ them)." ... '400': - description: A referenced ExperimentGroup does not exist, or experiment_ids + description: A referenced Experiment does not exist, or experiment_ids is emptyAlso applies to: 3674-3677
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openapi/ga/openapi.yaml` around lines 3639 - 3647, Update the Patch Evaluation documentation in the operation description and its 400 response to consistently use “Experiment” terminology instead of “ExperimentGroup” and “group,” including membership and referenced-resource wording. Preserve the existing PATCH behavior and replace only the stale terminology.
🧹 Nitpick comments (2)
web/packages/studio/src/components/IntakeDetail/README.md (1)
90-90: 📐 Maintainability & Code Quality | 🔵 Trivial | 🏗️ Heavy liftSplit this into Diataxis pages.
Keep architecture here; move route syntax to REFERENCE and “Adding a kind” to a HOW-TO. Add prerequisites and Next Steps.
As per coding guidelines, each documentation page must fit one Diataxis quadrant, list prerequisites at the top, and include a Next Steps section.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/IntakeDetail/README.md` at line 90, Split the Intake detail documentation into Diataxis-aligned pages: keep the architecture overview in the current README, move route syntax into a REFERENCE page, and move “Adding a kind” into a HOW-TO page. Add a prerequisites section at the top and a Next Steps section to every resulting page, ensuring each page remains focused on one Diataxis quadrant.Source: Coding guidelines
openapi/openapi.yaml (1)
11193-11217: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value
ExperimentFilterlackscreated_at/updated_atfilters present on sibling filter schemas.
EvaluationFilter,ModelEntityFilter,FilesetFilter, etc. all exposeDatetimeFilter-basedcreated_at/updated_at.ExperimentFilteronly hasname,insight_id,is_deleted,metadata. Consider adding for parity if the backend model supports it.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openapi/openapi.yaml` around lines 11193 - 11217, The ExperimentFilter schema is missing created_at and updated_at fields. Add both properties to ExperimentFilter using the same DatetimeFilter-based definitions and descriptions as the sibling filter schemas, preserving the existing name, insight_id, is_deleted, and metadata properties.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@openapi/ga/openapi.yaml`:
- Around line 10498-10501: Update the EvaluationFilter schema entry for
experiment_group_id to mark it deprecated, and add the experiment_ids filter
alongside it when supported by this endpoint, preserving the existing type and
descriptions for unaffected fields.
In
`@web/packages/studio/src/components/dataViews/ExperimentDataView/useSortErrorRecovery.ts`:
- Around line 25-26: Update isSameSort to compare the complete SortingState
arrays, including every sort entry’s id and desc direction rather than only
index 0. Preserve equality for identical multi-sort states and inequality when
any primary or secondary sort differs, and add a test covering recovery from a
failed secondary sort.
---
Outside diff comments:
In `@openapi/ga/openapi.yaml`:
- Around line 3639-3647: Update the Patch Evaluation documentation in the
operation description and its 400 response to consistently use “Experiment”
terminology instead of “ExperimentGroup” and “group,” including membership and
referenced-resource wording. Preserve the existing PATCH behavior and replace
only the stale terminology.
In `@web/packages/studio/src/components/ExperimentEditModal/index.tsx`:
- Around line 69-97: Finish the visible terminology migration by updating the
error toast, modal title, and instruction in ExperimentEditModal to use
“experiment” terminology instead of “group”; update the load-error message in
ExperimentDetailRoute likewise. Apply the requested wording changes at
web/packages/studio/src/components/ExperimentEditModal/index.tsx lines 69-97 and
web/packages/studio/src/routes/ExperimentDetailRoute/index.tsx lines 23-26.
---
Nitpick comments:
In `@openapi/openapi.yaml`:
- Around line 11193-11217: The ExperimentFilter schema is missing created_at and
updated_at fields. Add both properties to ExperimentFilter using the same
DatetimeFilter-based definitions and descriptions as the sibling filter schemas,
preserving the existing name, insight_id, is_deleted, and metadata properties.
In `@web/packages/studio/src/components/IntakeDetail/README.md`:
- Line 90: Split the Intake detail documentation into Diataxis-aligned pages:
keep the architecture overview in the current README, move route syntax into a
REFERENCE page, and move “Adding a kind” into a HOW-TO page. Add a prerequisites
section at the top and a Next Steps section to every resulting page, ensuring
each page remains focused on one Diataxis quadrant.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 84aea33a-2dcb-480e-b5e5-39a3361acca2
📒 Files selected for processing (76)
openapi/ga/individual/platform.openapi.yamlopenapi/ga/openapi.yamlopenapi/openapi.yamlservices/core/auth/src/nmp/core/auth/assets/static-authz.yamlservices/intake/scripts/spans/seed_experiment_rollup_data.pyservices/intake/scripts/spans/seed_experiments_demo.pyservices/intake/src/nmp/intake/api/v2/experiments/endpoints.pyservices/intake/src/nmp/intake/api/v2/experiments/schemas.pyservices/intake/src/nmp/intake/service.pyservices/intake/src/nmp/intake/spans/api/traces.pyservices/intake/src/nmp/intake/spans/api/traces_schemas.pyservices/intake/src/nmp/intake/spans/ingest/evaluation_context.pyservices/intake/tests/integration/spans/test_atif_ingest.pyservices/intake/tests/integration/spans/test_chat_completions_ingest.pyservices/intake/tests/integration/spans/test_experiment_metric_sort.pyservices/intake/tests/integration/spans/test_experiment_rollups.pyservices/intake/tests/integration/spans/test_experiment_sessions.pyservices/intake/tests/integration/spans/test_traces_read.pyservices/intake/tests/integration/test_experiments_crud.pyservices/intake/tests/test_atif_v17.pyservices/intake/tests/test_experiment_default_sort.pyservices/intake/tests/test_experiment_metric_filter.pyservices/intake/tests/test_experiment_optimization_fields.pyservices/intake/tests/test_experiment_sort_endpoint.pyservices/intake/tests/test_spans_schemas.pyservices/intake/tests/test_traces_api.pyweb/packages/studio/src/components/ExperimentCreateModal/constants.tsweb/packages/studio/src/components/ExperimentCreateModal/index.tsxweb/packages/studio/src/components/ExperimentEditModal/index.test.tsxweb/packages/studio/src/components/ExperimentEditModal/index.tsxweb/packages/studio/src/components/IntakeDetail/README.mdweb/packages/studio/src/components/charts/ExperimentParetoChart/MetricSelect.tsxweb/packages/studio/src/components/charts/ExperimentParetoChart/ParetoTooltip.tsxweb/packages/studio/src/components/charts/ExperimentParetoChart/index.tsxweb/packages/studio/src/components/charts/ExperimentParetoChart/useParetoEvaluations.tsweb/packages/studio/src/components/charts/ExperimentParetoChart/utils.test.tsweb/packages/studio/src/components/charts/ExperimentParetoChart/utils.tsweb/packages/studio/src/components/dataViews/EvaluationSessionsDataView/Empty.tsxweb/packages/studio/src/components/dataViews/EvaluationSessionsDataView/index.test.tsxweb/packages/studio/src/components/dataViews/EvaluationSessionsDataView/index.tsxweb/packages/studio/src/components/dataViews/ExperimentDataView/AddToGroupModal.test.tsxweb/packages/studio/src/components/dataViews/ExperimentDataView/AddToGroupModal.tsxweb/packages/studio/src/components/dataViews/ExperimentDataView/Empty.tsxweb/packages/studio/src/components/dataViews/ExperimentDataView/ExperimentDataView.cssweb/packages/studio/src/components/dataViews/ExperimentDataView/MeanValueTooltipCell.tsxweb/packages/studio/src/components/dataViews/ExperimentDataView/index.tsxweb/packages/studio/src/components/dataViews/ExperimentDataView/useExperimentEvaluations.test.tsweb/packages/studio/src/components/dataViews/ExperimentDataView/useExperimentEvaluations.tsweb/packages/studio/src/components/dataViews/ExperimentDataView/useSortErrorRecovery.test.tsweb/packages/studio/src/components/dataViews/ExperimentDataView/useSortErrorRecovery.tsweb/packages/studio/src/components/dataViews/ExperimentDataView/util.test.tsweb/packages/studio/src/components/dataViews/ExperimentDataView/util.tsweb/packages/studio/src/constants/routes.tsweb/packages/studio/src/mocks/handlers.tsweb/packages/studio/src/mocks/intake/experiments.tsweb/packages/studio/src/mocks/intake/telemetry.tsweb/packages/studio/src/routes/EvaluationDetailRoute/index.test.tsxweb/packages/studio/src/routes/EvaluationDetailRoute/index.tsxweb/packages/studio/src/routes/EvaluationDetailRoute/optimizerDisabled.test.tsxweb/packages/studio/src/routes/EvaluationSessionDetailRoute/TestCaseCompare.tsxweb/packages/studio/src/routes/EvaluationSessionDetailRoute/index.test.tsxweb/packages/studio/src/routes/EvaluationSessionDetailRoute/index.tsxweb/packages/studio/src/routes/EvaluationSessionDetailRoute/useSessionCompareRuns.tsweb/packages/studio/src/routes/ExperimentDetailRoute/ExperimentMetrics.tsxweb/packages/studio/src/routes/ExperimentDetailRoute/index.test.tsxweb/packages/studio/src/routes/ExperimentDetailRoute/index.tsxweb/packages/studio/src/routes/ExperimentRoute/ExperimentCard.tsxweb/packages/studio/src/routes/ExperimentRoute/index.tsxweb/packages/studio/src/routes/agents/ClaudeCodeChatRoute/artifacts.tsweb/packages/studio/src/routes/groups/experimentRoutes.tsxweb/packages/studio/src/routes/optimizer/OptimizerInsightRoute/InsightExperiments.tsxweb/packages/studio/src/routes/optimizer/OptimizerInsightRoute/index.test.tsxweb/packages/studio/src/routes/optimizer/OptimizerInsightRoute/index.tsxweb/packages/studio/src/routes/optimizer/OptimizerRoute/index.test.tsxweb/packages/studio/src/routes/utils.tsweb/packages/studio/src/tests/title-change.test.tsx
💤 Files with no reviewable changes (3)
- web/packages/studio/src/mocks/intake/telemetry.ts
- services/intake/tests/integration/spans/test_traces_read.py
- services/intake/tests/test_spans_schemas.py
Contributor
There was a problem hiding this comment.
Caution
Inline review comments failed to post. This is likely due to GitHub's internal server error or limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
web/packages/studio/src/components/ExperimentEditModal/index.tsx (1)
69-97: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winFinish the visible terminology migration.
web/packages/studio/src/components/ExperimentEditModal/index.tsx#L69-L97: change the toast, modal title, and instruction to “experiment”.web/packages/studio/src/routes/ExperimentDetailRoute/index.tsx#L23-L26: change the load-error message to “experiment”.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/ExperimentEditModal/index.tsx` around lines 69 - 97, Finish the visible terminology migration by updating the error toast, modal title, and instruction in ExperimentEditModal to use “experiment” terminology instead of “group”; update the load-error message in ExperimentDetailRoute likewise. Apply the requested wording changes at web/packages/studio/src/components/ExperimentEditModal/index.tsx lines 69-97 and web/packages/studio/src/routes/ExperimentDetailRoute/index.tsx lines 23-26.openapi/ga/openapi.yaml (1)
3639-3647: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winStale "ExperimentGroup"/"group" wording in Patch Evaluation docs.
The PR's objective is renaming ExperimentGroup children to Experiment, but this endpoint's
description("curating an evaluation into another ExperimentGroup", "any new group must exist... belongs to >=1 group") and its400response ("A referenced ExperimentGroup does not exist") still use the old terminology, contradicting the rest of this same file (e.g.EvaluationPatchRequestdocstring at Line 10607-10612 already says "experiment").🔧 Suggested fix
description: "Partially update an evaluation: only fields present in the request\ - \ are changed.\n\nThe common case is curating an evaluation into another ExperimentGroup\ - \ \u2014 PATCH with the merged\n``experiment_ids``. Membership is replaced\ - \ (not appended), so send the full desired set; any new\ngroup must exist\ - \ and the set must be non-empty (an evaluation always belongs to >=1 group).\ + \ are changed.\n\nThe common case is curating an evaluation into another Experiment\ + \ \u2014 PATCH with the merged\n``experiment_ids``. Membership is replaced\ + \ (not appended), so send the full desired set; any new\nexperiment must exist\ + \ and the set must be non-empty (an evaluation always belongs to >=1 experiment).\ \ Omitted\nfields are left untouched (unlike the full-body PUT, which overwrites\ \ them)." ... '400': - description: A referenced ExperimentGroup does not exist, or experiment_ids + description: A referenced Experiment does not exist, or experiment_ids is emptyAlso applies to: 3674-3677
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openapi/ga/openapi.yaml` around lines 3639 - 3647, Update the Patch Evaluation documentation in the operation description and its 400 response to consistently use “Experiment” terminology instead of “ExperimentGroup” and “group,” including membership and referenced-resource wording. Preserve the existing PATCH behavior and replace only the stale terminology.
🧹 Nitpick comments (2)
web/packages/studio/src/components/IntakeDetail/README.md (1)
90-90: 📐 Maintainability & Code Quality | 🔵 Trivial | 🏗️ Heavy liftSplit this into Diataxis pages.
Keep architecture here; move route syntax to REFERENCE and “Adding a kind” to a HOW-TO. Add prerequisites and Next Steps.
As per coding guidelines, each documentation page must fit one Diataxis quadrant, list prerequisites at the top, and include a Next Steps section.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/IntakeDetail/README.md` at line 90, Split the Intake detail documentation into Diataxis-aligned pages: keep the architecture overview in the current README, move route syntax into a REFERENCE page, and move “Adding a kind” into a HOW-TO page. Add a prerequisites section at the top and a Next Steps section to every resulting page, ensuring each page remains focused on one Diataxis quadrant.Source: Coding guidelines
openapi/openapi.yaml (1)
11193-11217: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value
ExperimentFilterlackscreated_at/updated_atfilters present on sibling filter schemas.
EvaluationFilter,ModelEntityFilter,FilesetFilter, etc. all exposeDatetimeFilter-basedcreated_at/updated_at.ExperimentFilteronly hasname,insight_id,is_deleted,metadata. Consider adding for parity if the backend model supports it.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openapi/openapi.yaml` around lines 11193 - 11217, The ExperimentFilter schema is missing created_at and updated_at fields. Add both properties to ExperimentFilter using the same DatetimeFilter-based definitions and descriptions as the sibling filter schemas, preserving the existing name, insight_id, is_deleted, and metadata properties.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@openapi/ga/openapi.yaml`:
- Around line 10498-10501: Update the EvaluationFilter schema entry for
experiment_group_id to mark it deprecated, and add the experiment_ids filter
alongside it when supported by this endpoint, preserving the existing type and
descriptions for unaffected fields.
In
`@web/packages/studio/src/components/dataViews/ExperimentDataView/useSortErrorRecovery.ts`:
- Around line 25-26: Update isSameSort to compare the complete SortingState
arrays, including every sort entry’s id and desc direction rather than only
index 0. Preserve equality for identical multi-sort states and inequality when
any primary or secondary sort differs, and add a test covering recovery from a
failed secondary sort.
---
Outside diff comments:
In `@openapi/ga/openapi.yaml`:
- Around line 3639-3647: Update the Patch Evaluation documentation in the
operation description and its 400 response to consistently use “Experiment”
terminology instead of “ExperimentGroup” and “group,” including membership and
referenced-resource wording. Preserve the existing PATCH behavior and replace
only the stale terminology.
In `@web/packages/studio/src/components/ExperimentEditModal/index.tsx`:
- Around line 69-97: Finish the visible terminology migration by updating the
error toast, modal title, and instruction in ExperimentEditModal to use
“experiment” terminology instead of “group”; update the load-error message in
ExperimentDetailRoute likewise. Apply the requested wording changes at
web/packages/studio/src/components/ExperimentEditModal/index.tsx lines 69-97 and
web/packages/studio/src/routes/ExperimentDetailRoute/index.tsx lines 23-26.
---
Nitpick comments:
In `@openapi/openapi.yaml`:
- Around line 11193-11217: The ExperimentFilter schema is missing created_at and
updated_at fields. Add both properties to ExperimentFilter using the same
DatetimeFilter-based definitions and descriptions as the sibling filter schemas,
preserving the existing name, insight_id, is_deleted, and metadata properties.
In `@web/packages/studio/src/components/IntakeDetail/README.md`:
- Line 90: Split the Intake detail documentation into Diataxis-aligned pages:
keep the architecture overview in the current README, move route syntax into a
REFERENCE page, and move “Adding a kind” into a HOW-TO page. Add a prerequisites
section at the top and a Next Steps section to every resulting page, ensuring
each page remains focused on one Diataxis quadrant.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 84aea33a-2dcb-480e-b5e5-39a3361acca2
📒 Files selected for processing (76)
openapi/ga/individual/platform.openapi.yamlopenapi/ga/openapi.yamlopenapi/openapi.yamlservices/core/auth/src/nmp/core/auth/assets/static-authz.yamlservices/intake/scripts/spans/seed_experiment_rollup_data.pyservices/intake/scripts/spans/seed_experiments_demo.pyservices/intake/src/nmp/intake/api/v2/experiments/endpoints.pyservices/intake/src/nmp/intake/api/v2/experiments/schemas.pyservices/intake/src/nmp/intake/service.pyservices/intake/src/nmp/intake/spans/api/traces.pyservices/intake/src/nmp/intake/spans/api/traces_schemas.pyservices/intake/src/nmp/intake/spans/ingest/evaluation_context.pyservices/intake/tests/integration/spans/test_atif_ingest.pyservices/intake/tests/integration/spans/test_chat_completions_ingest.pyservices/intake/tests/integration/spans/test_experiment_metric_sort.pyservices/intake/tests/integration/spans/test_experiment_rollups.pyservices/intake/tests/integration/spans/test_experiment_sessions.pyservices/intake/tests/integration/spans/test_traces_read.pyservices/intake/tests/integration/test_experiments_crud.pyservices/intake/tests/test_atif_v17.pyservices/intake/tests/test_experiment_default_sort.pyservices/intake/tests/test_experiment_metric_filter.pyservices/intake/tests/test_experiment_optimization_fields.pyservices/intake/tests/test_experiment_sort_endpoint.pyservices/intake/tests/test_spans_schemas.pyservices/intake/tests/test_traces_api.pyweb/packages/studio/src/components/ExperimentCreateModal/constants.tsweb/packages/studio/src/components/ExperimentCreateModal/index.tsxweb/packages/studio/src/components/ExperimentEditModal/index.test.tsxweb/packages/studio/src/components/ExperimentEditModal/index.tsxweb/packages/studio/src/components/IntakeDetail/README.mdweb/packages/studio/src/components/charts/ExperimentParetoChart/MetricSelect.tsxweb/packages/studio/src/components/charts/ExperimentParetoChart/ParetoTooltip.tsxweb/packages/studio/src/components/charts/ExperimentParetoChart/index.tsxweb/packages/studio/src/components/charts/ExperimentParetoChart/useParetoEvaluations.tsweb/packages/studio/src/components/charts/ExperimentParetoChart/utils.test.tsweb/packages/studio/src/components/charts/ExperimentParetoChart/utils.tsweb/packages/studio/src/components/dataViews/EvaluationSessionsDataView/Empty.tsxweb/packages/studio/src/components/dataViews/EvaluationSessionsDataView/index.test.tsxweb/packages/studio/src/components/dataViews/EvaluationSessionsDataView/index.tsxweb/packages/studio/src/components/dataViews/ExperimentDataView/AddToGroupModal.test.tsxweb/packages/studio/src/components/dataViews/ExperimentDataView/AddToGroupModal.tsxweb/packages/studio/src/components/dataViews/ExperimentDataView/Empty.tsxweb/packages/studio/src/components/dataViews/ExperimentDataView/ExperimentDataView.cssweb/packages/studio/src/components/dataViews/ExperimentDataView/MeanValueTooltipCell.tsxweb/packages/studio/src/components/dataViews/ExperimentDataView/index.tsxweb/packages/studio/src/components/dataViews/ExperimentDataView/useExperimentEvaluations.test.tsweb/packages/studio/src/components/dataViews/ExperimentDataView/useExperimentEvaluations.tsweb/packages/studio/src/components/dataViews/ExperimentDataView/useSortErrorRecovery.test.tsweb/packages/studio/src/components/dataViews/ExperimentDataView/useSortErrorRecovery.tsweb/packages/studio/src/components/dataViews/ExperimentDataView/util.test.tsweb/packages/studio/src/components/dataViews/ExperimentDataView/util.tsweb/packages/studio/src/constants/routes.tsweb/packages/studio/src/mocks/handlers.tsweb/packages/studio/src/mocks/intake/experiments.tsweb/packages/studio/src/mocks/intake/telemetry.tsweb/packages/studio/src/routes/EvaluationDetailRoute/index.test.tsxweb/packages/studio/src/routes/EvaluationDetailRoute/index.tsxweb/packages/studio/src/routes/EvaluationDetailRoute/optimizerDisabled.test.tsxweb/packages/studio/src/routes/EvaluationSessionDetailRoute/TestCaseCompare.tsxweb/packages/studio/src/routes/EvaluationSessionDetailRoute/index.test.tsxweb/packages/studio/src/routes/EvaluationSessionDetailRoute/index.tsxweb/packages/studio/src/routes/EvaluationSessionDetailRoute/useSessionCompareRuns.tsweb/packages/studio/src/routes/ExperimentDetailRoute/ExperimentMetrics.tsxweb/packages/studio/src/routes/ExperimentDetailRoute/index.test.tsxweb/packages/studio/src/routes/ExperimentDetailRoute/index.tsxweb/packages/studio/src/routes/ExperimentRoute/ExperimentCard.tsxweb/packages/studio/src/routes/ExperimentRoute/index.tsxweb/packages/studio/src/routes/agents/ClaudeCodeChatRoute/artifacts.tsweb/packages/studio/src/routes/groups/experimentRoutes.tsxweb/packages/studio/src/routes/optimizer/OptimizerInsightRoute/InsightExperiments.tsxweb/packages/studio/src/routes/optimizer/OptimizerInsightRoute/index.test.tsxweb/packages/studio/src/routes/optimizer/OptimizerInsightRoute/index.tsxweb/packages/studio/src/routes/optimizer/OptimizerRoute/index.test.tsxweb/packages/studio/src/routes/utils.tsweb/packages/studio/src/tests/title-change.test.tsx
💤 Files with no reviewable changes (3)
- web/packages/studio/src/mocks/intake/telemetry.ts
- services/intake/tests/integration/spans/test_traces_read.py
- services/intake/tests/test_spans_schemas.py
🛑 Comments failed to post (1)
web/packages/studio/src/components/dataViews/ExperimentDataView/useSortErrorRecovery.ts (1)
25-26: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Compare the complete multi-sort state.
ExperimentDataViewenablesmultiSort, but this compares only index 0. A failed secondary sort skips recovery and replaces the table with the full error state. Compare all entries and add a secondary-sort failure test.Proposed fix
const isSameSort = (a: SortingState, b: SortingState): boolean => - a[0]?.id === b[0]?.id && (a[0]?.desc ?? false) === (b[0]?.desc ?? false); + a.length === b.length && + a.every( + (sort, index) => + sort.id === b[index]?.id && + (sort.desc ?? false) === (b[index]?.desc ?? false) + );📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.const isSameSort = (a: SortingState, b: SortingState): boolean => a.length === b.length && a.every( (sort, index) => sort.id === b[index]?.id && (sort.desc ?? false) === (b[index]?.desc ?? false) );🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/dataViews/ExperimentDataView/useSortErrorRecovery.ts` around lines 25 - 26, Update isSameSort to compare the complete SortingState arrays, including every sort entry’s id and desc direction rather than only index 0. Preserve equality for identical multi-sort states and inequality when any primary or secondary sort differs, and add a test covering recovery from a failed secondary sort.
Contributor
|
Signed-off-by: shanaiabuggy <59746633+shanaiabuggy@users.noreply.github.com>
Update nemo-experimentalist and nemo-evaluator to the regenerated SDK: client.experiment_groups -> client.experiments, and the deprecated evaluation write experiment_group_id -> experiment_ids. No behavior change; the intake API keeps the legacy surface for other callers. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: shanaiabuggy <59746633+shanaiabuggy@users.noreply.github.com>
…ASE-550] Update the nemo-experiments-upload skill to the renamed parent resource: Experiment Group -> Experiment, /experiment-groups -> /experiments, and the deprecated experiment_group_id write -> experiment_ids. Fix the stale gotcha that called /experiments a deprecated evaluations alias (it is now the parent resource; /experiment-groups is the hidden alias). Sync the vendored SDK copy. Also apply Prettier formatting to the renamed Studio components. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: shanaiabuggy <59746633+shanaiabuggy@users.noreply.github.com>
shanaiabuggy
commented
Jul 28, 2026
shanaiabuggy
commented
Jul 28, 2026
shanaiabuggy
commented
Jul 28, 2026
BrianNewsom
approved these changes
Jul 28, 2026
Signed-off-by: shanaiabuggy <59746633+shanaiabuggy@users.noreply.github.com>
Signed-off-by: shanaiabuggy <59746633+shanaiabuggy@users.noreply.github.com>
Signed-off-by: shanaiabuggy <59746633+shanaiabuggy@users.noreply.github.com>
Signed-off-by: shanaiabuggy <59746633+shanaiabuggy@users.noreply.github.com>
Signed-off-by: shanaiabuggy <59746633+shanaiabuggy@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ASE-550: Rename parent resource
ExperimentGroup→Experiment(non-breaking)Completes the Experiment→Evaluation rename pair: the child rename shipped earlier, and this renames the parent API resource from
ExperimentGrouptoExperimentacross intake, Studio, the Python SDK, and the CLI. The API now reads Experiment → Evaluation. Stored entity types are unchanged.Non-breaking at the API layer: legacy
/experiment-groupsroutes remain as hidden aliases and the deprecatedexperiment_group_idfield is still accepted and returned, so existing callers keep working.Also in this PR:
parent_experiment_id,experiment_context,experiment_name).nemo-experimentalist,nemo-evaluator) to the new SDK surface (client.experiments,experiment_ids).Summary by CodeRabbit
New Features
Breaking Changes
experiment_ids.Documentation
Compatibility