refactor(evaluator): rename experiment_id to evaluation_name in publish-to-intake

[shanaiabuggy]

What

publish_to_intake and its mapping helpers took an argument called experiment_id. It was never an id, and after the Experiment → Evaluation rename it is not an Experiment either:

• Intake resolves the entity by name (validate_evaluation_context does entity_client.get(Experiment, name=...)), so an id was always the wrong thing to pass. Passing one gets 400 "Evaluation 'X' must be created before it can be logged".
• client.experiments now means Experiment groups, while the entity this argument names is an Evaluation (client.evaluations). The old name pointed at the wrong entity as well as the wrong field.

Changes

Renamed experiment_id → evaluation_name on:

• publish_to_intake(...)
• mapping.trial_to_atif_ingest(...)
• mapping.run_task_to_evaluation_context(...)
• the PublishReport.experiment_id field

Docstrings updated to say which entity has to exist first and who creates it. The wire field stays evaluation_id since that is Intake's contract; the docstring now calls out that it carries a name despite the _id suffix.

Notes

No behavior change. Safe as a plain rename because the module has no production callers yet — publish_to_intake is referenced only by its own tests, so nothing outside this diff passes the keyword.

experiment_ids=[group.id] in the integration test is deliberately untouched: that is the real SDK field linking an Evaluation to its Experiment groups.

Verification

• pytest plugins/nemo-evaluator/tests/intake → 32 passed
• ty check on the changed source and tests → clean
• ruff check + ruff format --check → clean
• pytest plugins/nemo-evaluator/tests/integration/test_publish_to_intake.py --collect-only → 2 collected (needs Docker to actually run)

🤖 Generated with Claude Code

Summary by CodeRabbit

• Breaking Changes

  • Publishing and evaluation context workflows now use an evaluation name instead of an experiment ID.
  • Callers must create the Evaluation and its associated Experiment group separately before publishing.

• Documentation

  • Updated publishing guidance to reflect the new Evaluation-based workflow.

• Tests

  • Updated unit and integration coverage for evaluation naming, publishing, and failed-score scenarios.

[coderabbitai]

📝 Walkthrough

Walkthrough

experiment_id is renamed to evaluation_name across intake mapping and publishing APIs. Evaluation names populate evaluation context and publish reports. Unit and integration tests use the updated terminology and assertions.

Changes

Evaluation naming migration

Layer / File(s)	Summary
Intake API and context updates plugins/nemo-evaluator/src/nemo_evaluator/intake/mapping.py, plugins/nemo-evaluator/src/nemo_evaluator/intake/publish.py	Mapping and publishing functions now accept evaluation_name. Evaluation context and PublishReport use the evaluation name. Documentation requires the Evaluation to exist before publishing.
Evaluation naming validation plugins/nemo-evaluator/tests/intake/test_mapping.py, plugins/nemo-evaluator/tests/intake/test_publish.py, plugins/nemo-evaluator/tests/integration/test_publish_to_intake.py	Unit and integration tests use evaluation names for publishing, evaluation creation, and evaluation-context assertions.

Possibly related PRs

• NVIDIA-NeMo/nemo-platform#618: Updates intake publishing and mapping to use evaluation_context.evaluation_id.
• NVIDIA-NeMo/nemo-platform#742: Extends the evaluation-context migration across intake APIs and report fields.

Suggested labels: chore

Suggested reviewers: arpitsardhana, ngoncharenko, sandychapman

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the rename of experiment_id to evaluation_name in publish-to-intake.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch sbuggy/rename-experiment-id-to-evaluation-name

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

🧹 Nitpick comments (1)

plugins/nemo-evaluator/tests/intake/test_publish.py (1)

138-138: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Assert PublishReport.evaluation_name.

Add assert report.evaluation_name == "exp-1" after the publish call. This verifies the renamed report field, not only the wire payload.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@plugins/nemo-evaluator/tests/intake/test_publish.py` at line 138, Add an
assertion immediately after the publish_to_intake call verifying that
report.evaluation_name equals "exp-1", so the test checks the renamed
PublishReport field in addition to the wire payload.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@plugins/nemo-evaluator/tests/intake/test_publish.py`:
- Line 138: Add an assertion immediately after the publish_to_intake call
verifying that report.evaluation_name equals "exp-1", so the test checks the
renamed PublishReport field in addition to the wire payload.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 4ce3239c-2f53-4295-9352-0ce2a5bea457

📥 Commits

Reviewing files that changed from the base of the PR and between c71ca67 and 92467c0.

📒 Files selected for processing (5)

• plugins/nemo-evaluator/src/nemo_evaluator/intake/mapping.py
• plugins/nemo-evaluator/src/nemo_evaluator/intake/publish.py
• plugins/nemo-evaluator/tests/intake/test_mapping.py
• plugins/nemo-evaluator/tests/intake/test_publish.py
• plugins/nemo-evaluator/tests/integration/test_publish_to_intake.py