NVIDIA-NeMo · shanaiabuggy · Aug 13, 2026 · Aug 11, 2026 · Aug 11, 2026 · Aug 13, 2026
@@ -231,8 +231,8 @@ session with the Evaluation's identity:
 - For Agent Trajectory Interchange Format (ATIF) and chat-completions, add a **top-level**
   `evaluation_context` object to the ingest payload
   carrying `evaluation_id` (the Evaluation's **name**) and `test_case_id`.
-- For OpenTelemetry Protocol (OTLP), set the `nemo.experiment.id` and `nemo.test_case.id` root-span
-  attributes.
+- For OpenTelemetry Protocol (OTLP), set the `nemo.evaluation.name` (the Evaluation's **name**) and
+  `nemo.test_case.id` root-span attributes.
 
 The per-evaluator scores on the leaderboard come from **evaluator results** captured on those
 sessions, either automatically from ATIF verifier rewards or explicitly through the evaluator-results

@@ -130,7 +130,7 @@ curl -sf -X POST \
 Pick the ingest endpoint that matches your producer. **Read `../nemo-intake/references/ingest-formats.md` for the full schema and a copy-pasteable example for each.** How you attach evaluation identity depends on the endpoint:
 
 - **ATIF and chat-completions** (JSON body) — add an `evaluation_context = {evaluation_id: "<the Evaluation name>", test_case_id: "<task id>"}` object to the payload.
-- **OTLP** — there is no body field; set `nemo.experiment.id` (the Evaluation **name**) and
+- **OTLP** — there is no body field; set `nemo.evaluation.name` (the Evaluation **name**) and
   `nemo.test_case.id` (the task ID) as **attributes on the root span**. Spans missing these still
   ingest but won't associate to an Evaluation.
 
@@ -176,23 +176,23 @@ You succeeded when `GET .../evaluations/my-eval-baseline` shows:
 - `run_count` ≥ 1 (each ingested session counts as one run), and
 - non-empty `evaluator_names` / `aggregate_scores` if you logged rewards, and/or `cost_usd` if your spans carried cost.
 
-If `run_count` is 0 after ingesting, the traces didn't associate — almost always a wrong evaluation identity: `evaluation_context.evaluation_id` for ATIF/chat-completions, or the `nemo.experiment.id` root-span attribute for OTLP (see Gotchas).
+If `run_count` is 0 after ingesting, the traces didn't associate — almost always a wrong evaluation identity: `evaluation_context.evaluation_id` for ATIF/chat-completions, or the `nemo.evaluation.name` root-span attribute for OTLP (see Gotchas).
 
 ## If verification fails
 
 | Symptom | Cause | Recovery |
 |---|---|---|
 | `400 "…must be created before it can be logged."` | Ingested before the Evaluation existed, or `evaluation_id` doesn't match | Create the Evaluation (step 2); ensure `evaluation_context.evaluation_id` equals its **name** |
 | `422 Unprocessable` on ingest | Unknown/typo'd top-level key (ATIF/chat-completions are `extra="forbid"`) or bad `schema_version` | Check the exact schema in `../nemo-intake/references/ingest-formats.md`; remove stray keys |
-| Ingest 2xx but `run_count` stays 0 | Evaluation identity missing/wrong — `evaluation_context.evaluation_id` (ATIF/chat-completions) or the `nemo.experiment.id` root-span attribute (OTLP) ≠ the Evaluation's name | Attach the identity for your endpoint; use the Evaluation **name**, not its id |
+| Ingest 2xx but `run_count` stays 0 | Evaluation identity missing/wrong — `evaluation_context.evaluation_id` (ATIF/chat-completions) or the `nemo.evaluation.name` root-span attribute (OTLP) ≠ the Evaluation's name | Attach the identity for your endpoint; use the Evaluation **name**, not its id |
 | `503` on GET evaluation / sessions | ClickHouse (telemetry store) not running | Start ClickHouse; rollups and sessions require it |
 | Scores don't show up | Rewards not under `extra.verifier_result.rewards`, or wrong `data_type` on `/evaluator-results` | See `references/troubleshooting.md` |
 
 ## Gotchas
 
 - **Create before you log.** The Evaluation entity must exist before any ingest referencing it — otherwise `400`.
 - **`evaluation_id` is the Evaluation's `name`, not its entity id.** But **`experiment_ids` holds the Experiment's `id`.** Different identifiers; easy to swap.
-- **OTLP uses the attribute key `nemo.experiment.id`** (and `nemo.test_case.id`) — the span-attribute key still says "experiment" even though the JSON body field is `evaluation_context`. Set `nemo.experiment.id` on your root span.
+- **OTLP uses the attribute key `nemo.evaluation.name`** (and `nemo.test_case.id`) — set it to the Evaluation's **name** on your root span, matching the `evaluation_id` field the JSON `evaluation_context` carries on the other endpoints.
 - **The parent lives at `/experiments`; `/experiment-groups` is a deprecated hidden alias.** Prefer `/experiments`. Evaluations are created and logged under `/evaluations`.
 - **`metadata` is `dict[str, str]`** — stringify non-string values or you'll get a `422`.
 - **ATIF and chat-completions are `extra="forbid"`** (unknown keys → 422); `evaluation_context` itself is lenient (`extra="ignore"`).
@@ -29,7 +29,7 @@ string — read it first.
 
 | Symptom | Cause | Fix |
 |---|---|---|
-| Ingest returned 2xx but `run_count` stays 0 | `evaluation_context` missing, or `evaluation_id` ≠ the Evaluation's name | Attach `evaluation_context`; use the Evaluation **name**. For OTLP, set the span attribute `nemo.experiment.id` on the root span |
+| Ingest returned 2xx but `run_count` stays 0 | `evaluation_context` missing, or `evaluation_id` ≠ the Evaluation's name | Attach `evaluation_context`; use the Evaluation **name**. For OTLP, set the span attribute `nemo.evaluation.name` on the root span |
 | No scores on the evaluation | Rewards not under `extra.verifier_result.rewards` (ATIF), or wrong `data_type` (`/evaluator-results`) | ATIF: `extra.verifier_result.rewards = {criterion: value}`. Explicit: `NUMERIC`/`BOOLEAN` need `value`, `CATEGORICAL`/`TEXT` need `string_value` |
 | No cost on the rollup | The producer never emitted cost | Cost is pass-through — set `cost_usd` (chat-completions / ATIF step `metrics`) or `llm.cost.total` / `gen_ai.usage.cost` (OTLP) |
 | `503` on `GET .../evaluations/{name}` or `/sessions` | ClickHouse (telemetry store) not running | Start ClickHouse; rollups, sessions, and metric sorts/filters all need it |
@@ -39,5 +39,5 @@ string — read it first.
 
 - `evaluation_context.evaluation_id` → the Evaluation's **`name`**.
 - `experiment_ids` (on create evaluation) → a list with the Experiment's **`id`**.
-- OTLP evaluation attribute key → **`nemo.experiment.id`** (test case → `nemo.test_case.id`).
+- OTLP evaluation attribute key → **`nemo.evaluation.name`** (test case → `nemo.test_case.id`).
 - Parent → **`/experiments`** (`/experiment-groups` is a deprecated hidden alias); evaluations → **`/evaluations`**.
@@ -128,11 +128,11 @@ the root span:
 
 | Meaning | Span attribute key |
 |---|---|
-| Evaluation (by name) | **`nemo.experiment.id`** |
+| Evaluation (by name) | **`nemo.evaluation.name`** |
 | Test case | **`nemo.test_case.id`** |
 
-> Note the key is `nemo.experiment.id` (still "experiment"), even though the REST body field elsewhere
-> is `evaluation_context`. Set `nemo.experiment.id` to the Evaluation's **name**.
+> Set `nemo.evaluation.name` to the Evaluation's **name** (not its id), matching the `evaluation_id`
+> field used by the JSON `evaluation_context` on the other endpoints.
 
 Cost/token/model attributes are read from standard GenAI / OpenInference keys (first match wins):
 
@@ -158,7 +158,7 @@ export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT="${NMP_BASE_URL}/apis/intake/v2/worksp
 export OTEL_EXPORTER_OTLP_TRACES_PROTOCOL="http/protobuf"
 ```
 
-Then set `nemo.experiment.id` (+ `nemo.test_case.id`) on the root span of each run.
+Then set `nemo.evaluation.name` (+ `nemo.test_case.id`) on the root span of each run.
 
 ---
 

@@ -43,7 +43,7 @@ def _positive_int(value: str) -> int:
     return parsed
 
 
-def _experiment_id() -> str:
+def _evaluation_name() -> str:
     stamp = datetime.now(UTC).strftime("%Y%m%d-%H%M%S")
     return f"tau3-airline-{stamp}-{secrets.token_hex(2)}"
 
@@ -96,7 +96,7 @@ def _resolve_harbor_output_dir(path: Path) -> tuple[Path, Path]:
 def _write_upload_summary(
     run_dir: Path,
     *,
-    experiment_id: str,
+    evaluation_name: str,
     workspace: str,
     agent_name: str,
     agent_version: str,
@@ -108,7 +108,7 @@ def _write_upload_summary(
     summary_path.write_text(
         json.dumps(
             {
-                "experiment_id": experiment_id,
+                "evaluation_name": evaluation_name,
                 "workspace": workspace,
                 "agent_name": agent_name,
                 "agent_version": agent_version,
@@ -133,7 +133,7 @@ async def _upload_trials(
     trials: list[TrialResult],
     *,
     workspace: str,
-    experiment_id: str,
+    evaluation_name: str,
     agent_name: str,
     agent_version: str,
     model: str,
@@ -149,7 +149,7 @@ async def _upload_trials(
             raise RuntimeError(f"Trace {trace_id} was produced by more than one trial")
 
         attrs = {
-            "nemo.experiment.id": experiment_id,
+            "nemo.evaluation.name": evaluation_name,
             "nemo.test_case.id": trial.task_id,
             "nemo.trial.id": trial.id,
             "gen_ai.agent.name": agent_name,
@@ -220,7 +220,7 @@ async def run(args: argparse.Namespace) -> Path:
         if not uploadable_trials:
             raise RuntimeError(f"No completed Harbor trials with trace artifacts found in {job_dir}")
 
-        experiment_id = args.experiment_id or run_dir.name
+        evaluation_name = args.evaluation_name or run_dir.name
         client = make_client(args.base_url)
         try:
             await client.workspaces.create(
@@ -232,7 +232,7 @@ async def run(args: argparse.Namespace) -> Path:
                 client,
                 uploadable_trials,
                 workspace=args.workspace,
-                experiment_id=experiment_id,
+                evaluation_name=evaluation_name,
                 agent_name=args.agent_name,
                 agent_version=args.agent_version,
                 model=args.model,
@@ -243,7 +243,7 @@ async def run(args: argparse.Namespace) -> Path:
 
         summary_path = _write_upload_summary(
             run_dir,
-            experiment_id=experiment_id,
+            evaluation_name=evaluation_name,
             workspace=args.workspace,
             agent_name=args.agent_name,
             agent_version=args.agent_version,
@@ -255,8 +255,8 @@ async def run(args: argparse.Namespace) -> Path:
         print(summary_path)
         return summary_path
 
-    experiment_id = args.experiment_id or _experiment_id()
-    run_dir = args.output.expanduser().resolve() / experiment_id
+    evaluation_name = args.evaluation_name or _evaluation_name()
+    run_dir = args.output.expanduser().resolve() / evaluation_name
     if run_dir.exists():
         raise FileExistsError(f"Output directory already exists: {run_dir}")
 
@@ -294,7 +294,7 @@ async def run(args: argparse.Namespace) -> Path:
             client,
             trials,
             workspace=args.workspace,
-            experiment_id=experiment_id,
+            evaluation_name=evaluation_name,
             agent_name=args.agent_name,
             agent_version=args.agent_version,
             model=args.model,
@@ -305,7 +305,7 @@ async def run(args: argparse.Namespace) -> Path:
 
     summary_path = _write_upload_summary(
         run_dir,
-        experiment_id=experiment_id,
+        evaluation_name=evaluation_name,
         workspace=args.workspace,
         agent_name=args.agent_name,
         agent_version=args.agent_version,
@@ -338,7 +338,7 @@ def parse_args() -> argparse.Namespace:
     parser.add_argument("--user-model", default=os.environ.get("TAU2_USER_MODEL", DEFAULT_MODEL))
     parser.add_argument("--agent-name", default=DEFAULT_AGENT_NAME)
     parser.add_argument("--agent-version", default=DEFAULT_AGENT_VERSION)
-    parser.add_argument("--experiment-id")
+    parser.add_argument("--evaluation-name")
     parser.add_argument("--expected-task-count", type=_positive_int, default=20)
     parser.add_argument(
         "--task-id",

@@ -51,7 +51,7 @@ def read_session_id(ref: ResourceRef) -> str:
 def build_ingest_payload(
     ref: ResourceRef,
     *,
-    experiment_id: str,
+    evaluation_name: str,
     task_id: str,
     agent_attrs: dict[str, str],
 ) -> dict[str, Any]:
@@ -66,7 +66,7 @@ def build_ingest_payload(
 
     Args:
         ref(ResourceRef): Resource reference to the ATIF trajectory file.
-        experiment_id(str): Evaluation name, used as ``evaluation_context.evaluation_id``.
+        evaluation_name(str): Evaluation name, used as ``evaluation_context.evaluation_id``.
         task_id(str): Test case id, used as ``evaluation_context.test_case_id``.
         agent_attrs(dict[str, str]): OTLP-style agent attributes used as fallbacks.
 
@@ -75,7 +75,7 @@ def build_ingest_payload(
     """
     trajectory = _load(ref)
     trajectory["evaluation_context"] = {
-        "evaluation_id": experiment_id,
+        "evaluation_id": evaluation_name,
         "test_case_id": task_id,
     }
     agent = trajectory.get("agent")

@@ -145,7 +145,7 @@ async def ensure_experiment(self, candidate: Candidate, split: str) -> str:
         name (``opt-<run>-<label>-<split>``).
 
         The name — not the server-assigned id (``experiment-…``) — is what tags the trace's
-        ``nemo.experiment.id``: it is human-readable, greppable, and stable across resumes,
+        ``nemo.evaluation.name``: it is human-readable, greppable, and stable across resumes,
         and matches the Experiment's own ``name`` so the trace still joins back to it.
         """
         gname = group_name(candidate.run_id)

@@ -58,7 +58,7 @@ async def _upload_trace_otlp(
     workspace: str,
     ref: ResourceRef,
     *,
-    experiment_id: str,
+    evaluation_name: str,
     trial_id: str,
     task_id: str,
     extra_attrs: dict[str, str] | None = None,
@@ -67,7 +67,7 @@ async def _upload_trace_otlp(
 
     path = Path(urlparse(ref.uri).path)
     attrs: dict[str, str] = {
-        "nemo.experiment.id": experiment_id,
+        "nemo.evaluation.name": evaluation_name,
         "nemo.test_case.id": task_id,
         "nemo.trial.id": trial_id,
         **(extra_attrs or {}),
@@ -87,7 +87,7 @@ async def _upload_trace_atif(
     workspace: str,
     ref: ResourceRef,
     *,
-    experiment_id: str,
+    evaluation_name: str,
     task_id: str,
     extra_attrs: dict[str, str] | None = None,
 ) -> None:
@@ -100,7 +100,7 @@ async def _upload_trace_atif(
 
     payload = build_ingest_payload(
         ref,
-        experiment_id=experiment_id,
+        evaluation_name=evaluation_name,
         task_id=task_id,
         agent_attrs=extra_attrs or {},
     )
@@ -206,16 +206,16 @@ async def persist_evaluation(
     ) -> None:
         """Persist traces and metrics for a completed evaluation.
 
-        Derives experiment_id internally from candidate × split, ensuring the
-        Experiment entity exists before stamping traces.
+        Derives the Evaluation name internally from candidate × split, ensuring
+        the Evaluation exists before stamping traces.
         """
         ...
 
     @abstractmethod
-    async def get_experiment_id(self, *, workspace: str, candidate: Candidate, split: str) -> str:
-        """Best-effort native Experiment id for *candidate* × *split* in *workspace*.
+    async def get_evaluation_name(self, *, workspace: str, candidate: Candidate, split: str) -> str:
+        """Best-effort Evaluation name for *candidate* × *split* in *workspace*.
 
-        Returns "" when there is no projection (offline) or it fails — the id only
+        Returns "" when there is no projection (offline) or it fails — the name only
         tags Intake resource attributes, so a run must not break on it.
         """
         ...
@@ -630,9 +630,9 @@ async def persist_evaluation(
         if self.client is None:
             return  # pure-offline run: traces stay on local disk
 
-        # Derive experiment_id internally (ensures entity exists, returns deterministic name)
-        experiment_id = await self.get_experiment_id(workspace=workspace, candidate=candidate, split=split)
-        if not experiment_id:
+        # Derive the Evaluation name internally (ensures the entity exists and returns a deterministic name).
+        evaluation_name = await self.get_evaluation_name(workspace=workspace, candidate=candidate, split=split)
+        if not evaluation_name:
             return  # projection failed (shouldn't happen, but defensive)
 
         agent_attrs = {
@@ -649,13 +649,13 @@ async def persist_evaluation(
                 continue
             try:
                 await self._persist_trial(
-                    trial, workspace=workspace, experiment_id=experiment_id, agent_attrs=agent_attrs
+                    trial, workspace=workspace, evaluation_name=evaluation_name, agent_attrs=agent_attrs
                 )
             except Exception as exc:  # noqa: BLE001 - Intake persistence is best-effort
                 logger.warning(f"[INTAKE] persist_evaluation failed for trial {trial.id}: {exc}")
 
     async def _persist_trial(
-        self, trial: TrialResult, *, workspace: str, experiment_id: str, agent_attrs: dict[str, str]
+        self, trial: TrialResult, *, workspace: str, evaluation_name: str, agent_attrs: dict[str, str]
     ) -> None:
         assert trial.trace is not None
         assert self.client is not None
@@ -664,7 +664,7 @@ async def _persist_trial(
             trace_id = uri.removeprefix("intake://traces/")
             trace = await self._retrieve_trace_with_retry(trace_id, workspace=workspace)
             ctx = getattr(trace, "evaluation_context", None)
-            if ctx is None or getattr(ctx, "evaluation_id", None) != experiment_id:
+            if ctx is None or getattr(ctx, "evaluation_id", None) != evaluation_name:
                 rows: list[dict] = []
                 async for span in self.client.intake.spans.list(
                     workspace=workspace,
@@ -674,7 +674,7 @@ async def _persist_trial(
                 ):
                     rows.append(span.model_dump(mode="json", exclude_none=True))
                 attrs = {
-                    "nemo.experiment.id": experiment_id,
+                    "nemo.evaluation.name": evaluation_name,
                     "nemo.test_case.id": trial.task_id,
                     "nemo.trial.id": trial.id,
                     **agent_attrs,
@@ -702,7 +702,7 @@ async def _persist_trial(
                     self.client,
                     workspace,
                     trial.trace,
-                    experiment_id=experiment_id,
+                    evaluation_name=evaluation_name,
                     task_id=trial.task_id,
                     extra_attrs=agent_attrs,
                 )
@@ -711,7 +711,7 @@ async def _persist_trial(
                     self.client,
                     workspace,
                     trial.trace,
-                    experiment_id=experiment_id,
+                    evaluation_name=evaluation_name,
                     trial_id=trial.id,
                     task_id=trial.task_id,
                     extra_attrs=agent_attrs,
@@ -755,7 +755,7 @@ async def _retrieve_trace_with_retry(
                     delay *= 2  # Exponential backoff
         raise last_exc
 
-    async def get_experiment_id(self, *, workspace: str, candidate: Candidate, split: str) -> str:
+    async def get_evaluation_name(self, *, workspace: str, candidate: Candidate, split: str) -> str:
         if self.client is None:
             return ""
         mirror = self._mirrors.get(workspace)