Expose terminal failed-column and seed-row provenance in public results

[andreatnvidia]

Priority Level

High (Major improvement)

Is your feature request related to a problem? Please describe.

Consumers that compose many columns into one Data Designer graph cannot determine, from the public run result, which column terminally failed for each omitted seed row.

Data Designer internally knows the task column, row group, and row index when a task fails. DatasetCreationResults.task_traces can expose those fields when full async tracing is enabled, but that is a heavy per-task diagnostic mechanism and the row position is not a stable caller-provided record identity. On a partially successful run, a consumer can diff input and output IDs to discover that a row disappeared, but cannot attribute that loss to the originating column without parsing logs.

This blocks an integration in NVIDIA-NeMo/Anonymizer#237 and PR #240. Anonymizer's legacy rewrite implementation uses several Data Designer runs, so a missing row can be attributed to a stage such as replacement-map generation, initial rewrite, evaluation, or repair. The combined conditional graph uses one Data Designer run. Without column-level failure provenance, every omitted row can only be reported as failing the coarse rewrite-combined boundary.

The required contract is a stable mapping from each terminally omitted seed row to its originating failed column, plus a safe failure category or reason. It must be available without enabling full task tracing.

Describe the solution you'd like

Expose lightweight, structured terminal row failures through the public result API for create() and preview(). If a run raises instead of returning a result, make the same provenance available on the typed exception.

The exact class and field names are flexible, but consumers need this information for each terminal failure:

@dataclass(frozen=True)
class TerminalRowFailure:
    seed_record_id: str | int
    column: str
    failure_kind: str
    reason: str | None = None

The public API should also provide a way for the caller to designate a seed identity column, for example _anonymizer_record_id, whose value is copied into seed_record_id. The returned identity must be directly joinable to the original input and must not require reconstructing it from (row_group, row_index). If no identity column is configured, Data Designer may expose a documented stable seed-row locator as a fallback.

Required semantics:

• Capture terminal row omissions after retry and salvage behavior has completed.
• Attribute the root column that failed, not downstream columns that were skipped or cancelled because their dependency was missing.
• Do not report skip.when decisions as failures.
• Do not report transient failures that later recovered.
• Work for LLM, custom, expression, and other row-producing column types that can omit a row.
• Remain available when only part of a dataset succeeds and when every row fails.
• Be collected independently of async_trace; do not retain prompts, completions, row contents, or successful task traces.
• Document whether failure records cover the current invocation or the full resumed run.
• Preserve deterministic input ordering where practical.

A result shape such as DatasetCreationResults.failed_records and the equivalent preview result field would satisfy the need. A lazy public artifact referenced by the result would also work if it provides the same typed, current-run contract without requiring log parsing.

Acceptance examples:

1. Given two seed rows with designated IDs, if column evaluate_0 terminally fails for one row, the result contains exactly that seed ID and column="evaluate_0".
2. If downstream repair_0 is not scheduled because evaluate_0 failed, no separate repair_0 failure is reported.
3. If repair_0 is skipped because skip.when is true, no failure is reported.
4. If a retry succeeds, no terminal failure is reported.
5. The same information is available without async_trace=True and for an all-rows-failed typed exception.

Describe alternatives you've considered

• Keep the Anonymizer pipeline split across multiple Data Designer runs. This preserves coarse stage attribution but retains custom orchestration and prevents a single conditional graph from becoming the default.
• Diff input and output record IDs. This identifies omitted rows but not the column that caused the omission.
• Enable async_trace=True and map (row_group, row_index) back to the seed. This retains every task trace, is not appropriate as a scalable production result API, and does not expose the caller's stable record ID.
• Parse warning logs, or consume proposed structured log records from #386. Logs and telemetry are useful diagnostics, but they are not a reliable post-run outcome contract and can be filtered or missed by handlers.
• Preserve failed fields as null as proposed in #362. That changes row-retention semantics; this request only asks Data Designer to report the provenance of its existing terminal row omissions.

Agent Investigation

Investigation of current main found:

• packages/data-designer/src/data_designer/interface/results.py: DatasetCreationResults exposes task_traces, but no terminal failure collection.
• packages/data-designer-engine/src/data_designer/engine/dataset_builders/scheduling/task_model.py: TaskTrace contains column, row_group, row_index, status, and error, but traces are only created when tracing is enabled and do not contain a caller-provided seed identity.
• packages/data-designer-engine/src/data_designer/engine/dataset_builders/async_scheduler.py: the scheduler has the originating Task when recording terminal failures and already distinguishes failures, retries, and skips internally.
• packages/data-designer-engine/src/data_designer/engine/dataset_builders/dataset_builder.py: the builder exposes only the first non-retryable exception for the special zero-record error path, not per-row terminal failure provenance.
• packages/data-designer/src/data_designer/interface/data_designer.py: task traces are copied to DatasetCreationResults, while terminal failure records are not returned.

Existing issues #386 and #362 are adjacent but do not provide a typed result mapping terminally omitted rows to both the originating column and the caller's seed ID.

Additional context

This is not a request for full tracing, model-call telemetry, or preservation of failed row contents. The intended surface is a lightweight correctness and failure-attribution contract for applications that embed Data Designer graphs.

Checklist

• I've reviewed existing issues and the documentation
• This is a design proposal, not a "please build this" request

[andreatnvidia]

After taking another pass at this, including the requirements from NVIDIA-NeMo/Anonymizer#240, I think we can narrow the immediate scope substantially.

Anonymizer already:

• assigns a stable _anonymizer_record_id to every seed row;
• uses ordered seed sampling;
• detects missing record IDs after DataDesigner returns;
• handles output ordering itself;
• observes only final omissions, after retries and salvage have completed.

The missing piece is an authoritative mapping from an omitted seed row to the terminal column that failed. A smaller opt-in API should be enough for the merged Anonymizer pipeline:

• expose terminal failures from create() and preview() as (seed_row_index, column);
• expose the same information on the zero-row generation exception;
• record only actual terminal task failures after retry/salvage exhaustion;
• exclude skip.when, recovered failures, and downstream tasks that did not run;
• scope the information to the current invocation;
• expand batch failures to the affected seed-row indices.

Anonymizer can map seed_row_index to its existing record ID and map the failed column to the corresponding legacy stage such as rewrite-pipeline, rewrite-evaluate-N, or rewrite-repair-N. It can continue using its existing generic reason string.

For this use case, we should not need caller-configured identity columns, a failure-kind/reason taxonomy, resume-history semantics, or full task tracing in the first version. Those can remain follow-up work if other consumers need the broader contract.

Making collection opt-in should leave the successful path effectively unchanged. Memory would be O(number of terminal failures), rather than O(rows × columns) as with full task tracing.

One caveat is global early shutdown: rows cancelled before executing a task do not have a truthful failed column. Those should remain represented by the workflow-level exception rather than assigning a fabricated per-row stage.