fix(studio): Guardrails version mismatch fix when running tests

[nakolean]

Bugfix

Summary by CodeRabbit

• New Features

  • Added coordinated saving so edited guardrail checks are persisted before running tests.
  • Added clearer error details when batch guardrail runs fail.
  • Improved handling of concurrent updates to preserve changes and retry safely.

• Bug Fixes

  • Prevented unnecessary updates from focus-only interactions.
  • Ensured newly created and edited checks run correctly on the first attempt.
  • Guardrails are now disabled by default in new environments.

• Tests

  • Expanded coverage for saving, execution, conflicts, failures, and retry behavior.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 162df0e3-1b7d-48ed-aa7c-fb9301d1304d

📥 Commits

Reviewing files that changed from the base of the PR and between 73670d7 and 8ee2e04.

📒 Files selected for processing (11)

• services/studio/src/nmp/studio/env_mappings.py
• web/packages/studio/env/.env.dev.local.sample
• web/packages/studio/env/.env.fastapi
• web/packages/studio/src/api/common/utils.ts
• web/packages/studio/src/api/guardrail-checks/guardrailChecks.test.ts
• web/packages/studio/src/api/guardrail-checks/guardrailChecks.ts
• web/packages/studio/src/constants/featureFlags/featureFlags.ts
• web/packages/studio/src/mocks/handlers/guardrails.ts
• web/packages/studio/src/routes/guardrails/GuardrailChecksTab/GuardrailTestCard.tsx
• web/packages/studio/src/routes/guardrails/GuardrailChecksTab/GuardrailTestCasesEditor.tsx
• web/packages/studio/src/routes/guardrails/GuardrailChecksTab/index.test.tsx

---

📝 Walkthrough

Walkthrough

Guardrail defaults are now disabled. Guardrail execution persistence retries version conflicts. Mutable mocks support execution and concurrency tests. The test UI flushes pending edits before running checks.

Changes

Guardrail execution

Layer / File(s)	Summary
Guardrail feature flag defaults services/studio/src/nmp/studio/env_mappings.py, web/packages/studio/env/*, web/packages/studio/src/constants/featureFlags/featureFlags.ts	Added FastAPI flag wiring and changed guardrail defaults to disabled.
Execution persistence and mock state web/packages/studio/src/api/common/utils.ts, web/packages/studio/src/api/guardrail-checks/*, web/packages/studio/src/mocks/handlers/guardrails.ts	Added HTTP 409 detection, retry-on-conflict persistence, mutable mock state, request recording, and guardrail execution responses. Tests cover model resolution, execution, retries, and batch failures.
Coordinated UI save and execution web/packages/studio/src/routes/guardrails/GuardrailChecksTab/*	Test cards now expose asynchronous flushers. Batch execution flushes pending edits before running checks. UI tests cover first-click execution, error details, and redundant-update prevention.

Sequence Diagram(s)

sequenceDiagram
  participant User
  participant GuardrailTestCasesEditor
  participant GuardrailTestCard
  participant guardrailChecks
  participant GuardrailMockHandlers
  User->>GuardrailTestCasesEditor: Run all checks
  GuardrailTestCasesEditor->>GuardrailTestCard: Flush pending edits
  GuardrailTestCard->>guardrailChecks: Save dirty check
  GuardrailTestCasesEditor->>guardrailChecks: Run refreshed checks
  guardrailChecks->>GuardrailMockHandlers: Execute and persist run
  GuardrailMockHandlers-->>guardrailChecks: Result or version conflict
  guardrailChecks-->>GuardrailTestCasesEditor: Check results

Loading

Possibly related PRs

• NVIDIA-NeMo/nemo-platform#829 — Earlier guardrail execution and persistence logic extended by this change.
• NVIDIA-NeMo/nemo-platform#831 — Introduced the guardrail UI components and mock handlers modified here.
• NVIDIA-NeMo/nemo-platform#1043 — Also changes GuardrailTestCasesEditor.tsx and its tests.

Suggested reviewers: aray12, htolentino-nvidia, steramae-nvidia

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly describes the main change: fixing Guardrails version mismatches during Studio test execution.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch nkolean/bugfix-guardrail-config-test-failure

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Suite	Lines Covered	Line Rate	Branch Rate
Unit Tests	30781/39277	78.4%	62.8%
Integration Tests	18078/37229	48.6%	21.1%