feat(studio): Improve prompting on the Studio Code Agent

[htolentino-nvidia]

Summary by CodeRabbit

• Improvements
  • Updated Studio coding agent guidance to use UI picker-based selection for choosing agents, models, datasets/filesets, and evaluation configurations, reducing reliance on plain-text prompts.
  • Strengthened job progress handling to ensure progress is reported for every launched platform job before the final response, without replacing progress cards with text status.
• Tests
  • Updated unit tests to reflect the revised Studio system instructions and job-progress requirements.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 7c6f712c-266e-421b-9362-048fcb7920fe

📥 Commits

Reviewing files that changed from the base of the PR and between b25e9ba and 5861558.

📒 Files selected for processing (1)

• services/studio/src/nmp/studio/coding_agent_mcp_tools.py

🚧 Files skipped from review as they are similar to previous changes (1)

• services/studio/src/nmp/studio/coding_agent_mcp_tools.py

---

📝 Walkthrough

Walkthrough

MCP tool descriptions (select_agent, select_dataset_file, job_progress) are hardened with mandatory-use language. The STUDIO_CODING_AGENT_CONTEXT prompt is restructured to assume the platform is already running and mandate select_* tools for all UI choices, reserving AskUserQuestion for non-select clarifications. The coding_agents system prompt adds strict picker-tool-call rules and requires mcp__nemo_studio__job_progress after every launched job. Unit test assertions are updated to match.

Changes

MCP Tool Mandate Enforcement

Layer / File(s)	Summary
MCP tool description updates services/studio/src/nmp/studio/coding_agent_mcp_tools.py	SELECT_AGENT, SELECT_DATASET_FILE, and JOB_PROGRESS descriptions add mandatory-use language, return shapes, and job-name preconditions.
STUDIO_CODING_AGENT_CONTEXT restructuring services/studio/src/nmp/studio/coding_agent_mcp_tools.py	Consolidated prompt assumes platform is running, mandates select_* tools for all UI choices, restricts AskUserQuestion to non-select clarifications, and requires mcp__nemo_studio__job_progress before final response for every launched job.
coding_agents system prompt enforcement services/studio/src/nmp/studio/coding_agents.py	Replaces generic guidance with strict rules requiring picker tool calls for agent/model/dataset/eval-config and adds required job-progress behavior block prohibiting plain-text substitutes.
Unit test assertion updates services/studio/tests/unit/test_coding_agents.py	Replaces removed prompt lines with assertions for mandatory MCP tool calls and adds assertion for mcp__nemo_studio__job_progress requirement.

Suggested reviewers

• steramae-nvidia

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	Title accurately summarizes the main change: improving prompting guidance for the Studio Code Agent across multiple files.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch studio-code-agent-prompt-improvements/htolentino

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

Suite	Lines Covered	Line Rate	Branch Rate
Unit Tests	19462/25829	75.3%	60.9%
Integration Tests	11371/24601	46.2%	20.2%