feat(skills): add aiq-customize-prompts-models and aiq-maintain-ci by zhongxuanwang-nv · Pull Request #282 · NVIDIA-AI-Blueprints/aiq

zhongxuanwang-nv · 2026-06-23T21:19:19Z

Overview

Adds two maintainer skills under .agents/skills/, moving them out of the
DEVSKILLS-8 "later skill backlog" (epic AIQ-3362) now that both surfaces are
active 2.2 release work:

aiq-customize-prompts-models — Jinja2 prompt-template editing
(src/aiq_agent/agents/*/prompts/*.j2, load_prompt/render_prompt_template)
and per-agent model selection (the llms section + role fields like
orchestrator_llm/researcher_llm/planner_llm/writer_llm, bound via
LLMProvider/LLMRole). Prompt/model customization became release work in enhancement: make deep researcher customizable, flexible and less token expensive #267.
aiq-maintain-ci — the GitHub Actions workflows, pre-commit hooks,
CODEOWNERS/CodeRabbit/copy-pr-bot governance, and the .github/skill-eval harness.

Each skill is a SKILL.md + two references/ files + a .claude/skills
compatibility symlink, following the existing maintainer-skill conventions and
TEMPLATE.md. Both are now enumerated in the .agents/skills/README.md and
docs/source/integration/agent-skills.md skill tables (all six maintainer skills).
Docs-only; no runtime code. Rebased on develop after #281 merged.

The middle commit addresses code-review feedback, each item verified against the
repo: corrected the pre-commit push-stage behavior (pytest/helm-lint are
stages: [push], so pre-commit run --all-files skips them; CI runs them as
dedicated jobs), the deep-researcher vs clarifier LLM role binding (added a
field→LLMRole table; the deep research agent has no generic llm field), the
ui.yml job ids, the skill-eval stages (detect-changes → generate-datasets → harbor-eval), the "adding a new template needs load_prompt wiring" caveat, and
start_cli.sh --config_file in the validation smokes.

Validation

$ uv run python scripts/validate_skills.py .agents/skills
Skill validation passed: 8 skill(s) OK.

$ uv run pytest tests/test_agent_skills.py -q
4 passed

$ uv run pre-commit run --files <new skill markdown + doc tables>
Detect secrets ......... Passed
Validate agent skills .. Passed
Markdown Link Check .... Passed

I ran the relevant local checks or explained why they are not applicable.
I added or updated tests for behavior changes. (N/A — skill docs, not runtime code; validate_skills.py + tests/test_agent_skills.py cover them and pass.)
I updated documentation for user-facing or contributor-facing changes.
I confirmed this PR does not include secrets, credentials, or internal-only data.
I certify this contribution under the DCO and signed my commits with git commit -s.

Where should reviewers start?

.agents/skills/aiq-customize-prompts-models/SKILL.md and
.agents/skills/aiq-maintain-ci/SKILL.md, then their references/. The
model-selection field→LLMRole table mirrors deep_researcher/register.py, and
the CI reference reflects the real .pre-commit-config.yaml stages and ci.yml jobs.

Related Issues

Relates to: AIQ-3362 / DEVSKILLS-8 (builds the two skills whose gates have opened).

🤖 Generated with Claude Code

Summary by CodeRabbit

Documentation
- Added new skill guides for aiq-customize-prompts-models, including prompt template editing, model selection/config wiring, and related validation guidance
- Added new skill guides for aiq-maintain-ci, covering CI/governance maintenance with workflow and evaluation-harness references
- Included CLI and checklist-style validation steps and “common mistakes” sections across the new guides
- Updated agent skills integration docs and Claude Code repo-local symlink instructions for both new skills

copy-pr-bot · 2026-06-23T21:19:24Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

coderabbitai · 2026-06-23T21:19:28Z

Walkthrough

Adds two new agent skill documentation trees: aiq-customize-prompts-models and aiq-maintain-ci, each with reference docs and a .claude/skills/ symlink. The skills README and integration docs are updated to list both entries.

Changes

New Agent Skills and Registration

Layer / File(s)	Summary
aiq-customize-prompts-models skill and references `.agents/skills/aiq-customize-prompts-models/SKILL.md`, `.agents/skills/aiq-customize-prompts-models/references/model-selection.md`, `.agents/skills/aiq-customize-prompts-models/references/prompt-templates.md`	Adds complete skill documentation for customizing prompts and models: SKILL.md with metadata/workflow/pitfalls, `model-selection.md` covering `llms:` config wiring and role fallback behavior, and `prompt-templates.md` covering Jinja2 template locations, `load_prompt`/`render_prompt_template` flow, and citation-rule preservation.
aiq-maintain-ci skill and references `.agents/skills/aiq-maintain-ci/SKILL.md`, `.agents/skills/aiq-maintain-ci/references/skill-eval-harness.md`, `.agents/skills/aiq-maintain-ci/references/workflows-and-hooks.md`	Adds complete skill documentation for CI maintenance: SKILL.md with workflow checklist, `uv run pre-commit` validation commands (default vs. `--hook-stage push`), common mistakes, and related skills; `skill-eval-harness.md` covering `skills-eval.yml` stages and safe-change rules; `workflows-and-hooks.md` covering copy-pr-bot mirror flow and hook stage differences.
Symlinks and doc table registration `.claude/skills/aiq-customize-prompts-models`, `.claude/skills/aiq-maintain-ci`, `.agents/skills/README.md`, `docs/source/integration/agent-skills.md`	Creates Claude symlinks for both skills pointing into `.agents/skills/`; adds both to the maintainer-skills Examples row in the skills README and to the integration doc skills table and Claude symlink snippet.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~5 minutes

Possibly related PRs

NVIDIA-AI-Blueprints/aiq#281: Also updates .agents/skills/README.md and docs/source/integration/agent-skills.md by adding new maintainer-skill example entries to the same tables modified here.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title uses valid Conventional Commits syntax and accurately summarizes the two new maintainer skills.
Description check	✅ Passed	The description matches the required template and includes overview, validation, reviewer start points, and related issues.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

zhongxuanwang-nv

Withdrawn.

zhongxuanwang-nv

Withdrawn.

Address PR NVIDIA-AI-Blueprints#282 review feedback; every change verified against the repo. aiq-maintain-ci: - pytest and helm-lint are pre-commit `stages: [push]`, so `pre-commit run --all-files` does NOT run them (the prior "can be heavier than expected" note was backwards). Document `--hook-stage push` and that CI runs them as the dedicated test/helm-lint jobs; note the pre-commit job's SKIP= set. - Correct ui.yml job ids (install/lint/type-check/unit-test/build); name the skills-eval stages (detect-changes -> generate-datasets, which is creds-free -> harbor-eval); drop the tangential aiq-add-tool from Related Skills. aiq-customize-prompts-models: - The LLMProvider.configure(LLMRole.X) role binding is the deep-researcher pattern (add a field->role table: ORCHESTRATOR/ROUTER/RESEARCHER/PLANNER/ REPORT_WRITER); the clarifier passes planner_llm to its constructor instead. Unset deep-research roles fall back to orchestrator_llm (no generic `llm` field). Add source_router_llm to the example. - Adding a NEW template needs a one-line load_prompt wiring in the agent (prompts.md Step 3); soften "without changing agent code". Note prompts.md does not document every template's variables (source_router/writer/ source_registry) -- the .j2 files are authoritative. both: - Validation smoke must pass `--config_file <your config>` (a bare start_cli.sh runs the fixed default config); scope pytest to the agent's test dir. Validation: validate_skills.py (8 OK), pytest tests/test_agent_skills.py (4 passed), pre-commit (detect-secrets, validate-skills, markdown-link-check) on the changed files all pass. Signed-off-by: Zhongxuan Wang <daniewang@nvidia.com>

…intainer skills DEVSKILLS-8 (under epic AIQ-3362) tracks later maintainer skills until their surfaces stabilize or become active release work. Two of those gates have now opened, so these skills move from the backlog into the set: - aiq-customize-prompts-models: prompt/model customization became active release work in 2.2 (per-agent LLM role fields orchestrator_llm/source_router_llm/ researcher_llm/planner_llm/writer_llm via NVIDIA-AI-Blueprints#267, plus the documented Jinja2 prompt templates and swapping-models guide). Routes editing src/aiq_agent/agents/*/prompts/*.j2 and assigning LLMs per role in the llms section and agent config. - aiq-maintain-ci: CI/governance became active release work (4 workflows, contributor governance setup, the skill-eval regression gate, copy-pr-bot mirroring, and an expanded pre-commit hook set). Routes changes to .github/workflows, .pre-commit-config.yaml, CODEOWNERS, .coderabbit.yaml, and the .github/skill-eval harness. Each skill is a SKILL.md plus two references and a .claude/skills compatibility symlink, following the existing maintainer-skill conventions. The other two DEVSKILLS-8 skills stay deferred: aiq-auth-data-source-integration (protected- source UX/API deferred to 2.2/2.3 per PR NVIDIA-AI-Blueprints#212) and aiq-ui-change (2.2 UI auth controls not yet settled). Doc example listings (agent-skills.md table, README) that enumerate maintainer skills are reworked in the separate PR NVIDIA-AI-Blueprints#281; kept disjoint here. Validation: scripts/validate_skills.py (8 skills OK), pytest tests/test_agent_skills.py (4 passed), and pre-commit (detect-secrets, validate-skills, markdown-link-check) on the new files all pass. Signed-off-by: Zhongxuan Wang <daniewang@nvidia.com>

Address PR NVIDIA-AI-Blueprints#282 review feedback; every change verified against the repo. aiq-maintain-ci: - pytest and helm-lint are pre-commit `stages: [push]`, so `pre-commit run --all-files` does NOT run them (the prior "can be heavier than expected" note was backwards). Document `--hook-stage push` and that CI runs them as the dedicated test/helm-lint jobs; note the pre-commit job's SKIP= set. - Correct ui.yml job ids (install/lint/type-check/unit-test/build); name the skills-eval stages (detect-changes -> generate-datasets, which is creds-free -> harbor-eval); drop the tangential aiq-add-tool from Related Skills. aiq-customize-prompts-models: - The LLMProvider.configure(LLMRole.X) role binding is the deep-researcher pattern (add a field->role table: ORCHESTRATOR/ROUTER/RESEARCHER/PLANNER/ REPORT_WRITER); the clarifier passes planner_llm to its constructor instead. Unset deep-research roles fall back to orchestrator_llm (no generic `llm` field). Add source_router_llm to the example. - Adding a NEW template needs a one-line load_prompt wiring in the agent (prompts.md Step 3); soften "without changing agent code". Note prompts.md does not document every template's variables (source_router/writer/ source_registry) -- the .j2 files are authoritative. both: - Validation smoke must pass `--config_file <your config>` (a bare start_cli.sh runs the fixed default config); scope pytest to the agent's test dir. Validation: validate_skills.py (8 OK), pytest tests/test_agent_skills.py (4 passed), pre-commit (detect-secrets, validate-skills, markdown-link-check) on the changed files all pass. Signed-off-by: Zhongxuan Wang <daniewang@nvidia.com>

Now that NVIDIA-AI-Blueprints#281 has merged, enumerate aiq-customize-prompts-models and aiq-maintain-ci alongside the other four maintainer skills in the .agents/skills/README.md and docs/source/integration/agent-skills.md examples tables, and add their .claude/skills symlinks to the documented symlink block. Signed-off-by: Zhongxuan Wang <daniewang@nvidia.com>

AjayThorve · 2026-06-24T23:46:43Z

/ok to test fbc6062

AjayThorve · 2026-06-24T23:47:07Z

/nvskills-ci

cdgamarose-nv

Minor comment, otherwise looks good!

Address review feedback on aiq-customize-prompts-models: add guidance that prompt templates must stay task-agnostic and not hard-code specific queries, domains, or source/tool names, since source/domain selection is data-driven via the data_source_registry and source_router.j2. Signed-off-by: Zhongxuan Wang <daniewang@nvidia.com>

coderabbitai

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

.agents/skills/aiq-customize-prompts-models/references/prompt-templates.md (1)

43-54: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Clarify when a template edit is actually code-free.

This reads as if any existing .j2 edit needs no Python change, but adding/removing a Jinja variable still requires updating the caller to pass the new context or render_prompt_template() will fail under StrictUndefined.

Suggested wording

-4. Editing an existing `.j2` needs **no** code change.
+4. Editing an existing `.j2` needs no code change if you only change text or reorder existing variables.
+   If you add or remove a template variable, update the agent's Python to pass the same context.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.agents/skills/aiq-customize-prompts-models/references/prompt-templates.md
around lines 43 - 54, Clarify the “editing an existing .j2 needs no code change”
guidance in prompt-templates.md so it only applies when the template’s variable
contract stays the same. Mention that if an existing template adds, removes, or
renames a Jinja variable, the corresponding Python caller must be updated to
pass the new context before render_prompt_template is used, otherwise
StrictUndefined will fail. Use the existing “Adding a new template” section and
references to load_prompt and render_prompt_template to make the distinction
explicit.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In @.agents/skills/aiq-customize-prompts-models/references/prompt-templates.md:
- Around line 43-54: Clarify the “editing an existing .j2 needs no code change”
guidance in prompt-templates.md so it only applies when the template’s variable
contract stays the same. Mention that if an existing template adds, removes, or
renames a Jinja variable, the corresponding Python caller must be updated to
pass the new context before render_prompt_template is used, otherwise
StrictUndefined will fail. Use the existing “Adding a new template” section and
references to load_prompt and render_prompt_template to make the distinction
explicit.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: fcffa599-7fb5-4571-af07-1d526e248800

📥 Commits

Reviewing files that changed from the base of the PR and between fbc6062 and ba03b1b.

📒 Files selected for processing (2)

.agents/skills/aiq-customize-prompts-models/SKILL.md
.agents/skills/aiq-customize-prompts-models/references/prompt-templates.md

📜 Review details

🧰 Additional context used

📓 Path-based instructions (1)

{skills/**,.agents/skills/**,.claude/skills/**,.github/skill-eval/**}

⚙️ CodeRabbit configuration file

{skills/**,.agents/skills/**,.claude/skills/**,.github/skill-eval/**}: Review Agent Skill and skill-eval changes for valid skill metadata, deterministic eval specs, safe handling of
credentials, and clear generated-output boundaries. Do not flag SKILL.md files for missing SPDX headers when the
entrypoint intentionally starts with YAML frontmatter.

Files:

.agents/skills/aiq-customize-prompts-models/references/prompt-templates.md
.agents/skills/aiq-customize-prompts-models/SKILL.md

🔇 Additional comments (1)

.agents/skills/aiq-customize-prompts-models/SKILL.md (1)

13-119: LGTM!

cdgamarose-nv · 2026-06-25T22:52:45Z

/ok to test ba03b1b

cdgamarose-nv · 2026-06-25T23:02:54Z

/merge

zhongxuanwang-nv commented Jun 24, 2026

View reviewed changes

zhongxuanwang-nv added 3 commits June 23, 2026 18:04

zhongxuanwang-nv force-pushed the feat/maintainer-skills-prompts-and-ci branch from cc4c1a7 to fbc6062 Compare June 24, 2026 01:06

zhongxuanwang-nv marked this pull request as ready for review June 24, 2026 01:09

zhongxuanwang-nv changed the title ~~feat(skills): add aiq-customize-prompts-models and aiq-maintain-ci maintainer skills~~ feat(skills): add aiq-customize-prompts-models and aiq-maintain-ci Jun 24, 2026

cdgamarose-nv reviewed Jun 25, 2026

View reviewed changes

Comment thread .agents/skills/aiq-customize-prompts-models/SKILL.md

cdgamarose-nv approved these changes Jun 25, 2026

View reviewed changes

cdgamarose-nv and others added 2 commits June 25, 2026 11:54

Merge branch 'develop' into feat/maintainer-skills-prompts-and-ci

9ab1e88

coderabbitai Bot reviewed Jun 25, 2026

View reviewed changes

cdgamarose-nv merged commit 6fa5390 into NVIDIA-AI-Blueprints:develop Jun 25, 2026
10 checks passed

AjayThorve added this to the v2.2 milestone Jul 7, 2026

This was referenced Jul 7, 2026

refactor(clarifier): remove plan approval; keep context + output-type clarification #303

Merged

docs: document AI-Q 2.2 candidate scope #322

Merged

feat(mcp): add public AI-Q MCP server #319

Merged

coderabbitai Bot mentioned this pull request Jul 14, 2026

feat(skills): add aiq-configure-workflow skill for creating custom config workflows #334

Merged

5 tasks

Navigation Menu

Uh oh!

feat(skills): add aiq-customize-prompts-models and aiq-maintain-ci - #282

feat(skills): add aiq-customize-prompts-models and aiq-maintain-ci#282
cdgamarose-nv merged 5 commits into
NVIDIA-AI-Blueprints:developfrom
zhongxuanwang-nv:feat/maintainer-skills-prompts-and-ci

zhongxuanwang-nv commented Jun 23, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

copy-pr-bot Bot commented Jun 23, 2026

Uh oh!

coderabbitai Bot commented Jun 23, 2026 •

edited

Loading

Uh oh!

zhongxuanwang-nv left a comment •

edited

Loading

Uh oh!

zhongxuanwang-nv left a comment •

edited

Loading

Uh oh!

AjayThorve commented Jun 24, 2026

Uh oh!

AjayThorve commented Jun 24, 2026

Uh oh!

Uh oh!

cdgamarose-nv left a comment

Uh oh!

coderabbitai Bot left a comment

Uh oh!

cdgamarose-nv commented Jun 25, 2026

Uh oh!

cdgamarose-nv commented Jun 25, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

zhongxuanwang-nv commented Jun 23, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Validation

Where should reviewers start?

Related Issues

Summary by CodeRabbit

Uh oh!

copy-pr-bot Bot commented Jun 23, 2026

Uh oh!

coderabbitai Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Possibly related PRs

Uh oh!

zhongxuanwang-nv left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zhongxuanwang-nv left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AjayThorve commented Jun 24, 2026

Uh oh!

AjayThorve commented Jun 24, 2026

Uh oh!

Uh oh!

cdgamarose-nv left a comment

Choose a reason for hiding this comment

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cdgamarose-nv commented Jun 25, 2026

Uh oh!

cdgamarose-nv commented Jun 25, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

zhongxuanwang-nv commented Jun 23, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jun 23, 2026 •

edited

Loading

zhongxuanwang-nv left a comment •

edited

Loading

zhongxuanwang-nv left a comment •

edited

Loading