Skip to content

chore: skills signing batch 5 #4466

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
miyoungc merged 2 commits into from
May 28, 2026
Merged

chore: skills signing batch 5 #4466

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c077a35
chore: skills signing batch 5
miyoungc May 28, 2026
af2ba88
Attach NVSkills validation signatures
svc-nvskills-signing May 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 3 additions & 27 deletions .agents/skills/nemoclaw-user-agent-skills/evals/evals.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,42 +3,18 @@
"id": "docs-resources-agent-skills-001",
"question": "I'm looking at NemoClaw agent skills. Help me find a skill that can guide installation, policy, inference, or operations so I can delegate the right workflow to my AI coding assistant.",
"expected_skill": "nemoclaw-user-agent-skills",
"ground_truth": "A NemoClaw-specific answer that helps the user find a skill that can guide installation, policy, inference, or operations and gives enough concrete guidance, decision criteria, verification steps, or risk framing to delegate the right workflow to my AI coding assistant.",
"expected_behavior": [
"The output directly addresses the user's situation: looking at NemoClaw agent skills.",
"The AI coding assistant loads the expected_skill and references/agent-skills.md",
"The output helps the user find a skill that can guide installation, policy, inference, or operations with NemoClaw-specific guidance rather than generic advice.",
"The output gives enough concrete guidance, decision criteria, verification steps, or risk framing for the user to delegate the right workflow to the AI coding assistant.",
"The output avoids inventing unsupported NemoClaw behavior.",
"The output follows progressive disclosure: it answers the current request without dumping unrelated details other than the expected_skill and the references/agent-skills.md file."
]
"ground_truth": "A NemoClaw-specific answer that helps the user find a skill that can guide installation, policy, inference, or operations and gives enough concrete guidance, decision criteria, verification steps, or risk framing to delegate the right workflow to my AI coding assistant."

@coderabbitai coderabbitai Bot May 28, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Normalize mixed pronouns in ground_truth.

Line 6 mixes perspectives (the user + my AI coding assistant). Use a single POV (e.g., their AI coding assistant) to keep eval criteria unambiguous.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.agents/skills/nemoclaw-user-agent-skills/evals/evals.json at line 6, The
ground_truth string mixes pronouns/perspectives ("the user" and "my AI coding
assistant"); update the value of the ground_truth key in evals.json to use a
single consistent POV (e.g., change "my AI coding assistant" to "their AI coding
assistant") so the evaluation criterion is unambiguous; locate the ground_truth
property in .agents/skills/nemoclaw-user-agent-skills/evals/evals.json and
normalize the phrasing accordingly.

},
{
"id": "docs-resources-agent-skills-002",
"question": "I'm choosing among multiple NemoClaw skills. Help me understand what each skill is designed to do so I can avoid using a broad assistant when a targeted skill exists.",
"expected_skill": "nemoclaw-user-agent-skills",
"ground_truth": "A NemoClaw-specific answer that helps the user understand what each skill is designed to do and gives enough concrete guidance, decision criteria, verification steps, or risk framing to avoid using a broad assistant when a targeted skill exists.",
"expected_behavior": [
"The output directly addresses the user's situation: choosing among multiple NemoClaw skills.",
"The AI coding assistant loads the expected_skill and references/agent-skills.md",
"The output helps the user understand what each skill is designed to do with NemoClaw-specific guidance rather than generic advice.",
"The output gives enough concrete guidance, decision criteria, verification steps, or risk framing for the user to avoid using a broad assistant when a targeted skill exists.",
"The output avoids inventing unsupported NemoClaw behavior.",
"The output follows progressive disclosure: it answers the current request without dumping unrelated details other than the expected_skill and the references/agent-skills.md file."
]
"ground_truth": "A NemoClaw-specific answer that helps the user understand what each skill is designed to do and gives enough concrete guidance, decision criteria, verification steps, or risk framing to avoid using a broad assistant when a targeted skill exists."
},
{
"id": "docs-resources-agent-skills-003",
"question": "I'm letting an agent follow NemoClaw-specific instructions. Help me see why the skill guidance is trustworthy and scoped so I can use agent assistance without losing operational control.",
"expected_skill": "nemoclaw-user-agent-skills",
"ground_truth": "A NemoClaw-specific answer that helps the user see why the skill guidance is trustworthy and scoped and gives enough concrete guidance, decision criteria, verification steps, or risk framing to use agent assistance without losing operational control.",
"expected_behavior": [
"The output directly addresses the user's situation: letting an agent follow NemoClaw-specific instructions.",
"The AI coding assistant loads the expected_skill and references/agent-skills.md",
"The output helps the user see why the skill guidance is trustworthy and scoped with NemoClaw-specific guidance rather than generic advice.",
"The output gives enough concrete guidance, decision criteria, verification steps, or risk framing for the user to use agent assistance without losing operational control.",
"The output avoids inventing unsupported NemoClaw behavior.",
"The output follows progressive disclosure: it answers the current request without dumping unrelated details other than the expected_skill and the references/agent-skills.md file."
]
"ground_truth": "A NemoClaw-specific answer that helps the user see why the skill guidance is trustworthy and scoped and gives enough concrete guidance, decision criteria, verification steps, or risk framing to use agent assistance without losing operational control."
}
]
Loading