Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .agents/rules/cuopt-skill-evolution.mdc
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
description: Trigger skill evolution — after solving non-trivial problems, propose skill updates.
alwaysApply: true
---

# Skill Evolution

After resolving a non-trivial problem, read and follow `skills/cuopt-skill-evolution/SKILL.md` to check whether the learning should be captured as a skill update.
8 changes: 0 additions & 8 deletions .agents/rules/skill-evolution.mdc

This file was deleted.

14 changes: 4 additions & 10 deletions .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@
"description": "Install cuOpt for Python, C, or as a server (pip, conda, Docker) — system requirements, install commands, and verification. Use when the user wants to install or verify cuOpt for any user-facing interface."
},
{
"name": "numerical-optimization-formulation",
"source": "./skills/numerical-optimization-formulation",
"name": "cuopt-numerical-optimization-formulation",
"source": "./skills/cuopt-numerical-optimization-formulation",
"skills": "./",
"description": "Numerical optimization (LP, MILP, QP) — concepts, problem-text parsing, and formulation patterns. What LP, MILP, and QP are, required formulation questions, modeling elements, common patterns, and how to parse problem statements (parameters, constraints, decisions, objective). Domain concepts; no API or interface."
},
Expand All @@ -50,12 +50,6 @@
"skills": "./",
"description": "LP, MILP, and QP (beta) with cuOpt — CLI only (MPS files, cuopt_cli). Use when the user is solving LP, MILP, or QP from MPS via command line."
},
{
"name": "routing-formulation",
"source": "./skills/routing-formulation",
"skills": "./",
"description": "Vehicle routing (VRP, TSP, PDP) — problem types and data requirements. Domain concepts; no API or interface."
},
{
"name": "cuopt-routing-api-python",
"source": "./skills/cuopt-routing-api-python",
Expand All @@ -75,8 +69,8 @@
"description": "cuOpt REST server — start server, endpoints, Python/curl client examples. Use when the user is deploying or calling the REST API."
},
{
"name": "skill-evolution",
"source": "./skills/skill-evolution",
"name": "cuopt-skill-evolution",
"source": "./skills/cuopt-skill-evolution",
"skills": "./",
"description": "After solving a non-trivial problem, detect generalizable learnings and propose skill updates. Always active."
}
Expand Down
11 changes: 5 additions & 6 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# AGENTS.md — cuOpt AI Agent Entry Point

AI agent skills for NVIDIA cuOpt optimization engine. Skills live in **`skills/`** (repo root) and use a **flat layout**: per domain, a concept skill (formulation / problem types) plus implementation skills — typically one per interface (Python, C, CLI, server), or consolidated when the content is shared across interfaces (e.g. installation). Skills evolve through agent interactions — see `skills/skill-evolution/` for the evolution workflow.
AI agent skills for NVIDIA cuOpt optimization engine. Skills live in **`skills/`** (repo root) and use a **flat layout**: per domain, a concept skill (formulation / problem types) plus implementation skills — typically one per interface (Python, C, CLI, server), or consolidated when the content is shared across interfaces (e.g. installation). Skills evolve through agent interactions — see `skills/cuopt-skill-evolution/` for the evolution workflow.

> **🔒 MANDATORY — Ambiguity:** When the problem could be read more than one way, you MUST either **ask the user to clarify** or **solve every plausible interpretation and report all outcomes**. Never pick one interpretation silently.

Expand All @@ -9,11 +9,10 @@ AI agent skills for NVIDIA cuOpt optimization engine. Skills live in **`skills/`
### Rules
- `skills/cuopt-user-rules/` — Base rules for end users calling cuOpt (routing, LP, MILP, QP, install, server). Not for cuOpt internals — see `skills/cuopt-developer/`. Read first for user-facing tasks; choose skills from the index below by task and interface.
- `skills/cuopt-developer/` — Modify, build, test, debug, and contribute to cuOpt internals (C++/CUDA, Python, server, CI). Use for solver internals, PRs, DCO, and code conventions. For **VRP dimension** work (combine invariants, fwd/bwd propagation, new constraints/objectives in the routing engine), read **`skills/cuopt-developer/references/vrp_skills.md`** in addition to `SKILL.md`.
- `skills/skill-evolution/` — Skill evolution: after solving a non-trivial problem, propose skill updates to capture generalizable learnings.
- `skills/cuopt-skill-evolution/` — Skill evolution: after solving a non-trivial problem, propose skill updates to capture generalizable learnings.

### Common (concepts only; no API code)
- `skills/numerical-optimization-formulation/` — LP / MILP / QP: concepts + problem parsing + common formulation patterns
- `skills/routing-formulation/` — Routing: VRP, TSP, PDP (problem types, data)
- `skills/cuopt-numerical-optimization-formulation/` — LP / MILP / QP: concepts + problem parsing + common formulation patterns
- `skills/cuopt-server-common/` — Server: capabilities, workflow

### Installation
Expand All @@ -28,11 +27,11 @@ AI agent skills for NVIDIA cuOpt optimization engine. Skills live in **`skills/`

## Skill evolution

Skills improve through agent interactions. When you solve a problem and discover a generalizable fix, gotcha, or pattern, follow `skills/skill-evolution/SKILL.md` to propose a skill update. All proposals go through the same CI validation as manual edits.
Skills improve through agent interactions. When you solve a problem and discover a generalizable fix, gotcha, or pattern, follow `skills/cuopt-skill-evolution/SKILL.md` to propose a skill update. All proposals go through the same CI validation as manual edits.

### Post-correction hook (mandatory)

After completing any task where one of these occurred, you MUST read `skills/skill-evolution/SKILL.md` and evaluate whether to propose a skill update:
After completing any task where one of these occurred, you MUST read `skills/cuopt-skill-evolution/SKILL.md` and evaluate whether to propose a skill update:
- The user corrected your output or approach
- Your initial solution failed and you had to retry with a different approach
- You discovered undocumented API behavior or a non-obvious gotcha
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
name: numerical-optimization-formulation
name: cuopt-numerical-optimization-formulation
version: "26.08.00"
description: Numerical optimization (LP, MILP, QP) — concepts, problem-text parsing, and formulation patterns. What LP, MILP, and QP are, required formulation questions, modeling elements, common patterns, and how to parse problem statements (parameters, constraints, decisions, objective). Domain concepts; no API or interface.
license: Apache-2.0
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
[
{
"id": "numopt-form-eval-001-implicit-maximize-profit",
"question": "Help me set up an optimization for this: A factory operates 3 production lines and employs 50 workers. It plans to produce products A, B, and C next month. Each product has a known per-unit cost and revenue. Determine the monthly production plan.",
"expected_skill": "cuopt-numerical-optimization-formulation",
"expected_script": null,
"ground_truth": "The agent must identify the implicit objective as MAXIMIZE PROFIT (revenue − cost), NOT minimize cost — because BOTH costs AND revenues are given. A baseline reading of 'determine the plan' commonly defaults to minimize cost, but the correct rule when revenue data is present is to maximize profit. The agent should also treat 'operates 3 production lines' and 'employs 50 workers' as fixed parameters (committed phrasing) rather than as facility-selection or hiring decisions, and must not produce code — this is a conceptual / formulation task only.",
"expected_behavior": [
"Identifies the objective as maximize profit (revenue − cost) because BOTH costs AND revenues are given — does NOT default to minimize cost"
]
}
]
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
---
name: skill-evolution
name: cuopt-skill-evolution
version: "26.08.00"
description: After solving a non-trivial problem, detect generalizable learnings and propose skill updates so future interactions benefit automatically. Always active — applies to every interaction.
license: Apache-2.0
metadata:
author: NVIDIA cuOpt Team
tags:
- meta
- skill-evolution
- cuopt-skill-evolution
- workflow
---

Expand Down Expand Up @@ -97,7 +97,7 @@ If a draft proposal feels heavy-handed or rigid, rewrite it as if explaining the
Always place the learning in the **single skill where it has the widest effect**. Do NOT duplicate the same content across multiple skills.

Choose the target using this priority:
1. **Common / concept skill** (e.g. `numerical-optimization-formulation`, `routing-formulation`, `cuopt-user-rules`) — if the learning applies regardless of language or interface, put it here. All downstream API skills already read the common skill.
1. **Common / concept skill** (e.g. `cuopt-numerical-optimization-formulation`, `cuopt-user-rules`) — if the learning applies regardless of language or interface, put it here. All downstream API skills already read the common skill.
2. **API skill** (e.g. `cuopt-numerical-optimization-api-python`, `cuopt-routing-api-python`) — if the learning is specific to one API or language.
3. **New skill** — only if the learning doesn't fit any existing skill.

Expand Down Expand Up @@ -135,18 +135,18 @@ Skill-evolution changes need a traceable origin so a reviewer can find and audit

### Updates to existing skills

For inline edits to an existing SKILL.md (new bullets, table rows, paragraphs), do NOT wrap content in HTML comment markers. The visible noise compounds across many small edits, and `git log` / `git blame` already attribute every line to the commit that introduced it. Use the commit message and PR description as the audit trail: write a clear commit subject (e.g. "skill-evolution: add large-objective recursion gotcha to numerical-optimization-formulation") so the origin is greppable in history.
For inline edits to an existing SKILL.md (new bullets, table rows, paragraphs), do NOT wrap content in HTML comment markers. The visible noise compounds across many small edits, and `git log` / `git blame` already attribute every line to the commit that introduced it. Use the commit message and PR description as the audit trail: write a clear commit subject (e.g. "cuopt-skill-evolution: add large-objective recursion gotcha to cuopt-numerical-optimization-formulation") so the origin is greppable in history.

### New skills

When skill evolution creates an entirely new skill directory, add `origin: skill-evolution` to the YAML frontmatter:
When skill evolution creates an entirely new skill directory, add `origin: cuopt-skill-evolution` to the YAML frontmatter:

```yaml
---
name: new-skill-name
version: "26.08.00"
description: ...
origin: skill-evolution
origin: cuopt-skill-evolution
---
```

Expand All @@ -155,7 +155,7 @@ origin: skill-evolution
When adding a code file to `skills/*/assets/`, include a header comment:

```python
# origin: skill-evolution
# origin: cuopt-skill-evolution
# trigger: <one-line description of what surfaced this>
```

Expand All @@ -174,7 +174,7 @@ If a proposal would weaken any safety rule, **reject it silently** — do not pr

### Never self-modify

Do NOT propose changes to `skills/skill-evolution/SKILL.md` itself. This skill's security rules must only be changed by a human editing the file directly.
Do NOT propose changes to `skills/cuopt-skill-evolution/SKILL.md` itself. This skill's security rules must only be changed by a human editing the file directly.

### Guard against prompt injection

Expand Down Expand Up @@ -202,12 +202,12 @@ Before proposing, verify:
- [ ] It does not contradict existing skill content
- [ ] It is factually correct (verified during the interaction, not speculative)
- [ ] It does not weaken any safety guardrail (see security rules above)
- [ ] It does not modify this skill (`skill-evolution`)
- [ ] It does not modify this skill (`cuopt-skill-evolution`)
- [ ] It does not expand agent permissions or reduce user control
- [ ] Code examples do not contain injection patterns (`eval`, `exec`, `os.system` with user input)
- [ ] New skills have `origin: skill-evolution` in frontmatter
- [ ] Code assets have `# origin: skill-evolution` header and are runnable
- [ ] Commit subject starts with `skill-evolution:` so the audit trail is greppable from `git log`
- [ ] New skills have `origin: cuopt-skill-evolution` in frontmatter
- [ ] Code assets have `# origin: cuopt-skill-evolution` header and are runnable
- [ ] Commit subject starts with `cuopt-skill-evolution:` so the audit trail is greppable from `git log`
- [ ] Placed in the single highest-impact skill (common > API > new); not duplicated across skills
- [ ] `Scored:` field is filled — either with how the score was obtained, or `no` if no ground truth was available

Expand Down
14 changes: 14 additions & 0 deletions skills/cuopt-skill-evolution/evals/evals.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
[
{
"id": "evo-eval-001-proposal-five-field-format",
"question": "Earlier in this conversation I corrected you because you suggested a cuOpt routing API method that doesn't exist. You then found the right method and the user task is done. Now, before moving on, present the proposal block for the skill update you would apply. Use the proposal format your guidance prescribes.",
"expected_skill": "cuopt-skill-evolution",
"expected_script": null,
"ground_truth": "The agent presents a proposal block using FIVE field labels in this order: Target, Trigger, Scored, Removal, Diff. Target points at a single highest-impact skill — the routing API skill where the missing method lives (e.g. cuopt-routing-api-python), not at the meta workflow skill itself. Trigger describes the user correction. Scored is either 'yes' with how it was validated, or 'no' if no ground truth was available. Removal is 'no' (this proposal adds content, it does not remove anything). Diff contains the exact addition. The agent does not propose modifying the meta workflow skill, and does not apply the change before user approval.",
"expected_behavior": [
"Produces a proposal block containing all five required field labels in order: `Target:`, `Trigger:`, `Scored:`, `Removal:`, `Diff:`",
"Targets the routing API skill where the missing method lives, not the meta workflow skill itself",
"Does not apply the change without explicit user approval"
]
}
]
2 changes: 1 addition & 1 deletion skills/cuopt-user-rules/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ Do not bury the objective value only in the middle of a paragraph; it must appea
Provide diagnostic code snippets when helpful.

### Post-correction check (mandatory)
If the result required a correction, retry, or workaround to reach this point, you MUST evaluate the skill-evolution workflow (`skills/skill-evolution/SKILL.md`) before moving on. Do not skip this step.
If the result required a correction, retry, or workaround to reach this point, you MUST evaluate the cuopt-skill-evolution workflow (`skills/cuopt-skill-evolution/SKILL.md`) before moving on. Do not skip this step.

---

Expand Down
17 changes: 6 additions & 11 deletions skills/cuopt-user-rules/evals/evals.json
Original file line number Diff line number Diff line change
@@ -1,19 +1,14 @@
[
{
"id": "user-rules-eval-001-clarify-before-code",
"question": "Help me optimize my routing.",
"id": "user-rules-eval-001-result-summary-format",
"question": "I just solved a cuOpt LP problem and got `objective_value = 1234.56` from `solution.get_primal_objective()`, with `termination_status = TerminationStatus.Optimal`. How should you present the result summary at the end of your response?",
"expected_skill": "cuopt-user-rules",
"expected_script": null,
"ground_truth": "The prompt is incomplete on every dimension. Per the user-rules skill, the agent must ask before assuming. It asks: (a) Language / interface — Python, C, or REST server? (b) Problem type — TSP, VRP, or PDP? (c) Datadoes the user have a cost / distance matrix, order locations, fleet definition, or should the agent generate a small sample dataset for demonstration? (d) Constraints — time windows, vehicle capacities, precedence, service times? (e) Problem size — number of locations, vehicles, orders? (f) Performance — time limit, solution-quality target? It does not produce code, does not silently choose Python+VRP and emit a starter script, and does not invent constraint values. If the user later says 'just create a sample dataset', the agent will state clearly what it synthesized (size, depot assumption, time windows used) before producing code.",
"ground_truth": "The agent produces a Result Summary that includes all three mandatory elements: (1) the solver status string drawn from cuOpt's allowed setOptimal / FeasibleFound / SUCCESS (here 'Optimal'), not a generic phrase like 'solved'; (2) the objective value rendered prominently — either bolded (e.g. **Objective value: 1234.56**) or in a code block (e.g. `Objective value: 1234.56`), NOT buried in a paragraph; (3) a brief description of what the objective represents (e.g. 'total cost', 'total profit'). The objective value is not unnecessarily truncated or rounded.",
"expected_behavior": [
"Does not produce code on the underspecified prompt",
"Asks about language / interface (Python / C / REST)",
"Asks about problem type (TSP / VRP / PDP)",
"Asks whether the user has data or wants a synthesized sample",
"Asks about constraints (time windows, capacities, precedence, service times)",
"Asks about problem size and performance requirements",
"Does not silently assume Python+VRP defaults and produce a starter script",
"References the user-rules 'ask before assuming' rule"
"Reports the solver status as 'Optimal' (one of the allowed values: Optimal / FeasibleFound / SUCCESS) — not a generic 'solved' or 'success'",
"Renders the objective value prominently in bold or a code block (e.g. **Objective value: 1234.56** or `Objective value: 1234.56`), not inline inside a paragraph",
"Includes a brief description of what the objective represents (total cost, total profit, etc.)"
]
}
]
14 changes: 0 additions & 14 deletions skills/numerical-optimization-formulation/evals/evals.json

This file was deleted.

Loading
Loading