Skip to content

skills: add evals/evals.json smoke suite (group A) #1308

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ramakrishnap-nv merged 2 commits into from
May 27, 2026
Merged

skills: add evals/evals.json smoke suite (group A) #1308

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b3f7c8f
skills: add evals/evals.json smoke suite (group A)
ramakrishnap-nv May 27, 2026
cedb505
Attach NVSkills validation signatures
svc-nvskills-signing May 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions skills/cuopt-numerical-optimization-api-cli/evals/evals.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
[
{
"id": "numopt-cli-eval-001-mps-sections-and-cli-command",
"question": "I have an LP problem I want to solve with cuopt_cli from an MPS file, with a 60-second time limit and 1% MIP gap (in case I add integers later). List the MPS sections in required order, and the cuopt_cli command line.",
"expected_skill": "cuopt-numerical-optimization-api-cli",
"expected_script": null,
"ground_truth": "The agent lists the MPS sections in the required order: NAME, ROWS (N row for the objective, L/G/E rows for constraints), COLUMNS (variable-name, row-name, coefficient triples), RHS (right-hand-side values), BOUNDS (optional — LO/UP/FX/BV/LI/UI), ENDATA. For integer variables, integer markers are 'MARKER' 'INTORG' before and 'MARKER' 'INTEND' after the integer columns. The cuopt_cli invocation is: cuopt_cli problem.mps --time-limit 60 --mip-relative-tolerance 0.01. The agent mentions cuopt_cli --help as the canonical source for all flags. Does not invent flags like --max-time or --gap that are not in the skill. Notes that cuopt_cli ships with the cuopt Python package (install via pip or conda first if not present).",
"expected_behavior": [
"Lists MPS sections in required order: NAME, ROWS, COLUMNS, RHS, [BOUNDS], ENDATA",
"Mentions N row for objective and L/G/E for constraint types",
"Mentions integer markers ('MARKER' 'INTORG' / 'INTEND') for integer columns",
"Gives the cuopt_cli command with --time-limit 60 and --mip-relative-tolerance 0.01",
"References cuopt_cli --help as the canonical flag source",
"Does not invent flag names that are not in the skill (e.g. --max-time, --gap)",
"Mentions that cuopt_cli ships with the cuopt Python package"
]
}
]
4 changes: 2 additions & 2 deletions skills/cuopt-numerical-optimization-api-cli/skill-card.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ This skill is ready for commercial/non-commercial use. <br>
### License/Terms of Use: <br>
Apache 2.0 <br>
## Use Case: <br>
Developers and engineers solving linear programming, mixed-integer linear programming, or quadratic programming problems from MPS files via the cuopt_cli command-line interface. <br>
Developers and engineers solving linear programming (LP), mixed-integer linear programming (MILP), or quadratic programming (QP) optimization problems via the cuopt_cli command-line tool using MPS-format input files. <br>

### Deployment Geography for Use: <br>
Global <br>
Expand All @@ -24,7 +24,7 @@ Mitigation: Review and scan skill before deployment. <br>


## Skill Output: <br>
**Output Type(s):** [Shell commands, Configuration instructions] <br>
**Output Type(s):** [Shell commands, Configuration instructions, Code] <br>
**Output Format:** [Markdown with inline bash code blocks] <br>
**Output Parameters:** [1D] <br>
**Other Properties Related to Output:** [None] <br>
Expand Down
2 changes: 1 addition & 1 deletion skills/cuopt-numerical-optimization-api-cli/skill.oms.sig
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{"mediaType":"application/vnd.dev.sigstore.bundle.v0.3+json","verificationMaterial":{"x509CertificateChain":{"certificates":[{"rawBytes":"MIICgzCCAgmgAwIBAgIUKIyS7SxNteQIiWzK1dWj85E6520wCgYIKoZIzj0EAwMwVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwHhcNMjYwNDAxMDAwMDAwWhcNMjgwNDIyMTUzMzA5WjBUMQswCQYDVQQGEwJVUzEbMBkGA1UECgwSTlZJRElBIENvcnBvcmF0aW9uMSgwJgYDVQQDDB9OVklESUEgQWdlbnQgU2tpbGxzIFNpZ25pbmcgMDAxMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEYoRM9bQl/dGlwSRNi6bTpIJUXH8Nv9GciP6LSflJYYMLCc296kpyuTSsk5ddbAWiDcFX3C/ydX3jwc+qCLYP6uHy9XphyLjOQ27Yb2J6rBLVtRBS1mgGco/Gr7fL6ODco4GaMIGXMB0GA1UdDgQWBBRQ/5ZW3nJ6lmo9SVk7I15o7UGmpTAfBgNVHSMEGDAWgBRPGpILxMBBleJSsBGjrMKsby1CgjAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQEAwIHgDA3BggrBgEFBQcBAQQrMCkwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vY3NwLm5kaXMubnZpZGlhLmNvbTAKBggqhkjOPQQDAwNoADBlAjAUygu/GiOCIXrgGr4SmLgeEVDcEitfFUv7ALbvLVGVyMysB3mxmO/uInZfXzWcJZsCMQDxuoxj4ZmO30jhkPIcCxGFCOvnUsnfU3TfGcouYm4M6iRpbKvtVnHPiy4bi6pcKf0="},{"rawBytes":"MIICiDCCAg6gAwIBAgIUZsIuSv9NkpJCNqtYEfCouVv5BzowCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAASI72cR3ctKGg4VWnB3bNja6g1Z2PnOmFEopkPof+QeIcPk9rT+g9MjJnq51EQXL93a7C2GJ9J985G4o2V85VD7wJ1RaXhluHW2rf3y8bQGeAYaKMr5s/hUgn+M3/9WlWejgaAwgZ0wHQYDVR0OBBYEFE8akgvEwEGV4lKwEaOswqxvLUKCMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgEGMDcGCCsGAQUFBwEBBCswKTAnBggrBgEFBQcwAYYbaHR0cDovL29jc3AubmRpcy5udmlkaWEuY29tMAoGCCqGSM49BAMDA2gAMGUCMQCeIMMfAbyzPDacw2MxG+Yt1cikrJX/DVxiGfXuHmkkXn6VgSzE79+lkqDErpVO2gYCMCNEColOyvUvkzZGUEI1hQ3PfMgi3FIo9tHoBKMw4/wGBLFpu/0ubtmbBXM6/UMOEw=="},{"rawBytes":"MIICRTCCAcygAwIBAgIUeJdY3rV86EdvFmG7L8LJBsyQFYkwCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTB2MBAGByqGSM49AgEGBSuBBAAiA2IABAYpiXCDjJ9NT2eSDhyHJVSw1Tbze18cGG2F/578oWvHxg23eQAhNRYdq88i1iOshZSO6C29doKui5Xpmo/7Ctw9Sx4PP2RzOmIuOLCuTdNtKcTRwi4GEsd5BAFvWj42M6NjMGEwHQYDVR0OBBYEFItnoAjjfuCEUvzyvWyI2vOGvwPjMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMAoGCCqGSM49BAMDA2cAMGQCMCwtAjWLaNwgGWNCgdyNoTyvNhqWRECRJV2r3+7w8g0PL6NHLOsbkgE09BH95h8XlgIwTaQmbbUh2ChAJ5TA1wRiVDnCcvbzHlZl2jM2FcwQQZlk19LOAbyGMRixbu2Ww/rj"}]},"tlogEntries":[]},"dsseEnvelope":{"payload":"ewogICJfdHlwZSI6ICJodHRwczovL2luLXRvdG8uaW8vU3RhdGVtZW50L3YxIiwKICAic3ViamVjdCI6IFsKICAgIHsKICAgICAgIm5hbWUiOiAiY3VvcHQtbnVtZXJpY2FsLW9wdGltaXphdGlvbi1hcGktY2xpIiwKICAgICAgImRpZ2VzdCI6IHsKICAgICAgICAic2hhMjU2IjogIjc4NmIxZGUxZjUyZGU3YjU1ZDFhZmE3OWFhZDkzYTZkMGJlM2NmYjRmZWEzMzU4YmFiNDE3NjJkNzBmYWE5MGEiCiAgICAgIH0KICAgIH0KICBdLAogICJwcmVkaWNhdGVUeXBlIjogImh0dHBzOi8vbW9kZWxfc2lnbmluZy9zaWduYXR1cmUvdjEuMCIsCiAgInByZWRpY2F0ZSI6IHsKICAgICJzZXJpYWxpemF0aW9uIjogewogICAgICAibWV0aG9kIjogImZpbGVzIiwKICAgICAgImlnbm9yZV9wYXRocyI6IFsKICAgICAgICAiLmdpdGF0dHJpYnV0ZXMiLAogICAgICAgICIuZ2l0IiwKICAgICAgICAiLmdpdGh1YiIsCiAgICAgICAgIi5naXRpZ25vcmUiCiAgICAgIF0sCiAgICAgICJoYXNoX3R5cGUiOiAic2hhMjU2IiwKICAgICAgImFsbG93X3N5bWxpbmtzIjogZmFsc2UKICAgIH0sCiAgICAicmVzb3VyY2VzIjogWwogICAgICB7CiAgICAgICAgImRpZ2VzdCI6ICI4NjI5YTY0YTY2YzY2NWYwNjRkMWNiNTdhYzEzNDdhZGE5NmMxMmJmZDE4MzAyZDhiNGJlZmYzN2ZkZmVmOGZlIiwKICAgICAgICAibmFtZSI6ICJTS0lMTC5tZCIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiZGlnZXN0IjogImUwYmExMDBlNzc2YWU3MDQ3YjQwMTRkYTQ4OWMzZTRmM2Q3MzA3NTIyNzE5OThkZWJjOWFmMDI5OTUzYWNjNDUiLAogICAgICAgICJuYW1lIjogImFzc2V0cy9SRUFETUUubWQiLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IgogICAgICB9LAogICAgICB7CiAgICAgICAgImRpZ2VzdCI6ICJiMjVhZjUzMTk3YzVkMjBlYzI5NzZiYTc4NmU2ZTZhMjVmMGViNTY4ZDdjNTMyZWVmZWVjMDY3MzZkMTkyNWE4IiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvbHBfcHJvZHVjdGlvbi9SRUFETUUubWQiLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IgogICAgICB9LAogICAgICB7CiAgICAgICAgImRpZ2VzdCI6ICJlNTZhZTJmZWY5OGRmYTZiMzQxNTY2OTQyYWFkNWI3NDY3OTU4ZmNkOTJiNzkzNDAyZjQ5ZDBjN2JmNzYyYzAxIiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvbHBfcHJvZHVjdGlvbi9wcm9kdWN0aW9uLm1wcyIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiZGlnZXN0IjogImJjMTNmZTY4ODRhMzJkOWRhNWIxMTQ3NWNjZjk4ZjY4ZjFmZGFiZTcwNmQ0YjNjMzhjN2JlMDg0MDE0MzViZjUiLAogICAgICAgICJuYW1lIjogImFzc2V0cy9scF9zaW1wbGUvUkVBRE1FLm1kIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIKICAgICAgfSwKICAgICAgewogICAgICAgICJkaWdlc3QiOiAiMzBiM2Y4NzE5MTgxNjBlOWMxYzVlNzYwZTM5ZTllNWE5NzNlNTFhYWFkMDk3OTg3NjVjOGNhNzQxNjQxYmIwNCIsCiAgICAgICAgIm5hbWUiOiAiYXNzZXRzL2xwX3NpbXBsZS9zYW1wbGUubXBzIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIKICAgICAgfSwKICAgICAgewogICAgICAgICJkaWdlc3QiOiAiY2RhNWI3YWZlNjJiNzE2OWExNjA2MThhZDE2MzExOTI0ZDNhMDdmYWRlZGU1ZGI0MWVkYjdkZjMyNWM2M2IwNSIsCiAgICAgICAgIm5hbWUiOiAiYXNzZXRzL21pbHBfZmFjaWxpdHkvUkVBRE1FLm1kIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIKICAgICAgfSwKICAgICAgewogICAgICAgICJkaWdlc3QiOiAiMzY4NzA1Njk3ZGI0NWIzYjNlMTcwOGJkMTM5MWUxNjdkNWE5ZjUzNDg4MTY1MTRkMGE4MDFjMmNlNTc5ZjA4YyIsCiAgICAgICAgIm5hbWUiOiAiYXNzZXRzL21pbHBfZmFjaWxpdHkvZmFjaWxpdHkubXBzIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIKICAgICAgfSwKICAgICAgewogICAgICAgICJkaWdlc3QiOiAiNTJkMDAxNWQzZTI2MjNhYWQ5ZDU5OTM2N2YyZjE3NmQ4NTg0MjM5NTBmNGY0Nzg5YzdlYmEwYWRmZGYzNTBmYiIsCiAgICAgICAgIm5hbWUiOiAic2tpbGwtY2FyZC5tZCIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiCiAgICAgIH0KICAgIF0KICB9Cn0=","payloadType":"application/vnd.in-toto+json","signatures":[{"sig":"MGUCMEDE1yY1Vr2Nv8ue+prjm5wgvaXZQuANqpUJufSwOx0vTQwYV7S4LGD33q2QoayNqAIxANRJq6AHV8D1L18wnaI2B3q/AodNOLNMzg0Q8IgqYZRb7VZVWSKCubaUQmKlGZOLqA==","keyid":""}]}}
{"mediaType":"application/vnd.dev.sigstore.bundle.v0.3+json","verificationMaterial":{"x509CertificateChain":{"certificates":[{"rawBytes":"MIICgzCCAgmgAwIBAgIUKIyS7SxNteQIiWzK1dWj85E6520wCgYIKoZIzj0EAwMwVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwHhcNMjYwNDAxMDAwMDAwWhcNMjgwNDIyMTUzMzA5WjBUMQswCQYDVQQGEwJVUzEbMBkGA1UECgwSTlZJRElBIENvcnBvcmF0aW9uMSgwJgYDVQQDDB9OVklESUEgQWdlbnQgU2tpbGxzIFNpZ25pbmcgMDAxMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEYoRM9bQl/dGlwSRNi6bTpIJUXH8Nv9GciP6LSflJYYMLCc296kpyuTSsk5ddbAWiDcFX3C/ydX3jwc+qCLYP6uHy9XphyLjOQ27Yb2J6rBLVtRBS1mgGco/Gr7fL6ODco4GaMIGXMB0GA1UdDgQWBBRQ/5ZW3nJ6lmo9SVk7I15o7UGmpTAfBgNVHSMEGDAWgBRPGpILxMBBleJSsBGjrMKsby1CgjAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQEAwIHgDA3BggrBgEFBQcBAQQrMCkwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vY3NwLm5kaXMubnZpZGlhLmNvbTAKBggqhkjOPQQDAwNoADBlAjAUygu/GiOCIXrgGr4SmLgeEVDcEitfFUv7ALbvLVGVyMysB3mxmO/uInZfXzWcJZsCMQDxuoxj4ZmO30jhkPIcCxGFCOvnUsnfU3TfGcouYm4M6iRpbKvtVnHPiy4bi6pcKf0="},{"rawBytes":"MIICiDCCAg6gAwIBAgIUZsIuSv9NkpJCNqtYEfCouVv5BzowCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAASI72cR3ctKGg4VWnB3bNja6g1Z2PnOmFEopkPof+QeIcPk9rT+g9MjJnq51EQXL93a7C2GJ9J985G4o2V85VD7wJ1RaXhluHW2rf3y8bQGeAYaKMr5s/hUgn+M3/9WlWejgaAwgZ0wHQYDVR0OBBYEFE8akgvEwEGV4lKwEaOswqxvLUKCMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgEGMDcGCCsGAQUFBwEBBCswKTAnBggrBgEFBQcwAYYbaHR0cDovL29jc3AubmRpcy5udmlkaWEuY29tMAoGCCqGSM49BAMDA2gAMGUCMQCeIMMfAbyzPDacw2MxG+Yt1cikrJX/DVxiGfXuHmkkXn6VgSzE79+lkqDErpVO2gYCMCNEColOyvUvkzZGUEI1hQ3PfMgi3FIo9tHoBKMw4/wGBLFpu/0ubtmbBXM6/UMOEw=="},{"rawBytes":"MIICRTCCAcygAwIBAgIUeJdY3rV86EdvFmG7L8LJBsyQFYkwCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTB2MBAGByqGSM49AgEGBSuBBAAiA2IABAYpiXCDjJ9NT2eSDhyHJVSw1Tbze18cGG2F/578oWvHxg23eQAhNRYdq88i1iOshZSO6C29doKui5Xpmo/7Ctw9Sx4PP2RzOmIuOLCuTdNtKcTRwi4GEsd5BAFvWj42M6NjMGEwHQYDVR0OBBYEFItnoAjjfuCEUvzyvWyI2vOGvwPjMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMAoGCCqGSM49BAMDA2cAMGQCMCwtAjWLaNwgGWNCgdyNoTyvNhqWRECRJV2r3+7w8g0PL6NHLOsbkgE09BH95h8XlgIwTaQmbbUh2ChAJ5TA1wRiVDnCcvbzHlZl2jM2FcwQQZlk19LOAbyGMRixbu2Ww/rj"}]},"tlogEntries":[]},"dsseEnvelope":{"payload":"ewogICJfdHlwZSI6ICJodHRwczovL2luLXRvdG8uaW8vU3RhdGVtZW50L3YxIiwKICAic3ViamVjdCI6IFsKICAgIHsKICAgICAgIm5hbWUiOiAiY3VvcHQtbnVtZXJpY2FsLW9wdGltaXphdGlvbi1hcGktY2xpIiwKICAgICAgImRpZ2VzdCI6IHsKICAgICAgICAic2hhMjU2IjogImRlNjE1NmZkZDgyOThjMDBkMTYzYWU5OGNiYWQ5ZjcwZGM5MWVlMDk0OGYzNjYzZDMyOGEwZmQ5MDVhY2Q0ZmYiCiAgICAgIH0KICAgIH0KICBdLAogICJwcmVkaWNhdGVUeXBlIjogImh0dHBzOi8vbW9kZWxfc2lnbmluZy9zaWduYXR1cmUvdjEuMCIsCiAgInByZWRpY2F0ZSI6IHsKICAgICJyZXNvdXJjZXMiOiBbCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgIm5hbWUiOiAiU0tJTEwubWQiLAogICAgICAgICJkaWdlc3QiOiAiODYyOWE2NGE2NmM2NjVmMDY0ZDFjYjU3YWMxMzQ3YWRhOTZjMTJiZmQxODMwMmQ4YjRiZWZmMzdmZGZlZjhmZSIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvUkVBRE1FLm1kIiwKICAgICAgICAiZGlnZXN0IjogImUwYmExMDBlNzc2YWU3MDQ3YjQwMTRkYTQ4OWMzZTRmM2Q3MzA3NTIyNzE5OThkZWJjOWFmMDI5OTUzYWNjNDUiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgIm5hbWUiOiAiYXNzZXRzL2xwX3Byb2R1Y3Rpb24vUkVBRE1FLm1kIiwKICAgICAgICAiZGlnZXN0IjogImIyNWFmNTMxOTdjNWQyMGVjMjk3NmJhNzg2ZTZlNmEyNWYwZWI1NjhkN2M1MzJlZWZlZWMwNjczNmQxOTI1YTgiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgIm5hbWUiOiAiYXNzZXRzL2xwX3Byb2R1Y3Rpb24vcHJvZHVjdGlvbi5tcHMiLAogICAgICAgICJkaWdlc3QiOiAiZTU2YWUyZmVmOThkZmE2YjM0MTU2Njk0MmFhZDViNzQ2Nzk1OGZjZDkyYjc5MzQwMmY0OWQwYzdiZjc2MmMwMSIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvbHBfc2ltcGxlL1JFQURNRS5tZCIsCiAgICAgICAgImRpZ2VzdCI6ICJiYzEzZmU2ODg0YTMyZDlkYTViMTE0NzVjY2Y5OGY2OGYxZmRhYmU3MDZkNGIzYzM4YzdiZTA4NDAxNDM1YmY1IgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogImFzc2V0cy9scF9zaW1wbGUvc2FtcGxlLm1wcyIsCiAgICAgICAgImRpZ2VzdCI6ICIzMGIzZjg3MTkxODE2MGU5YzFjNWU3NjBlMzllOWU1YTk3M2U1MWFhYWQwOTc5ODc2NWM4Y2E3NDE2NDFiYjA0IgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogImFzc2V0cy9taWxwX2ZhY2lsaXR5L1JFQURNRS5tZCIsCiAgICAgICAgImRpZ2VzdCI6ICJjZGE1YjdhZmU2MmI3MTY5YTE2MDYxOGFkMTYzMTE5MjRkM2EwN2ZhZGVkZTVkYjQxZWRiN2RmMzI1YzYzYjA1IgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogImFzc2V0cy9taWxwX2ZhY2lsaXR5L2ZhY2lsaXR5Lm1wcyIsCiAgICAgICAgImRpZ2VzdCI6ICIzNjg3MDU2OTdkYjQ1YjNiM2UxNzA4YmQxMzkxZTE2N2Q1YTlmNTM0ODgxNjUxNGQwYTgwMWMyY2U1NzlmMDhjIgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogImV2YWxzL2V2YWxzLmpzb24iLAogICAgICAgICJkaWdlc3QiOiAiNGZlZTgwMzJhNDRhNjQ5MjVhYWY5NTc1OWJlMTk4MGMyZWE5MTZiZDQ1OGY2ZTY4ZDdmYTVhMGYyMjc5ODEwMiIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJza2lsbC1jYXJkLm1kIiwKICAgICAgICAiZGlnZXN0IjogImQ3YmU2MmM5ODFmYzFiMWQxZDA5MjdhMWIzNmI0MzY1MjkwYWRkMzdiMWQxN2QzZDQwYzM3YTI5MGE2ZmQ2YjYiCiAgICAgIH0KICAgIF0sCiAgICAic2VyaWFsaXphdGlvbiI6IHsKICAgICAgImhhc2hfdHlwZSI6ICJzaGEyNTYiLAogICAgICAiYWxsb3dfc3ltbGlua3MiOiBmYWxzZSwKICAgICAgImlnbm9yZV9wYXRocyI6IFsKICAgICAgICAiLmdpdGh1YiIsCiAgICAgICAgIi5naXRhdHRyaWJ1dGVzIiwKICAgICAgICAiLmdpdCIsCiAgICAgICAgIi5naXRpZ25vcmUiCiAgICAgIF0sCiAgICAgICJtZXRob2QiOiAiZmlsZXMiCiAgICB9CiAgfQp9","payloadType":"application/vnd.in-toto+json","signatures":[{"sig":"MGUCMQDpCrSpNItqlo2uEmZHZiuGeMjeu1M0lWyfX27O9I76emoBD++heMrulglfzVpzvUsCMC1O139zjy4gHAO7hesIHvSf0UKX/etWuKdFPpQWCGFfEDVxsZ8Pw5/vVZ18J/9fpw==","keyid":""}]}}
19 changes: 19 additions & 0 deletions skills/cuopt-numerical-optimization-api-python/evals/evals.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
[
{
"id": "numopt-py-eval-001-lp-api-call-sequence",
"question": "I want to solve a small LP (continuous variables only, maximize a linear objective with linear constraints) using the cuOpt Python API. List the API calls in order — name each method, one line per method, no full runnable script.",
"expected_skill": "cuopt-numerical-optimization-api-python",
"expected_script": null,
"ground_truth": "The agent produces an ordered list of API calls without a runnable script. The list, in order: (1) Import Problem, CONTINUOUS, and MAXIMIZE from cuopt.linear_programming.problem, and SolverSettings from cuopt.linear_programming.solver_settings. (2) Construct Problem('name'). (3) For each decision variable, call problem.addVariable(lb=..., vtype=CONTINUOUS, name=...). (4) For each constraint, call problem.addConstraint(<linear expression> <= or >= or == <rhs>, name=...). (5) Call problem.setObjective(<linear expression>, sense=MAXIMIZE). (6) Construct SolverSettings(); call set_parameter('time_limit', ...) for time budget. (7) Call problem.solve(settings). (8) Check problem.Status.name in ['Optimal', 'PrimalFeasible'] (PascalCase status names — case-sensitive). (9) Read problem.ObjValue for the objective, and each variable's .getValue() for its optimal value. The agent uses LP (not MILP / QP) because all variables are continuous and the objective is linear. Mentions that status names are PascalCase (Optimal, not OPTIMAL or optimal) — case sensitivity matters.",
"expected_behavior": [
"Selects LP (not MILP or QP) given continuous variables and a linear objective",
"Lists the API calls in order without producing a full runnable script",
"Names Problem, addVariable (with vtype=CONTINUOUS), addConstraint, setObjective (sense=MAXIMIZE)",
"Names SolverSettings, set_parameter('time_limit', ...), and problem.solve(settings)",
"Names problem.Status.name and the PascalCase status values (Optimal / PrimalFeasible / FeasibleFound)",

@coderabbitai coderabbitai Bot May 27, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Description: Search for Status enum/class definition and all valid status values.

# Search for Status class or enum definition
ast-grep --pattern 'class Status'

# Search for status value references in examples and documentation
rg -n 'Status\.name|Status\.' skills/cuopt-numerical-optimization-api-python/ -g '*.{py,md}' -C2

# Search specifically for FeasibleFound
rg -n 'FeasibleFound|Feasible' skills/cuopt-numerical-optimization-api-python/ -g '*.{py,md}'

Repository: NVIDIA/cuopt

Length of output: 17947

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Show evals.json around the referenced line
echo "---- evals.json (lines 1-40) ----"
nl -ba skills/cuopt-numerical-optimization-api-python/evals/evals.json | sed -n '1,60p'

# Locate likely model.py referenced by the evals and inspect around line 29
echo "---- candidate model.py files under evals/ ----"
find skills/cuopt-numerical-optimization-api-python -maxdepth 3 -type f -name 'model.py' -path '*evals*' -print

echo "---- show any model.py under evals/ (best-effort around line 20-45) ----"
for f in $(find skills/cuopt-numerical-optimization-api-python -maxdepth 3 -type f -name 'model.py' -path '*evals*' -print); do
  echo "==== $f ===="
  nl -ba "$f" | sed -n '15,60p'
done

# Also search for FeasibleFound usage in evals.json region if present
echo "---- FeasibleFound references in evals/ ----"
rg -n 'FeasibleFound' skills/cuopt-numerical-optimization-api-python/evals -S || true

Repository: NVIDIA/cuopt

Length of output: 151

🏁 Script executed:

#!/bin/bash
set -euo pipefail

EVALS_JSON="skills/cuopt-numerical-optimization-api-python/evals/evals.json"

echo "---- evals.json (first 60 lines) ----"
cat -n "$EVALS_JSON" | sed -n '1,60p'

echo "---- evals.json (around line 13) ----"
cat -n "$EVALS_JSON" | sed -n '8,18p'

echo "---- model.py files under evals/ (if any) ----"
find "skills/cuopt-numerical-optimization-api-python/evals" -type f -name "model.py" -print

echo "---- FeasibleFound references in evals/ ----"
rg -n "FeasibleFound" "skills/cuopt-numerical-optimization-api-python/evals" -S || true

echo "---- FeasibleFound references anywhere in skill examples/docs (context) ----"
rg -n "FeasibleFound" "skills/cuopt-numerical-optimization-api-python" -g "*.{py,md,json}" -S | head -n 50

Repository: NVIDIA/cuopt

Length of output: 4774

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "---- repo-wide occurrences of FeasibleFound ----"
rg -n "FeasibleFound" -S . | head -n 50

echo "---- repo-wide occurrences of PrimalFeasible ----"
rg -n "PrimalFeasible" -S . | head -n 50

echo "---- search for Status enum/class definitions ----"
rg -n "enum\s+.*Status|class\s+Status|Status\s*=\s*enum|Status.*(Optimal|PrimalFeasible|FeasibleFound)" -S . | head -n 50

echo "---- search for PascalCase status name mapping (common patterns) ----"
rg -n "\"Optimal\"|\"PrimalFeasible\"|\"FeasibleFound\"|\"PrimalInfeasible\"|\"DualInfeasible\"|\"IterationLimit\"|\"NoTermination\"|\"TimeLimit\"" -S . | head -n 50

Repository: NVIDIA/cuopt

Length of output: 25671

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect Python bindings for termination status enums
echo "---- solver_wrapper.pyx: termination status enum mappings ----"
sed -n '1,140p' python/cuopt/cuopt/linear_programming/solver/solver_wrapper.pyx

echo "---- solver.pxd: declarations for termination statuses ----"
sed -n '1,120p' python/cuopt/cuopt/linear_programming/solver/solver.pxd

# Inspect server-side mapping / status lists
echo "---- data_definition.py: status lists (around PrimalFeasible / FeasibleFound) ----"
sed -n '740,880p' python/cuopt_server/cuopt_server/utils/linear_programming/data_definition.py

echo "---- solver.py: where MILPTerminationStatus.FeasibleFound is used ----"
sed -n '240,320p' python/cuopt_server/cuopt_server/utils/linear_programming/solver.py

# Show any LPTerminationStatus / pdlp termination status values
echo "---- Search for LPTerminationStatus definition in python/cuopt_server ----"
rg -n "LPTerminationStatus|pdlp_termination_status_t|PDLP.*TerminationStatus" python/cuopt_server/cuopt_server/utils/linear_programming -S || true

echo "---- Search for LPTerminationStatus definition in python/cuopt ----"
rg -n "LPTerminationStatus|pdlp_termination_status_t|PDLP.*TerminationStatus" python/cuopt/cuopt -S || true

Repository: NVIDIA/cuopt

Length of output: 25054

Clarify FeasibleFound status naming for the LP eval.
FeasibleFound exists as a MILP/IP termination status (MILPTerminationStatus) but is not part of the LP/continuous termination statuses (LPTerminationStatus); LP uses Optimal / PrimalFeasible (plus other PDLP statuses). Remove FeasibleFound from the expected problem.Status.name PascalCase values in skills/cuopt-numerical-optimization-api-python/evals/evals.json (line 13), since it also conflicts with the eval’s LP ground_truth check for ['Optimal', 'PrimalFeasible'].

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@skills/cuopt-numerical-optimization-api-python/evals/evals.json` at line 13,
The evals.json entry listing PascalCase problem.Status.name values incorrectly
includes "FeasibleFound"; remove "FeasibleFound" from the expected status list
so the LP/continuous check only expects "Optimal" and "PrimalFeasible" (i.e.,
align with LPTerminationStatus rather than MILPTerminationStatus) and ensure any
ground_truth checks that compare problem.Status.name use
['Optimal','PrimalFeasible'] only.

"Names problem.ObjValue and variable.getValue() for reading results",
"Mentions that status names are case-sensitive (PascalCase)",
"Does not invent method names that are not in the skill"
]
}
]
8 changes: 4 additions & 4 deletions skills/cuopt-numerical-optimization-api-python/skill-card.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
## Description: <br>
Solve Linear Programming (LP), Mixed-Integer Linear Programming (MILP), and Quadratic Programming (QP, beta) with the Python API. <br>
Solve Linear Programming (LP), Mixed-Integer Linear Programming (MILP), and Quadratic Programming (QP, beta) with the NVIDIA cuOpt Python API. <br>

This skill is ready for commercial/non-commercial use. <br>

Expand All @@ -8,7 +8,7 @@ This skill is ready for commercial/non-commercial use. <br>
### License/Terms of Use: <br>
Apache 2.0 <br>
## Use Case: <br>
Developers and engineers use this skill to model and solve linear, mixed-integer linear, and quadratic optimization problems using NVIDIA cuOpt's GPU-accelerated Python API for scheduling, resource allocation, production planning, portfolio optimization, and related tasks. <br>
Developers and engineers who need to formulate and solve linear, mixed-integer linear, and quadratic optimization problems using NVIDIA cuOpt's GPU-accelerated Python API for applications such as scheduling, resource allocation, facility location, production planning, and portfolio optimization. <br>

### Deployment Geography for Use: <br>
Global <br>
Expand All @@ -18,9 +18,9 @@ Risk: Review before execution as proposals could introduce incorrect or misleadi
Mitigation: Review and scan skill before deployment. <br>

## Reference(s): <br>
- [QP Examples (Python API)](references/qp_examples.md) <br>
- [QP Python API Examples](references/qp_examples.md) <br>
- [cuOpt User Guide](https://docs.nvidia.com/cuopt/user-guide/latest/introduction.html) <br>
- [cuOpt Examples Repository](https://github.com/NVIDIA/cuopt-examples) <br>
- [cuOpt API Reference](https://docs.nvidia.com/cuopt/user-guide/latest/api.html) <br>


## Skill Output: <br>
Expand Down
Loading