diff --git a/skills/cuopt-routing-api-python/BENCHMARK.md b/skills/cuopt-routing-api-python/BENCHMARK.md new file mode 100644 index 0000000000..72f6892ec7 --- /dev/null +++ b/skills/cuopt-routing-api-python/BENCHMARK.md @@ -0,0 +1,99 @@ +# Evaluation Report + +Evaluation of the `cuopt-routing-api-python` skill before publication through NVSkills-Eval. + +This benchmark summarizes 3-Tier Evaluation from NVSkills-Eval results for the skill. The goal is to document whether the skill is safe, discoverable, effective, and useful for agents before it is published for broader workflow use. + +## Evaluation Summary + +- Skill: `cuopt-routing-api-python` +- Evaluation date: 2026-05-29 +- NVSkills-Eval profile: `external` +- Environment: `local` +- Dataset: 1 evaluation tasks +- Attempts per task: 2 +- Pass threshold: 50% +- Overall verdict: FAIL + +## Agents Used + +- `claude-code` +- `codex` + +## Metrics Used + +Reported benchmark dimensions: + +- Security: checks whether skill-assisted execution avoids unsafe behavior such as secret leakage, destructive commands, or unauthorized access. +- Correctness: checks whether the agent follows the expected workflow and produces the correct final output. +- Discoverability: checks whether the agent loads the skill when relevant and avoids using it when irrelevant. +- Effectiveness: checks whether the agent performs measurably better with the skill than without it. +- Efficiency: checks whether the agent uses fewer tokens and avoids redundant work. + +Underlying evaluation signals used in this run: + +- `security` (Security): checks for unsafe operations, secret leakage, and unauthorized access. +- `skill_execution` (Skill Execution): verifies that the agent loaded the expected skill and workflow. +- `skill_efficiency` (Efficiency): checks routing quality, decoy avoidance, and redundant tool usage. +- `accuracy` (Accuracy): grades final-answer correctness against the reference answer. +- `goal_accuracy` (Goal Accuracy): checks whether the overall user task completed successfully. +- `behavior_check` (Behavior Check): verifies expected behavior steps, including safety expectations. +- `token_efficiency` (Token Efficiency): compares token usage with and without the skill. + +## Test Tasks + +The benchmark dataset contained 1 evaluation tasks: + +- Positive tasks: 1 tasks where the skill was expected to activate. +- Negative tasks: 0 tasks where no skill was expected. +- Unlabeled tasks: 0 tasks where positive/negative intent could not be inferred. + +Task composition is derived from the evaluation dataset when possible. Entries with `expected_skill` set are treated as positive skill-activation cases, while entries with `expected_skill: null` are treated as negative activation cases. + +## Results + +| Dimension | Num | `claude-code` | `codex` | +|---|---:|---:|---:| +| Security | 2 | 100% (+0%) | 100% (+0%) | +| Correctness | 2 | 100% (+0%) | 95% (+3%) | +| Discoverability | 2 | 100% (+0%) | 70% (-5%) | +| Effectiveness | 2 | 83% (+14%) | 83% (+12%) | +| Efficiency | 2 | 93% (-0%) | 56% (-5%) | + +Score values show skill-assisted performance. Values in parentheses show uplift versus the no-skill baseline when baseline data is available. + +## Tier 1: Static Validation Summary + +Tier 1 validation passed with observations. NVSkills-Eval ran 9 checks and found 8 total findings. + +Top findings: + +- MEDIUM SCHEMA/body_recommended_section: Missing recommended section: '## Instructions' (`skills/cuopt-routing-api-python/SKILL.md`) +- MEDIUM SECURITY/Unknown (SQP-2): Binding the cuOpt server to 0.0.0.0 exposes it on all network interfaces, making it accessible to any host that can reac (`references/server_examples.md:7`) +- LOW QUALITY/quality_discoverability: No '## Purpose' section (`skills/cuopt-routing-api-python/SKILL.md`) +- LOW QUALITY/quality_reliability: No prerequisites/requirements documented (`skills/cuopt-routing-api-python/SKILL.md`) +- LOW QUALITY/quality_reliability: No limitations documented (`skills/cuopt-routing-api-python/SKILL.md`) + +## Tier 2: Deduplication Summary + +Tier 2 validation reported findings. NVSkills-Eval ran 2 checks and found 4 total findings. + +Top findings: + +- HIGH DUPLICATE/duplicate: Duplicate content found within references/server_examples.md: + "# Poll for solution" in references/server_examples.md (lines 45-51) + vs "# Poll for solution" in references/server_examples.md (lines 156-162) (`references/server_examples.md:45`) +- HIGH DUPLICATE/duplicate: Duplicate content found across SKILL.md and references/examples.md: + "# Capacities" in SKILL.md (lines 30-35) + vs "# Add capacity dimension (name, demand_per_order, capacity_per_vehicle)" in references/examples.md (lines 73-75) + vs "# Add capacity dimension" in references/examples.md (lines 156-158) (`SKILL.md:30`) +- HIGH DUPLICATE/duplicate: Duplicate content found across references/examples.md and references/server_examples.md: + "## Additional References (tested in CI)" in references/examples.md (lines 237-249) + vs "## Additional References (tested in CI)" in references/server_examples.md (lines 193-204) (`references/examples.md:237`) +- HIGH DUPLICATE/duplicate: Duplicate content found across assets/pdp_basic/README.md and assets/pdp_basic/model.py: + "# Pickup-Delivery (PDP)" in assets/pdp_basic/README.md (lines 1-7) + vs "(module docstring)" in assets/pdp_basic/model.py (lines 1-2) (`assets/pdp_basic/README.md:1`) + +## Publication Recommendation + +The skill should be reviewed before NVSkills-Eval publication. Skill owners should address the findings above and rerun NVSkills-Eval to refresh this benchmark. diff --git a/skills/cuopt-routing-api-python/SKILL.md b/skills/cuopt-routing-api-python/SKILL.md index 4e07934354..421d68bbe7 100644 --- a/skills/cuopt-routing-api-python/SKILL.md +++ b/skills/cuopt-routing-api-python/SKILL.md @@ -14,6 +14,7 @@ metadata: --- + # cuOpt Routing — Python API Confirm problem type (TSP, VRP, PDP) and data (locations, orders, fleet, constraints) before coding. diff --git a/skills/cuopt-routing-api-python/skill-card.md b/skills/cuopt-routing-api-python/skill-card.md index 7f958cf60f..2e5e3a98fd 100644 --- a/skills/cuopt-routing-api-python/skill-card.md +++ b/skills/cuopt-routing-api-python/skill-card.md @@ -3,12 +3,13 @@ Vehicle routing (VRP, TSP, PDP) with cuOpt — Python API only. Use when the use This skill is ready for commercial/non-commercial use.
-## Owner: NVIDIA
+## Owner +NVIDIA
### License/Terms of Use:
-Apache 2.0
+Apache-2.0
## Use Case:
-Developers and engineers building or solving vehicle routing problems (VRP, TSP, PDP) using the NVIDIA cuOpt Python API for fleet logistics and delivery optimization.
+Developers and engineers building or solving vehicle routing problems (VRP, TSP, PDP) using the NVIDIA cuOpt Python API.
### Deployment Geography for Use:
Global
@@ -18,17 +19,55 @@ Risk: Review before execution as proposals could introduce incorrect or misleadi Mitigation: Review and scan skill before deployment.
## Reference(s):
-- [Routing Python API Examples](references/examples.md)
-- [Routing REST Server Examples](references/server_examples.md)
+- [Python API Examples (VRP, PDP, multi-depot)](references/examples.md)
+- [REST Server Examples](references/server_examples.md)
- [cuOpt User Guide](https://docs.nvidia.com/cuopt/user-guide/latest/introduction.html)
+- [cuOpt Examples Repository](https://github.com/NVIDIA/cuopt-examples)
## Skill Output:
**Output Type(s):** [Code, API Calls]
-**Output Format:** [Markdown with inline Python code blocks]
+**Output Format:** [Python code with cudf/cuOpt API calls]
**Output Parameters:** [1D]
**Other Properties Related to Output:** [None]
+## Evaluation Agents Used:
+- `claude-code`
+- `codex`
+ + + +## Evaluation Tasks:
+1 evaluation task (positive skill-activation), 2 attempts per task, pass threshold 50%.
+ +## Evaluation Metrics Used:
+Reported benchmark dimensions:
+- Security: Checks whether skill-assisted execution avoids unsafe behavior such as secret leakage, destructive commands, or unauthorized access.
+- Correctness: Checks whether the agent follows the expected workflow and produces the correct final output.
+- Discoverability: Checks whether the agent loads the skill when relevant and avoids using it when irrelevant.
+- Effectiveness: Checks whether the agent performs measurably better with the skill than without it.
+- Efficiency: Checks whether the agent uses fewer tokens and avoids redundant work.
+ +Underlying evaluation signals used in this run:
+- `security`: Checks for unsafe operations, secret leakage, and unauthorized access.
+- `skill_execution`: Verifies that the agent loaded the expected skill and workflow.
+- `skill_efficiency`: Checks routing quality, decoy avoidance, and redundant tool usage.
+- `accuracy`: Grades final-answer correctness against the reference answer.
+- `goal_accuracy`: Checks whether the overall user task completed successfully.
+- `behavior_check`: Verifies expected behavior steps, including safety expectations.
+- `token_efficiency`: Compares token usage with and without the skill.
+ + + +## Evaluation Results:
+| Dimension | Num | `claude-code` | `codex` | +|---|---:|---:|---:| +| Security | 2 | 100% (+0%) | 100% (+0%) | +| Correctness | 2 | 100% (+0%) | 95% (+3%) | +| Discoverability | 2 | 100% (+0%) | 70% (-5%) | +| Effectiveness | 2 | 83% (+14%) | 83% (+12%) | +| Efficiency | 2 | 93% (-0%) | 56% (-5%) | + ## Skill Version(s):
26.08.00 (source: frontmatter)
diff --git a/skills/cuopt-routing-api-python/skill.oms.sig b/skills/cuopt-routing-api-python/skill.oms.sig index 6c602ce844..70d7ec278d 100644 --- a/skills/cuopt-routing-api-python/skill.oms.sig +++ b/skills/cuopt-routing-api-python/skill.oms.sig @@ -1 +1 @@ -{"mediaType":"application/vnd.dev.sigstore.bundle.v0.3+json","verificationMaterial":{"x509CertificateChain":{"certificates":[{"rawBytes":"MIICgzCCAgmgAwIBAgIUKIyS7SxNteQIiWzK1dWj85E6520wCgYIKoZIzj0EAwMwVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwHhcNMjYwNDAxMDAwMDAwWhcNMjgwNDIyMTUzMzA5WjBUMQswCQYDVQQGEwJVUzEbMBkGA1UECgwSTlZJRElBIENvcnBvcmF0aW9uMSgwJgYDVQQDDB9OVklESUEgQWdlbnQgU2tpbGxzIFNpZ25pbmcgMDAxMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEYoRM9bQl/dGlwSRNi6bTpIJUXH8Nv9GciP6LSflJYYMLCc296kpyuTSsk5ddbAWiDcFX3C/ydX3jwc+qCLYP6uHy9XphyLjOQ27Yb2J6rBLVtRBS1mgGco/Gr7fL6ODco4GaMIGXMB0GA1UdDgQWBBRQ/5ZW3nJ6lmo9SVk7I15o7UGmpTAfBgNVHSMEGDAWgBRPGpILxMBBleJSsBGjrMKsby1CgjAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQEAwIHgDA3BggrBgEFBQcBAQQrMCkwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vY3NwLm5kaXMubnZpZGlhLmNvbTAKBggqhkjOPQQDAwNoADBlAjAUygu/GiOCIXrgGr4SmLgeEVDcEitfFUv7ALbvLVGVyMysB3mxmO/uInZfXzWcJZsCMQDxuoxj4ZmO30jhkPIcCxGFCOvnUsnfU3TfGcouYm4M6iRpbKvtVnHPiy4bi6pcKf0="},{"rawBytes":"MIICiDCCAg6gAwIBAgIUZsIuSv9NkpJCNqtYEfCouVv5BzowCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAASI72cR3ctKGg4VWnB3bNja6g1Z2PnOmFEopkPof+QeIcPk9rT+g9MjJnq51EQXL93a7C2GJ9J985G4o2V85VD7wJ1RaXhluHW2rf3y8bQGeAYaKMr5s/hUgn+M3/9WlWejgaAwgZ0wHQYDVR0OBBYEFE8akgvEwEGV4lKwEaOswqxvLUKCMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgEGMDcGCCsGAQUFBwEBBCswKTAnBggrBgEFBQcwAYYbaHR0cDovL29jc3AubmRpcy5udmlkaWEuY29tMAoGCCqGSM49BAMDA2gAMGUCMQCeIMMfAbyzPDacw2MxG+Yt1cikrJX/DVxiGfXuHmkkXn6VgSzE79+lkqDErpVO2gYCMCNEColOyvUvkzZGUEI1hQ3PfMgi3FIo9tHoBKMw4/wGBLFpu/0ubtmbBXM6/UMOEw=="},{"rawBytes":"MIICRTCCAcygAwIBAgIUeJdY3rV86EdvFmG7L8LJBsyQFYkwCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTB2MBAGByqGSM49AgEGBSuBBAAiA2IABAYpiXCDjJ9NT2eSDhyHJVSw1Tbze18cGG2F/578oWvHxg23eQAhNRYdq88i1iOshZSO6C29doKui5Xpmo/7Ctw9Sx4PP2RzOmIuOLCuTdNtKcTRwi4GEsd5BAFvWj42M6NjMGEwHQYDVR0OBBYEFItnoAjjfuCEUvzyvWyI2vOGvwPjMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMAoGCCqGSM49BAMDA2cAMGQCMCwtAjWLaNwgGWNCgdyNoTyvNhqWRECRJV2r3+7w8g0PL6NHLOsbkgE09BH95h8XlgIwTaQmbbUh2ChAJ5TA1wRiVDnCcvbzHlZl2jM2FcwQQZlk19LOAbyGMRixbu2Ww/rj"}]},"tlogEntries":[]},"dsseEnvelope":{"payload":"ewogICJfdHlwZSI6ICJodHRwczovL2luLXRvdG8uaW8vU3RhdGVtZW50L3YxIiwKICAic3ViamVjdCI6IFsKICAgIHsKICAgICAgIm5hbWUiOiAiY3VvcHQtcm91dGluZy1hcGktcHl0aG9uIiwKICAgICAgImRpZ2VzdCI6IHsKICAgICAgICAic2hhMjU2IjogIjQ1Yjk5ZWY2ZDhlMmZhZGY3MDE3YTg5MTE5OWRlNDIzNWZkMjZmZjQxYTM2ZDA5Mjc3YTQwMzdhZTIyZTk2OGUiCiAgICAgIH0KICAgIH0KICBdLAogICJwcmVkaWNhdGVUeXBlIjogImh0dHBzOi8vbW9kZWxfc2lnbmluZy9zaWduYXR1cmUvdjEuMCIsCiAgInByZWRpY2F0ZSI6IHsKICAgICJzZXJpYWxpemF0aW9uIjogewogICAgICAiYWxsb3dfc3ltbGlua3MiOiBmYWxzZSwKICAgICAgImhhc2hfdHlwZSI6ICJzaGEyNTYiLAogICAgICAiaWdub3JlX3BhdGhzIjogWwogICAgICAgICIuZ2l0aWdub3JlIiwKICAgICAgICAiLmdpdGF0dHJpYnV0ZXMiLAogICAgICAgICIuZ2l0IiwKICAgICAgICAiLmdpdGh1YiIKICAgICAgXSwKICAgICAgIm1ldGhvZCI6ICJmaWxlcyIKICAgIH0sCiAgICAicmVzb3VyY2VzIjogWwogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiZDk0MDVkMjgyMDhjZDBhMzIwOGFmOWExMzU4NTUyZTk0NzkyYjU2MzBlN2UyMmJlNDRmNDMwY2Y5ZDQwZmUyMSIsCiAgICAgICAgIm5hbWUiOiAiU0tJTEwubWQiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgImRpZ2VzdCI6ICJjNzAwMmEzMTIxOTgzZjMyOTRlZmJlOGM5NTQxOTQzYmYyNGM4OWEwN2JlYTZhMzIwMDdjNzc0YTJjODA4MDIxIiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvUkVBRE1FLm1kIgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiZTBkMWExZmQ3ZDBhZDRlNDU0ZDA4ZjU1ZGU5MWJiZWRlNzhmZjEyMjJkNmE1NDJkNTVhYWFjZjcxYzVhN2U2MiIsCiAgICAgICAgIm5hbWUiOiAiYXNzZXRzL3BkcF9iYXNpYy9SRUFETUUubWQiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgImRpZ2VzdCI6ICIxNDRkYTFkZjVkZTI4ZDc4NWE5YjQ2N2IzZDE0NDE3ZTcxNmY1MzJhYzliOTg5MDQ2ZWFmN2U0ZjUyOTlhNWZkIiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvcGRwX2Jhc2ljL21vZGVsLnB5IgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiMGE4NWFlZjFjMWJlNTk5ODlkZTQwYWE2Y2U5ZmU1NGU3MjBlNzA5NWYwODZiYzZmNjg0ZjJiM2M5ZGEzMzg5NCIsCiAgICAgICAgIm5hbWUiOiAiYXNzZXRzL3ZycF9iYXNpYy9SRUFETUUubWQiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgImRpZ2VzdCI6ICI1MjQ1Yjc3NDY1YTI2YjY4YWVmYmFhMzI0OWI1MWVmMGRhNDUwNWY0ZTE5NzRjZjZkMGY0NGIxYzc4ZmM4MDcwIiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvdnJwX2Jhc2ljL21vZGVsLnB5IgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiZGJhMjAxMTE5ZTRmMGM1YjdkN2IxN2RmOWM3MTFkNDQ4M2UwMTg5MzM1MDhlMzYxZWU5MjllNmM0NGU1NmE0YiIsCiAgICAgICAgIm5hbWUiOiAiZXZhbHMvZXZhbHMuanNvbiIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAiZGlnZXN0IjogIjVlYjM1NzM1NTU5ZDkzNWMyMWUyMGE0OTU5MWQ1NGM5YjZmYTkyNDcyMTdhYTQ3NzZhNWY4OTgzY2JkMjdmODEiLAogICAgICAgICJuYW1lIjogInJlZmVyZW5jZXMvZXhhbXBsZXMubWQiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgImRpZ2VzdCI6ICI1MDhiZjRhZThjYjViYzdlMjQ5YjM3NzI2MGYxNDIxYjcwZDlkMzQ1YmI1YTZkMTZjNmZhMGI1NmUyNTY4MjViIiwKICAgICAgICAibmFtZSI6ICJyZWZlcmVuY2VzL3NlcnZlcl9leGFtcGxlcy5tZCIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAiZGlnZXN0IjogImRlM2I4MTQxNjU4MTBmYmI3ZDE4NzM3YWVkOTNjZTM0NDRiZWJiYzA3MGM5NzEzNzA4MGRmZGI1ZmFiNWQzODciLAogICAgICAgICJuYW1lIjogInNraWxsLWNhcmQubWQiCiAgICAgIH0KICAgIF0KICB9Cn0=","payloadType":"application/vnd.in-toto+json","signatures":[{"sig":"MGQCMBBJe4Hso6bzkaOkc/KQFSQ63hhfLLVFZW49KcJCJIQFCAZxI7sqpJo2Jiu0oxACMAIweSCs4Xb1qPmKSrMV/ygqL3/EgFKotG6B8z8Npqpc++MWbq5zd9t/sU2vBdfcmm8j","keyid":""}]}} \ No newline at end of file +{"mediaType":"application/vnd.dev.sigstore.bundle.v0.3+json","verificationMaterial":{"x509CertificateChain":{"certificates":[{"rawBytes":"MIICgzCCAgmgAwIBAgIUKIyS7SxNteQIiWzK1dWj85E6520wCgYIKoZIzj0EAwMwVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwHhcNMjYwNDAxMDAwMDAwWhcNMjgwNDIyMTUzMzA5WjBUMQswCQYDVQQGEwJVUzEbMBkGA1UECgwSTlZJRElBIENvcnBvcmF0aW9uMSgwJgYDVQQDDB9OVklESUEgQWdlbnQgU2tpbGxzIFNpZ25pbmcgMDAxMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEYoRM9bQl/dGlwSRNi6bTpIJUXH8Nv9GciP6LSflJYYMLCc296kpyuTSsk5ddbAWiDcFX3C/ydX3jwc+qCLYP6uHy9XphyLjOQ27Yb2J6rBLVtRBS1mgGco/Gr7fL6ODco4GaMIGXMB0GA1UdDgQWBBRQ/5ZW3nJ6lmo9SVk7I15o7UGmpTAfBgNVHSMEGDAWgBRPGpILxMBBleJSsBGjrMKsby1CgjAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQEAwIHgDA3BggrBgEFBQcBAQQrMCkwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vY3NwLm5kaXMubnZpZGlhLmNvbTAKBggqhkjOPQQDAwNoADBlAjAUygu/GiOCIXrgGr4SmLgeEVDcEitfFUv7ALbvLVGVyMysB3mxmO/uInZfXzWcJZsCMQDxuoxj4ZmO30jhkPIcCxGFCOvnUsnfU3TfGcouYm4M6iRpbKvtVnHPiy4bi6pcKf0="},{"rawBytes":"MIICiDCCAg6gAwIBAgIUZsIuSv9NkpJCNqtYEfCouVv5BzowCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAASI72cR3ctKGg4VWnB3bNja6g1Z2PnOmFEopkPof+QeIcPk9rT+g9MjJnq51EQXL93a7C2GJ9J985G4o2V85VD7wJ1RaXhluHW2rf3y8bQGeAYaKMr5s/hUgn+M3/9WlWejgaAwgZ0wHQYDVR0OBBYEFE8akgvEwEGV4lKwEaOswqxvLUKCMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgEGMDcGCCsGAQUFBwEBBCswKTAnBggrBgEFBQcwAYYbaHR0cDovL29jc3AubmRpcy5udmlkaWEuY29tMAoGCCqGSM49BAMDA2gAMGUCMQCeIMMfAbyzPDacw2MxG+Yt1cikrJX/DVxiGfXuHmkkXn6VgSzE79+lkqDErpVO2gYCMCNEColOyvUvkzZGUEI1hQ3PfMgi3FIo9tHoBKMw4/wGBLFpu/0ubtmbBXM6/UMOEw=="},{"rawBytes":"MIICRTCCAcygAwIBAgIUeJdY3rV86EdvFmG7L8LJBsyQFYkwCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTB2MBAGByqGSM49AgEGBSuBBAAiA2IABAYpiXCDjJ9NT2eSDhyHJVSw1Tbze18cGG2F/578oWvHxg23eQAhNRYdq88i1iOshZSO6C29doKui5Xpmo/7Ctw9Sx4PP2RzOmIuOLCuTdNtKcTRwi4GEsd5BAFvWj42M6NjMGEwHQYDVR0OBBYEFItnoAjjfuCEUvzyvWyI2vOGvwPjMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMAoGCCqGSM49BAMDA2cAMGQCMCwtAjWLaNwgGWNCgdyNoTyvNhqWRECRJV2r3+7w8g0PL6NHLOsbkgE09BH95h8XlgIwTaQmbbUh2ChAJ5TA1wRiVDnCcvbzHlZl2jM2FcwQQZlk19LOAbyGMRixbu2Ww/rj"}]},"tlogEntries":[]},"dsseEnvelope":{"payload":"ewogICJfdHlwZSI6ICJodHRwczovL2luLXRvdG8uaW8vU3RhdGVtZW50L3YxIiwKICAic3ViamVjdCI6IFsKICAgIHsKICAgICAgIm5hbWUiOiAiY3VvcHQtcm91dGluZy1hcGktcHl0aG9uIiwKICAgICAgImRpZ2VzdCI6IHsKICAgICAgICAic2hhMjU2IjogIjFjNTJlZDRiNGI0NWMyOWQ5YmNlZDE2Yjc5MGQ3YmU3MjQ5MjM1NjQ2NzMwYTE4MjViNzIwZThmNTZkNDNjNzUiCiAgICAgIH0KICAgIH0KICBdLAogICJwcmVkaWNhdGVUeXBlIjogImh0dHBzOi8vbW9kZWxfc2lnbmluZy9zaWduYXR1cmUvdjEuMCIsCiAgInByZWRpY2F0ZSI6IHsKICAgICJzZXJpYWxpemF0aW9uIjogewogICAgICAibWV0aG9kIjogImZpbGVzIiwKICAgICAgImFsbG93X3N5bWxpbmtzIjogZmFsc2UsCiAgICAgICJpZ25vcmVfcGF0aHMiOiBbCiAgICAgICAgIi5naXRodWIiLAogICAgICAgICIuZ2l0aWdub3JlIiwKICAgICAgICAiLmdpdGF0dHJpYnV0ZXMiLAogICAgICAgICIuZ2l0IgogICAgICBdLAogICAgICAiaGFzaF90eXBlIjogInNoYTI1NiIKICAgIH0sCiAgICAicmVzb3VyY2VzIjogWwogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiMzRjZDY0YjIyYmQyMjEyNDc0MDZmZmFkMDhhZjFiYmNkYzE2NWRlODVmYTZkODM3NTVjMWY3OWViMjBjYjBkOSIsCiAgICAgICAgIm5hbWUiOiAiQkVOQ0hNQVJLLm1kIgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiNDhjNDc2MzIwYThmN2YxM2VhNmQzOTA5YzRkODkwZjk5ZDk5MmYyMzJiNDZjMTAwNmM3MDE0YTdiZTI5MzIzMyIsCiAgICAgICAgIm5hbWUiOiAiU0tJTEwubWQiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgImRpZ2VzdCI6ICJjNzAwMmEzMTIxOTgzZjMyOTRlZmJlOGM5NTQxOTQzYmYyNGM4OWEwN2JlYTZhMzIwMDdjNzc0YTJjODA4MDIxIiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvUkVBRE1FLm1kIgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiZTBkMWExZmQ3ZDBhZDRlNDU0ZDA4ZjU1ZGU5MWJiZWRlNzhmZjEyMjJkNmE1NDJkNTVhYWFjZjcxYzVhN2U2MiIsCiAgICAgICAgIm5hbWUiOiAiYXNzZXRzL3BkcF9iYXNpYy9SRUFETUUubWQiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgImRpZ2VzdCI6ICIxNDRkYTFkZjVkZTI4ZDc4NWE5YjQ2N2IzZDE0NDE3ZTcxNmY1MzJhYzliOTg5MDQ2ZWFmN2U0ZjUyOTlhNWZkIiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvcGRwX2Jhc2ljL21vZGVsLnB5IgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiMGE4NWFlZjFjMWJlNTk5ODlkZTQwYWE2Y2U5ZmU1NGU3MjBlNzA5NWYwODZiYzZmNjg0ZjJiM2M5ZGEzMzg5NCIsCiAgICAgICAgIm5hbWUiOiAiYXNzZXRzL3ZycF9iYXNpYy9SRUFETUUubWQiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgImRpZ2VzdCI6ICI1MjQ1Yjc3NDY1YTI2YjY4YWVmYmFhMzI0OWI1MWVmMGRhNDUwNWY0ZTE5NzRjZjZkMGY0NGIxYzc4ZmM4MDcwIiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvdnJwX2Jhc2ljL21vZGVsLnB5IgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiZGJhMjAxMTE5ZTRmMGM1YjdkN2IxN2RmOWM3MTFkNDQ4M2UwMTg5MzM1MDhlMzYxZWU5MjllNmM0NGU1NmE0YiIsCiAgICAgICAgIm5hbWUiOiAiZXZhbHMvZXZhbHMuanNvbiIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAiZGlnZXN0IjogIjVlYjM1NzM1NTU5ZDkzNWMyMWUyMGE0OTU5MWQ1NGM5YjZmYTkyNDcyMTdhYTQ3NzZhNWY4OTgzY2JkMjdmODEiLAogICAgICAgICJuYW1lIjogInJlZmVyZW5jZXMvZXhhbXBsZXMubWQiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgImRpZ2VzdCI6ICI1MDhiZjRhZThjYjViYzdlMjQ5YjM3NzI2MGYxNDIxYjcwZDlkMzQ1YmI1YTZkMTZjNmZhMGI1NmUyNTY4MjViIiwKICAgICAgICAibmFtZSI6ICJyZWZlcmVuY2VzL3NlcnZlcl9leGFtcGxlcy5tZCIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAiZGlnZXN0IjogIjcxOTRjN2FkMmIwMjY2MmU5MTFhNzFmOGIwN2Q2Nzk1NmNiNzdmMDMxZTU3NWEwZTcxZjVjMTZlYjk5ZTMzOWEiLAogICAgICAgICJuYW1lIjogInNraWxsLWNhcmQubWQiCiAgICAgIH0KICAgIF0KICB9Cn0=","payloadType":"application/vnd.in-toto+json","signatures":[{"sig":"MGUCMQC/if57e3mXxcr46MyoN7/Qlrwmk9leJtI83klm2/SuPZXkOfRclZp539nJbCqxcq4CMChXvVkzTj75l5w+zoaUK63MRHUujhIesZqb435AE2hAkKTOIrL4596BL+DxmL+QUA==","keyid":""}]}} \ No newline at end of file