diff --git a/skills/cuopt-server-api-python/BENCHMARK.md b/skills/cuopt-server-api-python/BENCHMARK.md index cc0dda87e0..a0e3db287c 100644 --- a/skills/cuopt-server-api-python/BENCHMARK.md +++ b/skills/cuopt-server-api-python/BENCHMARK.md @@ -1,81 +1,110 @@ -# Evaluation Report +# Skill Benchmark: cuopt-server-api-python -Evaluation of the `cuopt-server-api-python` skill before publication through NVSkills-Eval. +> ✅ **Overall verdict: PASS — Recommended for publication** -This benchmark summarizes 3-Tier Evaluation from NVSkills-Eval results for the skill. The goal is to document whether the skill is safe, discoverable, effective, and useful for agents before it is published for broader workflow use. +## Publication Recommendation + +Recommended for publication based on the completed evaluation evidence in this report. -## Evaluation Summary +## Evaluation Metadata - Skill: `cuopt-server-api-python` -- Evaluation date: 2026-06-29 -- NVSkills-Eval profile: `external` -- Environment: `astra-sandbox` -- Dataset: 1 evaluation tasks +- Evaluation date: 2026-08-25 +- Evaluator version: `1.3.2` +- Agents: Claude Code (`aws/anthropic/bedrock-claude-opus-4-8`), Codex (`openai/openai/gpt-5.5`) +- Tasks: 8 evaluation tasks (8 positive) +- Dataset digest: `sha256:89ad304517df359a8c33a1f67da3ad9df8aeae5ec234d2adbed1593bf49c4ebc` (skill-evaluator-dataset-snapshot/1) - Attempts per task: 1 -- Pass threshold: 50% -- Overall verdict: PASS +- Environment: `local` +- Tier 3 evidence: required for publication -## Agents Used +Tasks ran on the trusted local host; local mode is not sandboxed. -- `claude-code` -- `codex` +## Execution and Provenance -## Metrics Used +- Validation status: `passed` +- Report generation: `complete` +- Evaluator version: `1.3.2` +- Git commit: `0117bc2e3e54da4244a656466526c5b1b5a559ea` +- Content type: requested `auto`, detected `skill` +- Container image: `gitlab-master.nvidia.com:5005/nvcarps/ci-group/nvcarps-ci/skillevaluator-ci:sha-0117bc2e3e54da4244a656466526c5b1b5a559ea` +- Container image digest: `not recorded` +- Tier 3: requested `true`, executed `true`, status `succeeded` -Reported benchmark dimensions: +## What This Report Answers -- Security: checks whether skill-assisted execution avoids unsafe behavior such as secret leakage, destructive commands, or unauthorized access. -- Correctness: checks whether the agent follows the expected workflow and produces the correct final output. -- Discoverability: checks whether the agent loads the skill when relevant and avoids using it when irrelevant. -- Effectiveness: checks whether the agent performs measurably better with the skill than without it. -- Efficiency: checks whether the agent uses fewer tokens and avoids redundant work. +The three-tier evaluation checks whether the skill: -Underlying evaluation signals used in this run: +- is safe to use; +- produces correct answers; +- is discovered and activated when needed; +- helps the agent complete the user's goal and expected workflow; and +- avoids wasted skill and tool usage. -- `security` (Security): checks for unsafe operations, secret leakage, and unauthorized access. -- `skill_execution` (Skill Execution): verifies that the agent loaded the expected skill and workflow. -- `skill_efficiency` (Efficiency): checks routing quality, decoy avoidance, and redundant tool usage. -- `accuracy` (Accuracy): grades final-answer correctness against the reference answer. -- `goal_accuracy` (Goal Accuracy): checks whether the overall user task completed successfully. -- `behavior_check` (Behavior Check): verifies expected behavior steps, including safety expectations. -- `token_efficiency` (Token Efficiency): compares token usage with and without the skill. +## Results at a Glance -## Test Tasks +| Measure | Claude Code (Baseline → Skill Uplift) | Codex (Baseline → Skill Uplift) | +|---|---:|---:| +| Overall | 64% → 98% (+35 points) | 62% → 96% (+34 points) | +| Security | 100% → 100% (±0 points) | 69% → 100% (+31 points) | +| Correctness | 75% → 100% (+25 points) | 88% → 98% (+10 points) | +| Discoverability | 44% → 100% (+56 points) | 45% → 93% (+48 points) | +| Effectiveness | 56% → 95% (+38 points) | 72% → 88% (+15 points) | +| Efficiency | 44% → 97% (+53 points) | 37% → 100% (+63 points) | -The benchmark dataset contained 1 evaluation tasks: +**How to read this table:** baseline is the same task attempted without the target skill. Uplift is `skill score - baseline score`, shown in percentage points. -- Positive tasks: 1 tasks where the skill was expected to activate. -- Negative tasks: 0 tasks where no skill was expected. -- Unlabeled tasks: 0 tasks where positive/negative intent could not be inferred. +Example: `47% → 92% (+45 points)` means the skill-assisted run scored 92%, 45 percentage points above its 47% no-skill baseline. -Task composition is derived from the evaluation dataset when possible. Entries with `expected_skill` set are treated as positive skill-activation cases, while entries with `expected_skill: null` are treated as negative activation cases. +## Tier Status -## Results +| Tier | Purpose | Status | Evidence | +|---|---|---|---| +| Tier 1 | Static validation | **PASSED WITH OBSERVATIONS** | 1 validator(s); 2 finding(s) | +| Tier 2 | Semantic deduplication | **NOT RUN** | No result was recorded | +| Tier 3 | Live agent evaluation | **PASS** | 2 agent(s); 8 task(s) | -| Dimension | Num | `claude-code` | `codex` | -|---|---:|---:|---:| -| Security | 1 | 100% (+0%) | 100% (+0%) | -| Correctness | 1 | 100% (+70%) | 97% (+35%) | -| Discoverability | 1 | 100% (+100%) | 97% (+69%) | -| Effectiveness | 1 | 100% (+56%) | 100% (+40%) | -| Efficiency | 1 | 95% (+68%) | 95% (+67%) | +## Findings and Observations -Score values show skill-assisted performance. Values in parentheses show uplift versus the no-skill baseline when baseline data is available. +
+Show detailed findings and successful checks -## Tier 1: Static Validation Summary +- **MEDIUM** SCHEMA/frontmatter_field_placement: Root field 'version' is ignored; use 'metadata.version' (`skills/cuopt-server-api-python/SKILL.md`) +- **LOW** SCHEMA/author_format: Author must be of the form 'Name ' (`skills/cuopt-server-api-python/SKILL.md`) -Tier 1 validation passed with observations. NVSkills-Eval ran 1 checks and found 3 total findings. +
-Top findings: +## Scoring Methodology -- MEDIUM SCHEMA/body_recommended_section: Missing recommended section: '## Instructions' (`skills/cuopt-server-api-python/SKILL.md`) -- MEDIUM SCHEMA/body_recommended_section: Missing recommended section: '## Examples' (`skills/cuopt-server-api-python/SKILL.md`) -- LOW SCHEMA/author_format: Author must be of the form 'Name ' (`skills/cuopt-server-api-python/SKILL.md`) +
+Show dimension definitions, source signals, and thresholds -## Tier 2: Deduplication Summary +| Dimension | Question | Scored signals | +|---|---|---| +| Security | Is it safe to use? | `security` (100%) | +| Correctness | Is the answer correct? | `accuracy` (100%) | +| Discoverability | Was the right skill loaded when needed? | `skill_execution` (100%) | +| Effectiveness | Did the skill help complete the task? | `goal_accuracy` (50%) + `behavior_check` (50%) | +| Efficiency | Did it avoid wasted tool or skill usage? | `skill_efficiency` (100%) | -This tier was not run or did not produce findings in this report. +- Dimension bands: PASS at 50% or above; NEUTRAL from 40% to below 50%; FAIL below 40%. +- Overall Tier 3 lift: PASS at +5 points or more; FAIL at -10 points or less; values between those bands are NEUTRAL. +- Overall verdict: PASS only when every configured dimension passes for at least one supported agent. Lift is reported as diagnostic evidence and does not override this gate. +- The 50% attempt pass threshold is a separate per-task gate; it is not the dimension pass threshold. +- Effectiveness is the equal-weight mean of goal completion (`goal_accuracy`) and expected workflow adherence (`behavior_check`). +- Token efficiency is a separate report-only signal. It does not change a dimension score or the overall verdict. -## Publication Recommendation +Signals present in this run: + +- `security` (Security): unsafe operations, secret leakage, and unauthorized access. +- `skill_execution` (Skill Execution): whether the expected skill was found and executed. +- `skill_efficiency` (Efficiency): routing quality, workspace-aware skill reads, and productive tool use. +- `accuracy` (Accuracy): final-answer correctness against the reference answer. +- `goal_accuracy` (Goal Accuracy): whether the user's goal was achieved. +- `behavior_check` (Behavior Check): whether the expected workflow behavior was followed. + +
+ +## Freshness -The skill is suitable to proceed toward NVSkills-Eval publication based on this benchmark. Skill owners should keep this file with the skill and refresh it when the evaluation dataset, skill behavior, or target agents materially change. +Regenerate this benchmark when the skill, evaluation dataset, target agent/model, evaluator version, environment, or scoring policy changes. diff --git a/skills/cuopt-server-api-python/SKILL.md b/skills/cuopt-server-api-python/SKILL.md index 3fb28407ab..d952e856e7 100644 --- a/skills/cuopt-server-api-python/SKILL.md +++ b/skills/cuopt-server-api-python/SKILL.md @@ -13,12 +13,20 @@ metadata: - deployment --- - - # cuOpt Server — Deploy and client (Python/curl) This skill covers **starting the server** and **client examples** (curl, Python). Server has no separate C API (clients can be any language). +## Purpose + +Use this skill when the user is deploying the cuOpt REST server or writing a client against it — choosing a deployment target, mapping a problem onto the HTTP endpoints, translating between Python-API and REST field names, or debugging a rejected payload. + +## Prerequisites + +- An NVIDIA GPU with a working CUDA driver (the server requires one; `--gpus all` for Docker). +- `cuopt-server` installed, or Docker with the NVIDIA Container Toolkit. See the install skill. +- Python clients need `requests`. No API key or auth token is required by the server itself. + ## Problem types supported | Problem type | Supported | @@ -42,24 +50,30 @@ Ask these if not already clear: # Development python -m cuopt_server.cuopt_service --ip 0.0.0.0 --port 8000 -# Docker +# Docker — pick the tag matching your CUDA major version docker run --gpus all -d -p 8000:8000 -e CUOPT_SERVER_PORT=8000 \ - nvidia/cuopt:latest-cuda12.9-py3.13 + nvidia/cuopt:latest-cu13 ``` +Use `latest-cu12` or `latest-cu13` to match your driver's CUDA major version (`latest-cu13-ubi10` for a UBI10 base). Prefer these over the CUDA+Python-specific tags such as `latest-cuda12.9-py3.13` — those track a single Python line and go stale when it stops receiving builds. + +For production, pin rather than float: `latest-*` tags are mutable and can silently move to a different image. Use a full release tag (`nvidia/cuopt:-cuda-py`) or an immutable digest (`nvidia/cuopt@sha256:`). Check the `nvidia/cuopt` registry for available tags. + ## Verify ```bash curl http://localhost:8000/cuopt/health ``` -## Workflow +## Instructions 1. POST to `/cuopt/request` → get `reqId` 2. Poll `/cuopt/solution/{reqId}` until solution ready 3. Parse response -## Python client (routing) +Treat `reqId` as untrusted input: validate it (e.g. `re.fullmatch(r"[A-Za-z0-9_-]{1,64}", req_id)`) before interpolating it into the polling URL, and set an explicit `timeout` on every request. + +## Examples ```python import requests, time @@ -72,7 +86,7 @@ payload = { "fleet_data": {"vehicle_locations": [[0, 0]], "capacities": [[50]], "vehicle_time_windows": [[0, 200]]}, "solver_config": {"time_limit": 5} } -r = requests.post(f"{SERVER}/cuopt/request", json=payload, headers=HEADERS) +r = requests.post(f"{SERVER}/cuopt/request", json=payload, headers=HEADERS, timeout=30) req_id = r.json()["reqId"] # Poll: GET /cuopt/solution/{req_id} ``` @@ -87,13 +101,28 @@ req_id = r.json()["reqId"] Use `travel_time_matrix_data` (not transit_time_matrix_data). Capacities: `[[50, 50]]` not `[[50], [50]]`. -## Debugging (422 / payload) +## Troubleshooting + +| Error | Cause | Solution | +|-------|-------|----------| +| `422 Unprocessable Entity` | Field name not in the schema | Check names against the OpenAPI spec at `/cuopt.yaml`. Most common: `transit_time_matrix_data` → `travel_time_matrix_data` | +| `422` on `fleet_data` | Capacities nested per vehicle instead of per dimension | Use `[[50, 50]]` (one inner list per capacity dimension), not `[[50], [50]]` | +| Connection refused | Server not up, or bound to a different interface/port | `curl http://localhost:8000/cuopt/health`; start with `--ip 0.0.0.0 --port 8000` | +| Docker container exits immediately | No GPU visible to the container | Run with `--gpus all` and confirm the NVIDIA Container Toolkit is installed | +| Polling never returns a solution | Solve exceeds the client's poll budget | Raise `solver_config.time_limit` and the poll loop count together | + +Capture the `reqId` and the full response body for any failed request — both are needed to diagnose server-side rejections. + +## Limitations -**Validation errors:** Check field names against OpenAPI (`/cuopt.yaml`). Common mistakes: `transit_time_matrix_data` → `travel_time_matrix_data`; capacities per dimension `[[50, 50]]` not per vehicle `[[50], [50]]`. Capture `reqId` and response body for failed requests. +- **QP is not exposed over REST.** Use the Python or C API for quadratic objectives. +- **The server ships no authentication or TLS.** Anything that can reach the port can submit jobs. Put it behind a gateway and treat `--server`/base URLs as trusted-network endpoints only. +- Solutions are retrieved by polling; there is no push/webhook delivery. +- One request is solved at a time per server process; concurrency requires multiple replicas. ## Runnable assets -Run from each asset directory (server must be running; scripts exit 0 if server unreachable). All use Python `requests`: +Run from each asset directory (server must be running; scripts exit 0 if server unreachable). All use Python `requests` and accept `--server` (default `http://localhost:8000`): - [assets/vrp_simple/](assets/vrp_simple/) — Basic VRP (no time windows) - [assets/vrp_basic/](assets/vrp_basic/) — VRP with time windows diff --git a/skills/cuopt-server-api-python/assets/README.md b/skills/cuopt-server-api-python/assets/README.md index 1389f3eb7b..0bc8fb2821 100644 --- a/skills/cuopt-server-api-python/assets/README.md +++ b/skills/cuopt-server-api-python/assets/README.md @@ -11,4 +11,4 @@ REST client examples (Python requests). Each runs against a cuOpt server; if the | `milp_basic/` | MILP (integer + continuous variables) | Start server: `python -m cuopt_server.cuopt_service --ip 0.0.0.0 --port 8000` -Env: `CUOPT_SERVER_URL` (default `http://localhost:8000`). +Server URL: `--server` (default `http://localhost:8000`). diff --git a/skills/cuopt-server-api-python/assets/lp_basic/README.md b/skills/cuopt-server-api-python/assets/lp_basic/README.md index 34c10fb350..ad313bf0b9 100644 --- a/skills/cuopt-server-api-python/assets/lp_basic/README.md +++ b/skills/cuopt-server-api-python/assets/lp_basic/README.md @@ -7,4 +7,4 @@ Submit an LP to the cuOpt server (CSR format) and poll for the solution. **Run:** `python client.py` If the server is not reachable, the script exits 0 (skip). -**Env:** `CUOPT_SERVER_URL` (default `http://localhost:8000`). +**Server URL:** `--server` (default `http://localhost:8000`). diff --git a/skills/cuopt-server-api-python/assets/lp_basic/client.py b/skills/cuopt-server-api-python/assets/lp_basic/client.py index bca7b15295..8d15502c23 100644 --- a/skills/cuopt-server-api-python/assets/lp_basic/client.py +++ b/skills/cuopt-server-api-python/assets/lp_basic/client.py @@ -5,29 +5,44 @@ REST client: LP request (maximize 40x + 30y s.t. 2x+3y<=240, 4x+2y<=200). Requires cuOpt server running. Usage: python client.py - Set CUOPT_SERVER_URL (default http://localhost:8000). Exits 0 if server unreachable (e.g. in CI without server). + Pass --server (default http://localhost:8000). Exits 0 if server unreachable (e.g. in CI without server). """ -import os +import argparse +import re import sys import time import requests -SERVER = os.environ.get("CUOPT_SERVER_URL", "http://localhost:8000") +DEFAULT_SERVER = "http://localhost:8000" HEADERS = {"Content-Type": "application/json", "CLIENT-VERSION": "custom"} +REQUEST_TIMEOUT = 30 +REQ_ID_PATTERN = re.compile(r"[A-Za-z0-9_-]{1,64}") -def server_ok(): +def parse_args(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument( + "--server", + default=DEFAULT_SERVER, + help=f"cuOpt server base URL (default: {DEFAULT_SERVER})", + ) + return parser.parse_args() + + +def server_ok(server): try: - r = requests.get(f"{SERVER}/cuopt/health", timeout=2) + r = requests.get(f"{server}/cuopt/health", timeout=2) return r.status_code == 200 except Exception: return False def main(): - if not server_ok(): + server = parse_args().server + + if not server_ok(server): print( "Server not running, skipping. Start with: python -m cuopt_server.cuopt_service --ip 0.0.0.0 --port 8000" ) @@ -57,15 +72,23 @@ def main(): } response = requests.post( - f"{SERVER}/cuopt/request", json=payload, headers=HEADERS + f"{server}/cuopt/request", + json=payload, + headers=HEADERS, + timeout=REQUEST_TIMEOUT, ) response.raise_for_status() req_id = response.json()["reqId"] + if not REQ_ID_PATTERN.fullmatch(req_id): + print(f"Unexpected reqId from server: {req_id!r}") + sys.exit(1) print(f"Submitted: {req_id}") for _ in range(30): response = requests.get( - f"{SERVER}/cuopt/solution/{req_id}", headers=HEADERS + f"{server}/cuopt/solution/{req_id}", + headers=HEADERS, + timeout=REQUEST_TIMEOUT, ) result = response.json() diff --git a/skills/cuopt-server-api-python/assets/milp_basic/README.md b/skills/cuopt-server-api-python/assets/milp_basic/README.md index e490840557..8f528865c1 100644 --- a/skills/cuopt-server-api-python/assets/milp_basic/README.md +++ b/skills/cuopt-server-api-python/assets/milp_basic/README.md @@ -3,4 +3,4 @@ Same problem as LP (maximize 40x + 30y, 2x+3y≤240, 4x+2y≤200) with `variable_types`: first variable integer, second continuous. **Requires:** cuOpt server running. **Run:** `python client.py` (exits 0 if server unreachable). -**Env:** `CUOPT_SERVER_URL` (default `http://localhost:8000`). Variable types: `continuous`, `integer`, `binary`. +**Server URL:** `--server` (default `http://localhost:8000`). Variable types: `continuous`, `integer`, `binary`. diff --git a/skills/cuopt-server-api-python/assets/milp_basic/client.py b/skills/cuopt-server-api-python/assets/milp_basic/client.py index 1c18de60e9..998cadd913 100644 --- a/skills/cuopt-server-api-python/assets/milp_basic/client.py +++ b/skills/cuopt-server-api-python/assets/milp_basic/client.py @@ -6,26 +6,41 @@ Requires cuOpt server running. Exits 0 if server unreachable. """ -import os +import argparse +import re import sys import time import requests -SERVER = os.environ.get("CUOPT_SERVER_URL", "http://localhost:8000") +DEFAULT_SERVER = "http://localhost:8000" HEADERS = {"Content-Type": "application/json", "CLIENT-VERSION": "custom"} +REQUEST_TIMEOUT = 30 +REQ_ID_PATTERN = re.compile(r"[A-Za-z0-9_-]{1,64}") -def server_ok(): +def parse_args(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument( + "--server", + default=DEFAULT_SERVER, + help=f"cuOpt server base URL (default: {DEFAULT_SERVER})", + ) + return parser.parse_args() + + +def server_ok(server): try: - r = requests.get(f"{SERVER}/cuopt/health", timeout=2) + r = requests.get(f"{server}/cuopt/health", timeout=2) return r.status_code == 200 except Exception: return False def main(): - if not server_ok(): + server = parse_args().server + + if not server_ok(server): print( "Server not running, skipping. Start with: python -m cuopt_server.cuopt_service --ip 0.0.0.0 --port 8000" ) @@ -55,15 +70,23 @@ def main(): } response = requests.post( - f"{SERVER}/cuopt/request", json=payload, headers=HEADERS + f"{server}/cuopt/request", + json=payload, + headers=HEADERS, + timeout=REQUEST_TIMEOUT, ) response.raise_for_status() req_id = response.json()["reqId"] + if not REQ_ID_PATTERN.fullmatch(req_id): + print(f"Unexpected reqId from server: {req_id!r}") + sys.exit(1) print(f"Submitted: {req_id}") for _ in range(60): response = requests.get( - f"{SERVER}/cuopt/solution/{req_id}", headers=HEADERS + f"{server}/cuopt/solution/{req_id}", + headers=HEADERS, + timeout=REQUEST_TIMEOUT, ) result = response.json() diff --git a/skills/cuopt-server-api-python/assets/pdp_basic/README.md b/skills/cuopt-server-api-python/assets/pdp_basic/README.md index ca6c174c6c..e5c8a5aabb 100644 --- a/skills/cuopt-server-api-python/assets/pdp_basic/README.md +++ b/skills/cuopt-server-api-python/assets/pdp_basic/README.md @@ -3,4 +3,4 @@ Pickup-delivery pairs: (0,1) and (2,3). Pickup must be visited before the corresponding delivery. **Requires:** cuOpt server running. **Run:** `python client.py` (exits 0 if server unreachable). -**Env:** `CUOPT_SERVER_URL` (default `http://localhost:8000`). +**Server URL:** `--server` (default `http://localhost:8000`). diff --git a/skills/cuopt-server-api-python/assets/pdp_basic/client.py b/skills/cuopt-server-api-python/assets/pdp_basic/client.py index 52e5290988..c3cb1fc0a9 100644 --- a/skills/cuopt-server-api-python/assets/pdp_basic/client.py +++ b/skills/cuopt-server-api-python/assets/pdp_basic/client.py @@ -3,26 +3,41 @@ """REST client for the cuOpt pickup-and-delivery (PDP) example. See README.md.""" -import os +import argparse +import re import sys import time import requests -SERVER = os.environ.get("CUOPT_SERVER_URL", "http://localhost:8000") +DEFAULT_SERVER = "http://localhost:8000" HEADERS = {"Content-Type": "application/json", "CLIENT-VERSION": "custom"} +REQUEST_TIMEOUT = 30 +REQ_ID_PATTERN = re.compile(r"[A-Za-z0-9_-]{1,64}") -def server_ok(): +def parse_args(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument( + "--server", + default=DEFAULT_SERVER, + help=f"cuOpt server base URL (default: {DEFAULT_SERVER})", + ) + return parser.parse_args() + + +def server_ok(server): try: - r = requests.get(f"{SERVER}/cuopt/health", timeout=2) + r = requests.get(f"{server}/cuopt/health", timeout=2) return r.status_code == 200 except Exception: return False def main(): - if not server_ok(): + server = parse_args().server + + if not server_ok(server): print( "Server not running, skipping. Start with: python -m cuopt_server.cuopt_service --ip 0.0.0.0 --port 8000" ) @@ -64,15 +79,23 @@ def main(): } response = requests.post( - f"{SERVER}/cuopt/request", json=payload, headers=HEADERS + f"{server}/cuopt/request", + json=payload, + headers=HEADERS, + timeout=REQUEST_TIMEOUT, ) response.raise_for_status() req_id = response.json()["reqId"] + if not REQ_ID_PATTERN.fullmatch(req_id): + print(f"Unexpected reqId from server: {req_id!r}") + sys.exit(1) print(f"Submitted: {req_id}") for _ in range(30): response = requests.get( - f"{SERVER}/cuopt/solution/{req_id}", headers=HEADERS + f"{server}/cuopt/solution/{req_id}", + headers=HEADERS, + timeout=REQUEST_TIMEOUT, ) result = response.json() diff --git a/skills/cuopt-server-api-python/assets/vrp_basic/README.md b/skills/cuopt-server-api-python/assets/vrp_basic/README.md index 84b46f7240..35c2d6859e 100644 --- a/skills/cuopt-server-api-python/assets/vrp_basic/README.md +++ b/skills/cuopt-server-api-python/assets/vrp_basic/README.md @@ -7,4 +7,4 @@ Submit a VRP with time windows to the cuOpt server and poll for the solution. **Run:** `python client.py` If the server is not reachable, the script exits 0 (skip). -**Env:** `CUOPT_SERVER_URL` (default `http://localhost:8000`). +**Server URL:** `--server` (default `http://localhost:8000`). diff --git a/skills/cuopt-server-api-python/assets/vrp_basic/client.py b/skills/cuopt-server-api-python/assets/vrp_basic/client.py index 9285eb05cd..dce9eb8ba9 100644 --- a/skills/cuopt-server-api-python/assets/vrp_basic/client.py +++ b/skills/cuopt-server-api-python/assets/vrp_basic/client.py @@ -5,29 +5,44 @@ REST client: VRP with time windows. Requires cuOpt server running. Usage: python client.py - Set CUOPT_SERVER_URL (default http://localhost:8000). Exits 0 if server unreachable (e.g. in CI without server). + Pass --server (default http://localhost:8000). Exits 0 if server unreachable (e.g. in CI without server). """ -import os +import argparse +import re import sys import time import requests -SERVER = os.environ.get("CUOPT_SERVER_URL", "http://localhost:8000") +DEFAULT_SERVER = "http://localhost:8000" HEADERS = {"Content-Type": "application/json", "CLIENT-VERSION": "custom"} +REQUEST_TIMEOUT = 30 +REQ_ID_PATTERN = re.compile(r"[A-Za-z0-9_-]{1,64}") -def server_ok(): +def parse_args(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument( + "--server", + default=DEFAULT_SERVER, + help=f"cuOpt server base URL (default: {DEFAULT_SERVER})", + ) + return parser.parse_args() + + +def server_ok(server): try: - r = requests.get(f"{SERVER}/cuopt/health", timeout=2) + r = requests.get(f"{server}/cuopt/health", timeout=2) return r.status_code == 200 except Exception: return False def main(): - if not server_ok(): + server = parse_args().server + + if not server_ok(server): print( "Server not running, skipping. Start with: python -m cuopt_server.cuopt_service --ip 0.0.0.0 --port 8000" ) @@ -71,15 +86,23 @@ def main(): } response = requests.post( - f"{SERVER}/cuopt/request", json=payload, headers=HEADERS + f"{server}/cuopt/request", + json=payload, + headers=HEADERS, + timeout=REQUEST_TIMEOUT, ) response.raise_for_status() req_id = response.json()["reqId"] + if not REQ_ID_PATTERN.fullmatch(req_id): + print(f"Unexpected reqId from server: {req_id!r}") + sys.exit(1) print(f"Submitted: {req_id}") for _ in range(30): response = requests.get( - f"{SERVER}/cuopt/solution/{req_id}", headers=HEADERS + f"{server}/cuopt/solution/{req_id}", + headers=HEADERS, + timeout=REQUEST_TIMEOUT, ) result = response.json() diff --git a/skills/cuopt-server-api-python/assets/vrp_simple/README.md b/skills/cuopt-server-api-python/assets/vrp_simple/README.md index f9de54a24c..77af6d1869 100644 --- a/skills/cuopt-server-api-python/assets/vrp_simple/README.md +++ b/skills/cuopt-server-api-python/assets/vrp_simple/README.md @@ -3,4 +3,4 @@ Simple VRP: 4 locations, 3 tasks, 2 vehicles. No time windows. **Requires:** cuOpt server running. **Run:** `python client.py` (exits 0 if server unreachable). -**Env:** `CUOPT_SERVER_URL` (default `http://localhost:8000`). +**Server URL:** `--server` (default `http://localhost:8000`). diff --git a/skills/cuopt-server-api-python/assets/vrp_simple/client.py b/skills/cuopt-server-api-python/assets/vrp_simple/client.py index 35f37f5c72..029752de39 100644 --- a/skills/cuopt-server-api-python/assets/vrp_simple/client.py +++ b/skills/cuopt-server-api-python/assets/vrp_simple/client.py @@ -6,26 +6,41 @@ Requires cuOpt server running. Exits 0 if server unreachable. """ -import os +import argparse +import re import sys import time import requests -SERVER = os.environ.get("CUOPT_SERVER_URL", "http://localhost:8000") +DEFAULT_SERVER = "http://localhost:8000" HEADERS = {"Content-Type": "application/json", "CLIENT-VERSION": "custom"} +REQUEST_TIMEOUT = 30 +REQ_ID_PATTERN = re.compile(r"[A-Za-z0-9_-]{1,64}") -def server_ok(): +def parse_args(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument( + "--server", + default=DEFAULT_SERVER, + help=f"cuOpt server base URL (default: {DEFAULT_SERVER})", + ) + return parser.parse_args() + + +def server_ok(server): try: - r = requests.get(f"{SERVER}/cuopt/health", timeout=2) + r = requests.get(f"{server}/cuopt/health", timeout=2) return r.status_code == 200 except Exception: return False def main(): - if not server_ok(): + server = parse_args().server + + if not server_ok(server): print( "Server not running, skipping. Start with: python -m cuopt_server.cuopt_service --ip 0.0.0.0 --port 8000" ) @@ -65,15 +80,23 @@ def main(): } response = requests.post( - f"{SERVER}/cuopt/request", json=payload, headers=HEADERS + f"{server}/cuopt/request", + json=payload, + headers=HEADERS, + timeout=REQUEST_TIMEOUT, ) response.raise_for_status() req_id = response.json()["reqId"] + if not REQ_ID_PATTERN.fullmatch(req_id): + print(f"Unexpected reqId from server: {req_id!r}") + sys.exit(1) print(f"Submitted: {req_id}") for _ in range(30): response = requests.get( - f"{SERVER}/cuopt/solution/{req_id}", headers=HEADERS + f"{server}/cuopt/solution/{req_id}", + headers=HEADERS, + timeout=REQUEST_TIMEOUT, ) result = response.json() diff --git a/skills/cuopt-server-api-python/evals/evals.json b/skills/cuopt-server-api-python/evals/evals.json index c4d43365bc..a741499cee 100644 --- a/skills/cuopt-server-api-python/evals/evals.json +++ b/skills/cuopt-server-api-python/evals/evals.json @@ -10,5 +10,91 @@ "Names cost_matrix_data, travel_time_matrix_data, task_data, fleet_data, solver_config as the VRPTW payload fields and flags the travel_time_matrix_data (REST) vs transit_time_matrix_data (Python) naming", "Does not produce a full runnable client script" ] + }, + { + "id": "srv-py-eval-002-rest-lp-payload-fields", + "question": "I want to solve a linear program through the cuOpt REST server instead of the Python API. What does the request body look like — which top-level fields do I send, and in what format is the constraint matrix? Name the fields only, no runnable script.", + "expected_skill": "cuopt-server-api-python", + "expected_script": null, + "ground_truth": "The agent explains that the LP payload is posted to the same POST /cuopt/request endpoint and that the constraint matrix is supplied in CSR (compressed sparse row) form. The top-level fields are csr_constraint_matrix (with offsets, indices, values), constraint_bounds (lower_bounds / upper_bounds, where infinite bounds are the strings \"ninf\" and \"inf\"), objective_data (coefficients), variable_bounds (lower_bounds / upper_bounds), maximize (boolean), and solver_config (e.g. time_limit). Does not produce a runnable client script.", + "expected_behavior": [ + "States that LP uses the same POST /cuopt/request → GET /cuopt/solution/{reqId} flow as routing", + "Names csr_constraint_matrix (offsets/indices/values), constraint_bounds, objective_data, variable_bounds, maximize, and solver_config", + "Notes that the constraint matrix is CSR and that infinite bounds are expressed as the strings \"inf\"/\"ninf\"", + "Does not produce a full runnable client script" + ] + }, + { + "id": "srv-py-eval-003-rest-milp-vs-lp", + "question": "I already have an LP working against the cuOpt REST server. What is the minimum change to the request body to make some of the variables integer?", + "expected_skill": "cuopt-server-api-python", + "expected_script": null, + "ground_truth": "The agent explains that MILP over REST uses the identical payload structure as LP, with one addition: a variable_types array, one entry per variable, using the values \"integer\", \"continuous\", or \"binary\". No endpoint change and no other field change is required.", + "expected_behavior": [ + "Identifies variable_types as the only field that must be added", + "Gives the per-variable values \"integer\", \"continuous\", \"binary\"", + "Confirms the endpoint and remaining payload fields are unchanged from LP" + ] + }, + { + "id": "srv-py-eval-004-qp-not-supported-over-rest", + "question": "Can I solve a quadratic program (quadratic objective, linear constraints) by POSTing to the cuOpt REST server? If not, what should I use instead?", + "expected_skill": "cuopt-server-api-python", + "expected_script": null, + "ground_truth": "The agent states that QP is not exposed over the REST server — the server supports routing, LP, and MILP only — and redirects the user to the cuOpt Python or C API for quadratic objectives. It does not invent a REST payload field for a quadratic objective term.", + "expected_behavior": [ + "States clearly that QP is not available through the REST API", + "Redirects to the Python or C API for QP", + "Does not fabricate a REST request field for the quadratic objective" + ] + }, + { + "id": "srv-py-eval-005-debug-422-field-names", + "question": "My POST to /cuopt/request keeps coming back 422. My payload has a transit_time_matrix_data key and fleet_data.capacities set to [[50], [50]] for my two vehicles. What is wrong?", + "expected_skill": "cuopt-server-api-python", + "expected_script": null, + "ground_truth": "The agent identifies two distinct errors. First, the REST API expects travel_time_matrix_data; transit_time_matrix_data is the Python-API name and is not a valid REST field. Second, capacities is indexed per capacity dimension, not per vehicle — for two vehicles sharing a single capacity dimension of 50 the correct value is [[50, 50]], whereas [[50], [50]] declares two capacity dimensions. It also suggests checking field names against the OpenAPI spec at /cuopt.yaml.", + "expected_behavior": [ + "Flags transit_time_matrix_data → travel_time_matrix_data", + "Explains that capacities is nested per capacity dimension, not per vehicle, and gives [[50, 50]] as the fix", + "Points at the OpenAPI spec (/cuopt.yaml) as the authority for field names" + ] + }, + { + "id": "srv-py-eval-006-docker-deployment", + "question": "How do I run the cuOpt server in Docker so I can reach it on port 8000 from my host? Give me the command and tell me how to choose the image tag.", + "expected_skill": "cuopt-server-api-python", + "expected_script": null, + "ground_truth": "The agent gives a docker run command that passes --gpus all (the server requires a GPU) and publishes the port with -p 8000:8000. For the tag it recommends latest-cu12 or latest-cu13 chosen to match the driver's CUDA major version (latest-cu13-ubi10 for a UBI10 base), in preference to CUDA+Python-specific tags such as latest-cuda12.9-py3.13 which track a single Python line and go stale. It notes that for production the tag should be pinned to a full release tag or an immutable @sha256 digest, since latest-* tags are mutable. It also mentions verifying with a request to /cuopt/health.", + "expected_behavior": [ + "Includes --gpus all and -p 8000:8000 in the docker run command", + "Recommends latest-cu12 / latest-cu13 selected by CUDA major version over CUDA+Python-specific tags", + "Notes that latest-* is mutable and should be pinned to a release tag or digest for production", + "Mentions verifying the container with GET /cuopt/health" + ] + }, + { + "id": "srv-py-eval-007-client-hardening", + "question": "I'm writing a Python client that submits a job to the cuOpt server and polls for the result. What should I be careful about in the polling loop?", + "expected_skill": "cuopt-server-api-python", + "expected_script": null, + "ground_truth": "The agent advises setting an explicit timeout on every requests call, calling raise_for_status() on the submit response, and validating the server-supplied reqId (e.g. against a pattern like [A-Za-z0-9_-]{1,64}) before interpolating it into the /cuopt/solution/{reqId} URL, since it is untrusted network input. It also notes that the poll budget and solver_config.time_limit need to be raised together, and that the server offers no built-in auth or TLS so the base URL should point at a trusted endpoint.", + "expected_behavior": [ + "Recommends an explicit timeout on every request", + "Recommends validating the server-returned reqId before using it in a URL path", + "Notes that the poll loop budget must accommodate solver_config.time_limit" + ] + }, + { + "id": "srv-py-eval-008-runnable-assets", + "question": "Does this skill ship any ready-to-run REST client examples? If so, what do they cover and how do I point one at a server that is not on localhost:8000?", + "expected_skill": "cuopt-server-api-python", + "expected_script": null, + "ground_truth": "The agent lists the bundled assets — vrp_simple (basic VRP), vrp_basic (VRP with time windows), pdp_basic (pickup and delivery), lp_basic (LP via CSR), and milp_basic (MILP) — and explains that each is a client.py using requests that accepts a --server flag defaulting to http://localhost:8000. It notes the scripts exit 0 when the server is unreachable so they are safe to run in CI.", + "expected_behavior": [ + "Names the five asset directories and what each demonstrates", + "States that the server URL is set with the --server command-line flag (default http://localhost:8000)", + "Mentions that the scripts exit 0 if the server is unreachable" + ] } ] diff --git a/skills/cuopt-server-api-python/skill-card.md b/skills/cuopt-server-api-python/skill-card.md index e4dfcae72c..15ed0b683f 100644 --- a/skills/cuopt-server-api-python/skill-card.md +++ b/skills/cuopt-server-api-python/skill-card.md @@ -7,16 +7,16 @@ This skill is ready for commercial/non-commercial use.
NVIDIA
### License/Terms of Use:
-Apache-2.0
+Apache 2.0
## Use Case:
-Developers and engineers deploying the cuOpt optimization server and calling its REST API from Python or curl for vehicle routing, LP, and MILP workloads.
+Developers and engineers deploying the NVIDIA cuOpt REST server or writing Python/curl clients against its optimization endpoints for routing, LP, and MILP problems.
### Deployment Geography for Use:
Global
## Requirements / Dependencies:
-**Requires API Key or External Credential:** [Not Specified]
-**Credential Type(s):** [None identified]
+**Requires API Key or External Credential:** [No]
+**Credential Type(s):** [None]
Do not include secrets in prompts/logs/output; use least-privilege credentials; rotate keys as appropriate.
@@ -32,47 +32,47 @@ Mitigation: Review and scan skill before deployment.
## Skill Output:
-**Output Type(s):** [API Calls, Shell commands, Code]
-**Output Format:** [Markdown with inline bash and Python code blocks]
+**Output Type(s):** [Shell commands, API Calls, Code, Configuration instructions]
+**Output Format:** [Markdown with inline code blocks]
**Output Parameters:** [1D]
**Other Properties Related to Output:** [None]
## Evaluation Agents Used:
-- `claude-code`
-- `codex`
+- Claude Code (`aws/anthropic/bedrock-claude-opus-4-8`)
+- Codex (`openai/openai/gpt-5.5`)
## Evaluation Tasks:
-Evaluated against 1 evaluation task (1 positive skill-activation case, 0 negative cases) in the NVSkills-Eval external profile, astra-sandbox environment.
+8 evaluation tasks (8 positive) from skill-evaluator-dataset-snapshot.
## Evaluation Metrics Used:
Reported benchmark dimensions:
-- Security: Checks whether skill-assisted execution avoids unsafe behavior such as secret leakage, destructive commands, or unauthorized access.
-- Correctness: Checks whether the agent follows the expected workflow and produces the correct final output.
-- Discoverability: Checks whether the agent loads the skill when relevant and avoids using it when irrelevant.
-- Effectiveness: Checks whether the agent performs measurably better with the skill than without it.
-- Efficiency: Checks whether the agent uses fewer tokens and avoids redundant work.
+- Security: Checks for unsafe operations, secret leakage, and unauthorized access.
+- Correctness: Verifies final-answer correctness against reference answers.
+- Discoverability: Whether the expected skill was found and executed when needed.
+- Effectiveness: Whether the skill helped complete the user's goal and expected workflow.
+- Efficiency: Routing quality, workspace-aware skill reads, and productive tool use.
Underlying evaluation signals used in this run:
-- `security`: Checks for unsafe operations, secret leakage, and unauthorized access.
-- `skill_execution`: Verifies that the agent loaded the expected skill and workflow.
-- `skill_efficiency`: Checks routing quality, decoy avoidance, and redundant tool usage.
-- `accuracy`: Grades final-answer correctness against the reference answer.
-- `goal_accuracy`: Checks whether the overall user task completed successfully.
-- `behavior_check`: Verifies expected behavior steps, including safety expectations.
-- `token_efficiency`: Compares token usage with and without the skill.
+- `security`: Detects unsafe operations, secret leakage, and unauthorized access.
+- `skill_execution`: Whether the expected skill was found and executed.
+- `skill_efficiency`: Routing quality, workspace-aware skill reads, and productive tool use.
+- `accuracy`: Final-answer correctness against the reference answer.
+- `goal_accuracy`: Whether the user's goal was achieved.
+- `behavior_check`: Whether the expected workflow behavior was followed.
## Evaluation Results:
-| Dimension | Num | `claude-code` | `codex` | -|---|---:|---:|---:| -| Security | 1 | 100% (+0%) | 100% (+0%) | -| Correctness | 1 | 100% (+70%) | 97% (+35%) | -| Discoverability | 1 | 100% (+100%) | 97% (+69%) | -| Effectiveness | 1 | 100% (+56%) | 100% (+40%) | -| Efficiency | 1 | 95% (+68%) | 95% (+67%) | +| Measure | Claude Code (Baseline → Skill Uplift) | Codex (Baseline → Skill Uplift) | +|---|---:|---:| +| Overall | 64% → 98% (+35 points) | 62% → 96% (+34 points) | +| Security | 100% → 100% (±0 points) | 69% → 100% (+31 points) | +| Correctness | 75% → 100% (+25 points) | 88% → 98% (+10 points) | +| Discoverability | 44% → 100% (+56 points) | 45% → 93% (+48 points) | +| Effectiveness | 56% → 95% (+38 points) | 72% → 88% (+15 points) | +| Efficiency | 44% → 97% (+53 points) | 37% → 100% (+63 points) | ## Skill Version(s):
26.10.00 (source: frontmatter)
diff --git a/skills/cuopt-server-api-python/skill.oms.sig b/skills/cuopt-server-api-python/skill.oms.sig index 7a336c4da8..42caf46700 100644 --- a/skills/cuopt-server-api-python/skill.oms.sig +++ b/skills/cuopt-server-api-python/skill.oms.sig @@ -1 +1 @@ -{"mediaType":"application/vnd.dev.sigstore.bundle.v0.3+json","verificationMaterial":{"x509CertificateChain":{"certificates":[{"rawBytes":"MIICgzCCAgmgAwIBAgIUKIyS7SxNteQIiWzK1dWj85E6520wCgYIKoZIzj0EAwMwVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwHhcNMjYwNDAxMDAwMDAwWhcNMjgwNDIyMTUzMzA5WjBUMQswCQYDVQQGEwJVUzEbMBkGA1UECgwSTlZJRElBIENvcnBvcmF0aW9uMSgwJgYDVQQDDB9OVklESUEgQWdlbnQgU2tpbGxzIFNpZ25pbmcgMDAxMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEYoRM9bQl/dGlwSRNi6bTpIJUXH8Nv9GciP6LSflJYYMLCc296kpyuTSsk5ddbAWiDcFX3C/ydX3jwc+qCLYP6uHy9XphyLjOQ27Yb2J6rBLVtRBS1mgGco/Gr7fL6ODco4GaMIGXMB0GA1UdDgQWBBRQ/5ZW3nJ6lmo9SVk7I15o7UGmpTAfBgNVHSMEGDAWgBRPGpILxMBBleJSsBGjrMKsby1CgjAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQEAwIHgDA3BggrBgEFBQcBAQQrMCkwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vY3NwLm5kaXMubnZpZGlhLmNvbTAKBggqhkjOPQQDAwNoADBlAjAUygu/GiOCIXrgGr4SmLgeEVDcEitfFUv7ALbvLVGVyMysB3mxmO/uInZfXzWcJZsCMQDxuoxj4ZmO30jhkPIcCxGFCOvnUsnfU3TfGcouYm4M6iRpbKvtVnHPiy4bi6pcKf0="},{"rawBytes":"MIICiDCCAg6gAwIBAgIUZsIuSv9NkpJCNqtYEfCouVv5BzowCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAASI72cR3ctKGg4VWnB3bNja6g1Z2PnOmFEopkPof+QeIcPk9rT+g9MjJnq51EQXL93a7C2GJ9J985G4o2V85VD7wJ1RaXhluHW2rf3y8bQGeAYaKMr5s/hUgn+M3/9WlWejgaAwgZ0wHQYDVR0OBBYEFE8akgvEwEGV4lKwEaOswqxvLUKCMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgEGMDcGCCsGAQUFBwEBBCswKTAnBggrBgEFBQcwAYYbaHR0cDovL29jc3AubmRpcy5udmlkaWEuY29tMAoGCCqGSM49BAMDA2gAMGUCMQCeIMMfAbyzPDacw2MxG+Yt1cikrJX/DVxiGfXuHmkkXn6VgSzE79+lkqDErpVO2gYCMCNEColOyvUvkzZGUEI1hQ3PfMgi3FIo9tHoBKMw4/wGBLFpu/0ubtmbBXM6/UMOEw=="},{"rawBytes":"MIICRTCCAcygAwIBAgIUeJdY3rV86EdvFmG7L8LJBsyQFYkwCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTB2MBAGByqGSM49AgEGBSuBBAAiA2IABAYpiXCDjJ9NT2eSDhyHJVSw1Tbze18cGG2F/578oWvHxg23eQAhNRYdq88i1iOshZSO6C29doKui5Xpmo/7Ctw9Sx4PP2RzOmIuOLCuTdNtKcTRwi4GEsd5BAFvWj42M6NjMGEwHQYDVR0OBBYEFItnoAjjfuCEUvzyvWyI2vOGvwPjMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMAoGCCqGSM49BAMDA2cAMGQCMCwtAjWLaNwgGWNCgdyNoTyvNhqWRECRJV2r3+7w8g0PL6NHLOsbkgE09BH95h8XlgIwTaQmbbUh2ChAJ5TA1wRiVDnCcvbzHlZl2jM2FcwQQZlk19LOAbyGMRixbu2Ww/rj"}]},"tlogEntries":[]},"dsseEnvelope":{"payload":"ewogICJfdHlwZSI6ICJodHRwczovL2luLXRvdG8uaW8vU3RhdGVtZW50L3YxIiwKICAic3ViamVjdCI6IFsKICAgIHsKICAgICAgIm5hbWUiOiAiY3VvcHQtc2VydmVyLWFwaS1weXRob24iLAogICAgICAiZGlnZXN0IjogewogICAgICAgICJzaGEyNTYiOiAiNDJhZWQzNGVkOWU0Yzg4Y2ZhZmE1YWQzZWM3MzIwZjY5ZDAzMTJkYjVkZjkwMTZlNmVhZTNlNDIzYzI1M2FmNSIKICAgICAgfQogICAgfQogIF0sCiAgInByZWRpY2F0ZVR5cGUiOiAiaHR0cHM6Ly9tb2RlbF9zaWduaW5nL3NpZ25hdHVyZS92MS4wIiwKICAicHJlZGljYXRlIjogewogICAgInJlc291cmNlcyI6IFsKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJCRU5DSE1BUksubWQiLAogICAgICAgICJkaWdlc3QiOiAiYmI3NDQ3NGQwMmExMzAxMTA2YTA2ZWE1ZTBhNTI3MDE0ODQ2YjA1MGQxODQzOTk4NDQ1MGRlZGM3MmRmZTRkNCIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJTS0lMTC5tZCIsCiAgICAgICAgImRpZ2VzdCI6ICI5ZjdmMTkxNzA4MDc5YjdlYTgwZjgwYzRlMTZjYjBhMDNkMTc4MmMyYmZhNjM4MDg3NTBlNGNhNTBkMzhmMTE5IgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogImFzc2V0cy9SRUFETUUubWQiLAogICAgICAgICJkaWdlc3QiOiAiYTgzY2NlYjEwMWZlYjI4OTUzYmU5ZmEwNjg5ZjczMTc0Njc0OTE0ZTZlY2FhMmNlYTkzZGYxMDJlMDAxMTRmZiIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvbHBfYmFzaWMvUkVBRE1FLm1kIiwKICAgICAgICAiZGlnZXN0IjogIjFjNmU4OWVlZWU4OGRlN2QyOTY4YzY4NGNmOGI0NWJhOWJlMGMyZTQyNDFlMDk0ZjMwZjcyYzY3MDM0ZjFmN2IiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgIm5hbWUiOiAiYXNzZXRzL2xwX2Jhc2ljL2NsaWVudC5weSIsCiAgICAgICAgImRpZ2VzdCI6ICI2YTZmZjUyZmVjNWM4ZmMyZDVjNTJmMjhiOTBkNjNiODU1YjQ1OTVlODQ3MWZlNWQyOGE1MDcxOTAwMDc2ZWViIgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogImFzc2V0cy9taWxwX2Jhc2ljL1JFQURNRS5tZCIsCiAgICAgICAgImRpZ2VzdCI6ICJmYWE0ZWUxMGE2NTg1ODM5YWViNDQ5MmUwNzcxMzYwMzhlZWViMGJjZGViYjkyYTE4YjI0ZWFhNWRlZDJlNjU5IgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogImFzc2V0cy9taWxwX2Jhc2ljL2NsaWVudC5weSIsCiAgICAgICAgImRpZ2VzdCI6ICJlNmRjZWQxZWM1ZGM2NzJkMzNiMjgzZTA5MmIzOTA0YTQ4NzBhZTEwNWZiMTdlMzkwNDMwZDU4M2Y3MjQwOWE3IgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogImFzc2V0cy9wZHBfYmFzaWMvUkVBRE1FLm1kIiwKICAgICAgICAiZGlnZXN0IjogIjVjMGI3ZTNjMzVlYjExMWY5MjQ2ZDQ5YTgwMjIxMDhiNGQ0ZTRlZTI4NGJlZjI4M2Y1YWEyMTIwZmJjM2VkMGQiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgIm5hbWUiOiAiYXNzZXRzL3BkcF9iYXNpYy9jbGllbnQucHkiLAogICAgICAgICJkaWdlc3QiOiAiOTgxYmNhMDk1MWFjOWQ4NTAwNzRiY2MzMDViYTZmMjFhNzA5NDVmMDgwYzgyOTA5NDdjZTc4NDAyOGJlMTlmZSIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvdnJwX2Jhc2ljL1JFQURNRS5tZCIsCiAgICAgICAgImRpZ2VzdCI6ICI2OTI4MDc0NDEzZGZlMWVhNDE1YmYxZGVlNzljZWJjMTg2NTQ2ZmVjYThkYzQwZmVlMWU0MmEyOTY1MWU1MTY0IgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogImFzc2V0cy92cnBfYmFzaWMvY2xpZW50LnB5IiwKICAgICAgICAiZGlnZXN0IjogIjYzZDFlYjZkZjBjODQxNzcwOTM4MDRhNjYxODlmYjZhYWU1MGE3ZWE0ZDdjZGI3NDI1ZTdhNjE5Y2NjMGIxMzYiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgIm5hbWUiOiAiYXNzZXRzL3ZycF9zaW1wbGUvUkVBRE1FLm1kIiwKICAgICAgICAiZGlnZXN0IjogIjZlYzQ1YmI1YTVlMGYxYTEyMmYyNDExNzkxODdjNGU2MDhjYmNhODhkODA4MWRhNDIyOTE2NGQyODEzMTZiNWMiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgIm5hbWUiOiAiYXNzZXRzL3ZycF9zaW1wbGUvY2xpZW50LnB5IiwKICAgICAgICAiZGlnZXN0IjogImY3ZTFhZTQ5NjA3Yzk0ZTNjODE4YjE2OTAwNmEzODM0Yzc2MmMxNThmZmMyZjQzNjI5NWQxODBjODI1OWVlNTkiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgIm5hbWUiOiAiZXZhbHMvZXZhbHMuanNvbiIsCiAgICAgICAgImRpZ2VzdCI6ICJiNjk1OWJhYjEwM2E1YWQzYzY4NjY1N2NlMGQ0NWQ3OWVhYTg5OWI2ZjNiOTZkMTBkODcyMWJmZjhlZjM2NzE5IgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogInNraWxsLWNhcmQubWQiLAogICAgICAgICJkaWdlc3QiOiAiZDA2ZTc2ZmE4ZDllMDE2NTMzZDBiNDAwNDRlZTcxMjRlMzY2NGI4MzhhZDRlMTAyMDgzZGQ3ODkxNTRlZGMwZCIKICAgICAgfQogICAgXSwKICAgICJzZXJpYWxpemF0aW9uIjogewogICAgICAiaGFzaF90eXBlIjogInNoYTI1NiIsCiAgICAgICJhbGxvd19zeW1saW5rcyI6IGZhbHNlLAogICAgICAibWV0aG9kIjogImZpbGVzIiwKICAgICAgImlnbm9yZV9wYXRocyI6IFsKICAgICAgICAiLmdpdCIsCiAgICAgICAgIi5naXRhdHRyaWJ1dGVzIiwKICAgICAgICAiLmdpdGlnbm9yZSIsCiAgICAgICAgIi5naXRodWIiCiAgICAgIF0KICAgIH0KICB9Cn0=","payloadType":"application/vnd.in-toto+json","signatures":[{"sig":"MGUCMAa1Jdb+1CJ9sMsUOcXOh16duwRQhqlEiZSTVilJ+GovK1FjYC5xbormKRaUi6PZfQIxAKgbOJfCp0SqR7g46TRUldmvHLaNeE4gZp2ZBw8d90XKuhjT/mO7998tKfIkst3/dA==","keyid":""}]}} \ No newline at end of file +{"mediaType":"application/vnd.dev.sigstore.bundle.v0.3+json","verificationMaterial":{"x509CertificateChain":{"certificates":[{"rawBytes":"MIICgzCCAgmgAwIBAgIUKIyS7SxNteQIiWzK1dWj85E6520wCgYIKoZIzj0EAwMwVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwHhcNMjYwNDAxMDAwMDAwWhcNMjgwNDIyMTUzMzA5WjBUMQswCQYDVQQGEwJVUzEbMBkGA1UECgwSTlZJRElBIENvcnBvcmF0aW9uMSgwJgYDVQQDDB9OVklESUEgQWdlbnQgU2tpbGxzIFNpZ25pbmcgMDAxMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEYoRM9bQl/dGlwSRNi6bTpIJUXH8Nv9GciP6LSflJYYMLCc296kpyuTSsk5ddbAWiDcFX3C/ydX3jwc+qCLYP6uHy9XphyLjOQ27Yb2J6rBLVtRBS1mgGco/Gr7fL6ODco4GaMIGXMB0GA1UdDgQWBBRQ/5ZW3nJ6lmo9SVk7I15o7UGmpTAfBgNVHSMEGDAWgBRPGpILxMBBleJSsBGjrMKsby1CgjAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQEAwIHgDA3BggrBgEFBQcBAQQrMCkwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vY3NwLm5kaXMubnZpZGlhLmNvbTAKBggqhkjOPQQDAwNoADBlAjAUygu/GiOCIXrgGr4SmLgeEVDcEitfFUv7ALbvLVGVyMysB3mxmO/uInZfXzWcJZsCMQDxuoxj4ZmO30jhkPIcCxGFCOvnUsnfU3TfGcouYm4M6iRpbKvtVnHPiy4bi6pcKf0="},{"rawBytes":"MIICiDCCAg6gAwIBAgIUZsIuSv9NkpJCNqtYEfCouVv5BzowCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAASI72cR3ctKGg4VWnB3bNja6g1Z2PnOmFEopkPof+QeIcPk9rT+g9MjJnq51EQXL93a7C2GJ9J985G4o2V85VD7wJ1RaXhluHW2rf3y8bQGeAYaKMr5s/hUgn+M3/9WlWejgaAwgZ0wHQYDVR0OBBYEFE8akgvEwEGV4lKwEaOswqxvLUKCMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgEGMDcGCCsGAQUFBwEBBCswKTAnBggrBgEFBQcwAYYbaHR0cDovL29jc3AubmRpcy5udmlkaWEuY29tMAoGCCqGSM49BAMDA2gAMGUCMQCeIMMfAbyzPDacw2MxG+Yt1cikrJX/DVxiGfXuHmkkXn6VgSzE79+lkqDErpVO2gYCMCNEColOyvUvkzZGUEI1hQ3PfMgi3FIo9tHoBKMw4/wGBLFpu/0ubtmbBXM6/UMOEw=="},{"rawBytes":"MIICRTCCAcygAwIBAgIUeJdY3rV86EdvFmG7L8LJBsyQFYkwCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTB2MBAGByqGSM49AgEGBSuBBAAiA2IABAYpiXCDjJ9NT2eSDhyHJVSw1Tbze18cGG2F/578oWvHxg23eQAhNRYdq88i1iOshZSO6C29doKui5Xpmo/7Ctw9Sx4PP2RzOmIuOLCuTdNtKcTRwi4GEsd5BAFvWj42M6NjMGEwHQYDVR0OBBYEFItnoAjjfuCEUvzyvWyI2vOGvwPjMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMAoGCCqGSM49BAMDA2cAMGQCMCwtAjWLaNwgGWNCgdyNoTyvNhqWRECRJV2r3+7w8g0PL6NHLOsbkgE09BH95h8XlgIwTaQmbbUh2ChAJ5TA1wRiVDnCcvbzHlZl2jM2FcwQQZlk19LOAbyGMRixbu2Ww/rj"}]},"tlogEntries":[]},"dsseEnvelope":{"payload":"ewogICJfdHlwZSI6ICJodHRwczovL2luLXRvdG8uaW8vU3RhdGVtZW50L3YxIiwKICAic3ViamVjdCI6IFsKICAgIHsKICAgICAgIm5hbWUiOiAiY3VvcHQtc2VydmVyLWFwaS1weXRob24iLAogICAgICAiZGlnZXN0IjogewogICAgICAgICJzaGEyNTYiOiAiMjBiOWIxMDIxNDNiOGRkZGFiMDkzODI5ZWNiMWMwYWM5M2IxYmM0ZDkxOTE3YmU3MzAyYTM2YjVhNjgxYzJhOCIKICAgICAgfQogICAgfQogIF0sCiAgInByZWRpY2F0ZVR5cGUiOiAiaHR0cHM6Ly9tb2RlbF9zaWduaW5nL3NpZ25hdHVyZS92MS4wIiwKICAicHJlZGljYXRlIjogewogICAgInNlcmlhbGl6YXRpb24iOiB7CiAgICAgICJhbGxvd19zeW1saW5rcyI6IGZhbHNlLAogICAgICAiaGFzaF90eXBlIjogInNoYTI1NiIsCiAgICAgICJtZXRob2QiOiAiZmlsZXMiLAogICAgICAiaWdub3JlX3BhdGhzIjogWwogICAgICAgICIuZ2l0aWdub3JlIiwKICAgICAgICAiLmdpdGh1YiIsCiAgICAgICAgIi5naXRhdHRyaWJ1dGVzIiwKICAgICAgICAiLmdpdCIKICAgICAgXQogICAgfSwKICAgICJyZXNvdXJjZXMiOiBbCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgIm5hbWUiOiAiQkVOQ0hNQVJLLm1kIiwKICAgICAgICAiZGlnZXN0IjogIjY1YzdiNDliYTBjYTA3ZWJjZDg4MjQ3ZTcyYzUyZjkxYjIwMzAwMjk2ZDE1OGJhMGQxY2ViMDA3OWJmM2RlODAiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgIm5hbWUiOiAiU0tJTEwubWQiLAogICAgICAgICJkaWdlc3QiOiAiYTY2M2E2MjNhYTM4NWRhMjU2ODBmNDU0NDA2M2Q3MTJmZDMzNmRjMzdlYmJjYTJmMjQ1MjIyNzQzODRkMTgxNyIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvUkVBRE1FLm1kIiwKICAgICAgICAiZGlnZXN0IjogImJkZTE4ODkwZmQ5MTE5ZjQxYWY4NGRlYWU0ODk0ZjFiYjhmNDE0MmMxNGNhMTA5NjBkY2JkN2Y4MGQ3NmMzZmEiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgIm5hbWUiOiAiYXNzZXRzL2xwX2Jhc2ljL1JFQURNRS5tZCIsCiAgICAgICAgImRpZ2VzdCI6ICI4ZWNmZjRjODQ3YzVhNTU0YzM0ZTVhMDdjNGViMGM4ZTQyMmFhYjlhZjMxMDMzNGVmMjA3MDRhMmRhMjJkYTEzIgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogImFzc2V0cy9scF9iYXNpYy9jbGllbnQucHkiLAogICAgICAgICJkaWdlc3QiOiAiNTRhN2E4NTExZGM0MzA4OGZkMjdkMDcxODIyMTBiNDRiMzc0NDk2YzM5MWMwMmY4YWM2YTUyNzYwZjVhMDI2MCIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvbWlscF9iYXNpYy9SRUFETUUubWQiLAogICAgICAgICJkaWdlc3QiOiAiZDBmMWNkNTllMzY5NTFlYTdhZmEyZWRjNjkyMjNhNGY0MGYyNWUyYWU4MWQ5M2MyZDA5ZjVhMTQxMzAyYjZlYiIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvbWlscF9iYXNpYy9jbGllbnQucHkiLAogICAgICAgICJkaWdlc3QiOiAiN2E3NWQ1ZmNiM2JjZTI2YzQyMzgwMWJhMjkyZjc0MmJhOTEwNzZkYmI1MDM4YzdkZWMzYWQ5NWYwNmJlNWI4ZCIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvcGRwX2Jhc2ljL1JFQURNRS5tZCIsCiAgICAgICAgImRpZ2VzdCI6ICI4ZTc3MTI0MjlkZjQxZjQ1YjA2Nzk2Yjg3ZjliZjEwYzk1MTc2ZTkxYTM0NzA0YjFlMGQ0ZGQzNzdjZDBlMDcyIgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogImFzc2V0cy9wZHBfYmFzaWMvY2xpZW50LnB5IiwKICAgICAgICAiZGlnZXN0IjogIjljYzg4NDBiY2E3OWZhYTUxOGRmOTNkNzZkY2YzMGRkOTNiYTQzZTQwMjViY2M5NDEzMWU1Mjg5YjU4MDAyZGEiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgIm5hbWUiOiAiYXNzZXRzL3ZycF9iYXNpYy9SRUFETUUubWQiLAogICAgICAgICJkaWdlc3QiOiAiNzViZjRmN2ZlNDYwZjI3NWEwZTFiMTEzZTU2MGRjNjdmOWYzYTBmMjk3ZWNkNGRjZWZlYzc2OWEzMTI2OWVmMCIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvdnJwX2Jhc2ljL2NsaWVudC5weSIsCiAgICAgICAgImRpZ2VzdCI6ICJiNTFlMTQxNzZkYzJkYTQ4NDY1NzU4NTA4YTIzYTRiOTc1Y2FkYTNmMzcwYzM3ZWU1YWViZGI2MjlkODY1NDk4IgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogImFzc2V0cy92cnBfc2ltcGxlL1JFQURNRS5tZCIsCiAgICAgICAgImRpZ2VzdCI6ICI1ZWI1ODA1MTk2NDUzM2YxNGUyOGM5YzQwY2Q2N2RkZmU1OTUwNTdiMDE3ZmQ1NjEzYzZiZGRkYWNmZGI2YzdkIgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogImFzc2V0cy92cnBfc2ltcGxlL2NsaWVudC5weSIsCiAgICAgICAgImRpZ2VzdCI6ICI5MTg3YTEwYjk3M2U3N2E3ZGQzNmU3MjIzZDgwMTgxZGYyNWFmNDQ0ZTU3NWE4MmVlNTg3NDY3NTliN2Y3MGRkIgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogImV2YWxzL2V2YWxzLmpzb24iLAogICAgICAgICJkaWdlc3QiOiAiYTg2YmYzZWNkMDQ0MDk3OWYyNjUxMDZkNmNiNjMxNDU4MDIwMzllNDc3NTgzZGMzNGFlYzc0NzVhNTEzOGJhMSIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJza2lsbC1jYXJkLm1kIiwKICAgICAgICAiZGlnZXN0IjogIjAxNjc1MWE2ZmQ2Y2Y4YmQ0MzY0NjNhMjgxODAwN2VkMjY4ZWQ3ODJmNjRiYzhkMjMxY2JmNzZlNWEyMzRhOWMiCiAgICAgIH0KICAgIF0KICB9Cn0=","payloadType":"application/vnd.in-toto+json","signatures":[{"sig":"MGUCMQD4ZlBWRuqpc8UyqsGogxAUgYnjEpwDQk9YfFyXVGO0FTzk1I1w+GuuF0coWhxcY9oCMHgWT8odcyi9LYg9A8z0NUdLwSeBfDOJ3x0UY9XVdI0eBfof5LqdgIXxLI5YGVVqCA==","keyid":""}]}} \ No newline at end of file