feat(iron-swarm): add the nemo-iron-swarm plugin (agent red-teaming and hardening) - #1037
Open
koralchapnik wants to merge 56 commits into
Open
feat(iron-swarm): add the nemo-iron-swarm plugin (agent red-teaming and hardening)#1037koralchapnik wants to merge 56 commits into
koralchapnik wants to merge 56 commits into
Conversation
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughIron Swarm adds a plugin with REST APIs, jobs, SDK and CLI support, durable run events, manifest and fileset handling, benign-suite synthesis, defense validation, and feature-gated Studio workflows. ChangesIron Swarm feature
Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
Note
Due to the large number of review comments, Critical severity comments were prioritized as inline comments.
🟠 Major comments (19)
plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/provisioning.py-71-73 (1)
71-73: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winRedact the index URL before echoing it.
config.index_urlcan embed credentials (https://user:AKCp8token@host/simple).setupprints it verbatim into terminal scrollback and CI logs.doctoralready avoids this throughchecks.redact_index_url, andtest_doctor_never_prints_an_embedded_tokenlocks that behavior in. Use the same helper here.🔒️ Proposed fix
+from nemo_iron_swarm_plugin.cli.checks import redact_index_url ... if config.index_url: - typer.echo(f" using extra index {config.index_url}") + typer.echo(f" using extra index {redact_index_url(config.index_url)}") install_cmd += ["--index", config.index_url]🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/provisioning.py` around lines 71 - 73, Update the index URL output in the provisioning setup flow to pass config.index_url through the existing checks.redact_index_url helper before typer.echo, while continuing to use the original URL in install_cmd for installation.web/packages/studio/src/routes/NewIronSwarmManifestRoute/index.tsx-104-115 (1)
104-115: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winThe inspect effect overwrites operator edits and can apply a stale response.
The effect re-runs on every change of
source,selectedAgent, orworkspace. Two problems follow:
- If the operator edits
portorsecretsand then toggles the source control back toagent, the effect re-fires with the same agent and overwrites those edits.- If the operator switches agents quickly, an earlier in-flight response can resolve last and write stale values.
Track the agent the response belongs to and skip writes when it no longer matches.
🐛 Proposed fix
const selectedAgent = watch('agent'); const inspectAgent = useInspectAgent(); const { mutate: runInspectAgent } = inspectAgent; + const inspectedAgent = useRef<string | undefined>(undefined); useEffect(() => { if (source !== 'agent' || !selectedAgent) return; + if (inspectedAgent.current === selectedAgent) return; + inspectedAgent.current = selectedAgent; runInspectAgent( { workspace, agent: selectedAgent }, { onSuccess: (facts) => { + if (inspectedAgent.current !== selectedAgent) return; setValue('port', String(facts.port)); setValue('secrets', facts.secrets.join(', ')); }, } ); }, [source, selectedAgent, workspace, runInspectAgent, setValue]);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/routes/NewIronSwarmManifestRoute/index.tsx` around lines 104 - 115, Update the inspect effect around runInspectAgent so it does not overwrite operator edits when source toggles back to the same agent, and ignores responses for agents that are no longer selected. Track the agent associated with each inspection request, only apply setValue updates when that agent still matches the current selectedAgent, and preserve the existing source, workspace, and request behavior.web/packages/studio/src/components/ironSwarm/useSanityCheck.ts-112-122 (1)
112-122: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winUnvalidated
JSON.parsecan crash the report view.Line 117 asserts the parsed blob is a
ValidationReport.SanityCheckReportdestructuresreport.summaryand readssummary.attacks_blocked. If the result file is truncated, empty, or shaped differently, the render throws instead of showing an error.Validate the parsed payload before returning it, or default the missing fields.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/ironSwarm/useSanityCheck.ts` around lines 112 - 122, Update the queryFn in useSanityCheck to safely handle malformed, empty, or differently shaped validation-result JSON before returning a ValidationReport. Validate the parsed payload’s required report and summary fields, or supply defaults for missing fields, so SanityCheckReport can access summary.attacks_blocked without throwing during render.web/packages/studio/src/components/ironSwarm/useRunWarGame.ts-20-31 (1)
20-31: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winOne failed poll aborts the flow silently.
ironSwarmListRunsis not wrapped in try/catch. A single transient failure rejects the promise, skips the fallbacknavigate, and produces an unhandled rejection because the caller usesvoid. The user sees the success toast and stays on the manifest page.Also add an abort guard so the loop stops when the component unmounts.
Proposed fix
const openRunForJob = async (jobName: string): Promise<void> => { for (let attempt = 0; attempt < 60; attempt++) { - const { data } = await ironSwarmListRuns(workspace, { sort: '-created_at', page_size: 20 }); - const run = (data as IronSwarmRun[] | undefined)?.find((r) => r.job_id === jobName); - if (run?.name) { - navigate(getIronSwarmRunDetailsRoute(workspace, run.name)); - return; - } + try { + const { data } = await ironSwarmListRuns(workspace, { sort: '-created_at', page_size: 20 }); + const run = (data as IronSwarmRun[] | undefined)?.find((r) => r.job_id === jobName); + if (run?.name) { + navigate(getIronSwarmRunDetailsRoute(workspace, run.name)); + return; + } + } catch { + // transient failure — keep polling until the window expires + } await new Promise((resolve) => setTimeout(resolve, 500)); } navigate(getIronSwarmRunListRoute(workspace)); };🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/ironSwarm/useRunWarGame.ts` around lines 20 - 31, Update openRunForJob to catch transient errors from ironSwarmListRuns and continue polling so failures do not bypass the final fallback navigation. Add an unmount abort guard shared with the component lifecycle, checking it during each polling iteration and before navigation, and ensure the guard is released on cleanup.web/packages/studio/src/routes/IronSwarmManifestDetailRoute/index.tsx-86-93 (1)
86-93: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winCSV formula injection in the exported requests.csv.
escapeCsvonly quotes",,and\n. A value that begins with=,+,-,@, tab, or CR is written raw. The suite content is LLM-generated and operator-editable. When the downloaded file is opened in Excel or Sheets, such a value is evaluated as a formula.Prefix those values with a single quote or
\t, and quote the field.Proposed fix
-const escapeCsv = (value: string): string => - /[",\n]/.test(value) ? `"${value.replace(/"/g, '""')}"` : value; +const FORMULA_PREFIX = /^[=+\-@\t\r]/; +const escapeCsv = (value: string): string => { + const safe = FORMULA_PREFIX.test(value) ? `'${value}` : value; + return /["',\r\n]/.test(safe) ? `"${safe.replace(/"/g, '""')}"` : safe; +};🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/routes/IronSwarmManifestDetailRoute/index.tsx` around lines 86 - 93, Update escapeCsv to prevent formula injection in exported requests.csv: detect values beginning with =, +, -, @, tab, or carriage return, prefix them with a single quote (or tab), and ensure the resulting field is quoted. Preserve existing escaping of quotes, commas, and newlines, and keep toRequestsCsv using escapeCsv for every field.web/packages/studio/src/components/ironSwarm/useSanityCheck.ts-163-176 (1)
163-176: 🚀 Performance & Scalability | 🟠 Major | ⚡ Quick win
useLatestSanityCheckJobpolls forever.
refetchIntervalis a constant. The hook keeps listing 50 runs on every interval for as long as the Harden tab is open, including after the job is found and after it completes. Stop the interval once ajob_idis returned.Proposed fix
- refetchInterval: JOB_POLLING_INTERVAL_MS, + refetchInterval: (query) => (query.state.data ? false : JOB_POLLING_INTERVAL_MS),🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/ironSwarm/useSanityCheck.ts` around lines 163 - 176, Update useLatestSanityCheckJob so refetchInterval becomes conditional on the query result: continue polling while no job_id has been found, and disable polling once a job_id is returned. Preserve the existing query key, enabled condition, and latest-run lookup behavior.web/packages/studio/src/components/ironSwarm/useMitigations.ts-221-229 (1)
221-229: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winUnmemoized
defensesarray drives a downstream render loop.useMitigationsallocates a new array wheneverquery.datais undefined, andHardenPaneluses that array as an effect dependency that callssetSelectedwith a newSet.
web/packages/studio/src/components/ironSwarm/useMitigations.ts#L221-L229: wrapdefensesandrecommendationsinuseMemokeyed onquery.data.web/packages/studio/src/components/ironSwarm/HardenPanel.tsx#L162-L165: key the selection-reset effect on the joined defense ids instead of the array identity.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/ironSwarm/useMitigations.ts` around lines 221 - 229, Memoize the defenses and recommendations values in useMitigations, keyed by query.data, so unchanged data preserves array identity. In web/packages/studio/src/components/ironSwarm/HardenPanel.tsx lines 162-165, update the selection-reset effect dependency to use joined defense IDs rather than the defenses array identity.web/packages/studio/src/components/ironSwarm/useGenerateBenignSuite.ts-59-69 (1)
59-69: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winBoth run-resolution paths call
ironSwarmListRunswithout error handling or cleanup. A rejection becomes an unhandled promise rejection, and state setters can fire after unmount.
web/packages/studio/src/components/ironSwarm/useGenerateBenignSuite.ts#L59-L69: wrap the request intry/catch, and toast when the run never resolves sostartingdoes not staytrue.web/packages/studio/src/components/ironSwarm/useGenerateBenignSuite.ts#L78-L94: wrap the async IIFE intry/catchand add an ignore flag in the effect cleanup before callingsetRunName/setJobName.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/ironSwarm/useGenerateBenignSuite.ts` around lines 59 - 69, Update web/packages/studio/src/components/ironSwarm/useGenerateBenignSuite.ts lines 59-69 in resolveRun to catch ironSwarmListRuns failures and toast when all attempts expire, ensuring starting is reset instead of remaining true; update lines 78-94 in the async IIFE to catch request failures and add effect-cleanup ignore guards before setRunName and setJobName.web/packages/studio/src/routes/IronSwarmRunDetailsRoute/index.tsx-122-130 (1)
122-130: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winTab value becomes orphaned when the HITL prompt clears.
The effect switches to
interview, but nothing switches back. After the operator submits an answer,hitlPendingturns false, and both theinterviewtrigger and itsTabsContentunmount whiletabstill equals'interview'. The panel then renders no content until the user clicks another tab.Return to
swarmwhen the prompt clears.🐛 Proposed fix
useEffect(() => { - if (hitlPending) setTab('interview'); + setTab((current) => { + if (hitlPending) return 'interview'; + return current === 'interview' ? 'swarm' : current; + }); }, [hitlPending]);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/routes/IronSwarmRunDetailsRoute/index.tsx` around lines 122 - 130, Update the tab-selection effect near hitlPending and setTab so it selects interview while HITL is pending and returns to swarm when hitlPending becomes false, preventing the tab value from referencing the unmounted interview content.web/packages/studio/src/components/ironSwarm/swarm/SwarmGraph.tsx-196-201 (1)
196-201: 📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick winNodes are selectable only by pointer.
The
<g>element handlesonPointerDownonly. Keyboard users cannot select a node, soNodeDetailstays empty for them. Addrole="button",tabIndex={0}, an accessible name, and anonKeyDownhandler that callsonSelect(n.id)on Enter and Space.♿ Proposed fix
<g key={n.id} + role="button" + tabIndex={0} + aria-label={`${n.title} (${status})`} onPointerDown={(e) => onNodePointerDown(e, n)} + onKeyDown={(e) => { + if (e.key === 'Enter' || e.key === ' ') { + e.preventDefault(); + onSelect(n.id); + } + }} className="cursor-grab active:cursor-grabbing" >🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/ironSwarm/swarm/SwarmGraph.tsx` around lines 196 - 201, Update the node `<g>` element in the SwarmGraph rendering to be keyboard-accessible by adding button semantics, keyboard focus via tabIndex={0}, and an accessible name. Add an onKeyDown handler that calls onSelect(n.id) when the key is Enter or Space, while preserving the existing pointer selection behavior.web/packages/studio/src/components/ironSwarm/ReviewPanel.tsx-17-19 (1)
17-19: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winLocal rows never resync with a new
suiteprop.
useStateusessuiteonly for the first render. The parent route keeps this component mounted while it pollsstatus_details, so a second review round (or a late-arriving suite) leaves the operator editing and approving the previous suite.Reset rows when
suitechanges, or key the component by round in the parent.🔁 Proposed fix
-import { FC, useState } from 'react'; +import { FC, useEffect, useState } from 'react'; @@ const [rows, setRows] = useState<SuiteRow[]>(suite); + useEffect(() => { + setRows(suite); + }, [suite]);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/ironSwarm/ReviewPanel.tsx` around lines 17 - 19, Update ReviewPanel’s local rows state so it resynchronizes with the incoming suite prop whenever suite changes, preserving user edits between suite updates while replacing stale rows for new review rounds or late-arriving data.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/sdk.py-280-308 (1)
280-308: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winPreserve the async client's authentication.
make_sdk(str(self._platform.base_url))creates a direct-modeNeMoPlatformwithout auth headers. Both async methods therefore drop the caller's credentials and can fail on authenticated deployments. Preserve the async client's auth configuration when creating the sync client.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/sdk.py` around lines 280 - 308, Update the sync client creation in the async methods wrapping _run_war_game and _run_synth_benign to preserve the caller’s authentication configuration from self._platform. Ensure the client passed to asyncio.to_thread retains the async client’s auth headers while using the existing base URL.plugins/nemo-iron-swarm/tests/unit/test_events.py-88-120 (1)
88-120: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winThis test cannot detect a filename mismatch in the fallback.
_events_pathis patched to.../missing/events.jsonl, andfake_downloadwritesevents.jsonlinto the destination directory. The two names match only because of that patch. Production names the local log<safe-run-name>.jsonl, so a fileset that storesevents.jsonlwould produce an empty response while this test still passes. Patch_events_pathto a run-named file, for examplemy-run.jsonl, and keep the download writing the name the real fileset contains.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/tests/unit/test_events.py` around lines 88 - 120, Update test_get_events_falls_back_to_fileset_when_local_missing so the patched _events_path uses a run-named file such as my-run.jsonl, while fake_download continues writing events.jsonl from the fileset. Keep the existing fallback assertions and setup unchanged so the test detects mismatches between the local log filename and the downloaded fileset filename.plugins/nemo-iron-swarm/README.md-14-18 (1)
14-18: 📐 Maintainability & Code Quality | 🟠 Major | 🏗️ Heavy liftSplit this README by Diataxis type.
This page combines tutorial, how-to, troubleshooting, reference, and explanation content. Split these into separate pages. Put prerequisites before task steps. Add Next Steps links. Provide verified Python SDK and CLI alternatives in tab sets where the SDK supports the task.
As per coding guidelines, each documentation page must fit one Diataxis quadrant and task pages must list prerequisites first.
Also applies to: 287-294, 412-447
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/README.md` around lines 14 - 18, Restructure the README into separate Diataxis-focused pages, separating tutorial, how-to, troubleshooting, reference, and explanatory content. Ensure each task page begins with prerequisites, add Next Steps links, and provide verified Python SDK and CLI alternatives in tab sets wherever both support the task. Apply the same restructuring to the sections corresponding to the referenced later ranges, preserving the existing instructions and examples.Source: Coding guidelines
plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/manifests.py-555-566 (1)
555-566: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winRefresh leaks the new fileset when
entity_client.updatefails.If
updateraises, the freshly uploadedfilesetis never referenced by any entity and is never deleted. Wrap the update and delete the new fileset on failure. Also mapNemoEntityConflictError/NemoEntityNotFoundErrorto 409/404 asupdate_manifestdoes; today they surface as 500.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/manifests.py` around lines 555 - 566, Update the refresh flow around entity_client.update to clean up the newly assigned fileset when the update fails, deleting fileset rather than the stale fileset in the exception path. Add the same NemoEntityConflictError and NemoEntityNotFoundError mappings used by update_manifest so failures return 409 and 404 respectively, while preserving stale-fileset cleanup only after a successful update.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/manifest.py-236-242 (1)
236-242: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winReject an absolute or escaping
agent["workflow"].
manifest_dir / project_dir / workflowwritesdefense_workflowwherever the joined path resolves. An absoluteworkflowvalue, or one containing.., writes outside the job storage directory. The values come from a stored manifest that an API client controls at create time. Resolve the path and confirm it stays undermanifest_dir.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/manifest.py` around lines 236 - 242, Validate the resolved workflow path in the manifest-writing flow before creating directories or writing defense_workflow. Using workflow_file and manifest_dir, reject absolute agent["workflow"] values and any resolved path that escapes manifest_dir, including traversal through project_dir; only proceed with mkdir and write_text when the resolved path remains under manifest_dir.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/synth_benign.py-108-137 (1)
108-137: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick winA failure before
_run_serviceleaves a pre-created run record stuck atrunning.
compilecan hand arun_namein the step config. If_materialize_manifest,require_provisioned, orcheck_victim_secretsraises,run()returns a failed result but never touches that record. Studio then shows the run asrunningforever. Finalize the record in theexceptbranch whenconfig.get("run_name")is set, as the war-game job does.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/synth_benign.py` around lines 108 - 137, Update synth-benign’s run exception path to finalize the pre-created run record when config.get("run_name") is set, including failures from _materialize_manifest, require_provisioned, or check_victim_secrets before _run_service executes. Mirror the existing war-game job’s record-finalization behavior, then preserve the current classified failure result.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/manifests.py-585-590 (1)
585-590: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winGuard the fileset cleanup so a storage error does not turn a successful delete into a 500.
The entity is already deleted at this point. If
delete_filesetraises, the endpoint returns 500 and the caller assumes the manifest still exists.🛡️ Proposed fix
for ref in (existing.agent_fileset, existing.project_fileset): if ref: - await run_in_threadpool(delete_fileset, sdk, ref) + try: + await run_in_threadpool(delete_fileset, sdk, ref) + except Exception: # the entity is gone; an orphan bundle must not fail the delete + logger.warning("failed to delete fileset '%s' for manifest '%s'", ref, name, exc_info=True)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/manifests.py` around lines 585 - 590, Guard the post-deletion fileset cleanup loop in the manifest deletion flow so exceptions from delete_fileset do not propagate as endpoint failures. Keep the entity deletion successful and handle or log cleanup errors around each fileset independently, preserving cleanup attempts for both existing.agent_fileset and existing.project_fileset.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/execution.py-278-283 (1)
278-283: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winTear down the sandbox when
upfails.
upcan fail after it started containers, for example when the health wait times out. This early return skips thefinallyblock, so the victim container and its port forward stay allocated. Move the teardown to cover this path.🛡️ Proposed fix
if up_failure is not None: + _teardown_sandbox(bin_path, manifest, env, ctx) return RunOutcome("failed", up_done.returncode, record_name=record_name, failure=up_failure)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/execution.py` around lines 278 - 283, Ensure the sandbox teardown/finally scope also covers the _run_iron_swarm invocation for the “up” command. Restructure the up_failure handling so it records or propagates the failure only after cleanup runs, while preserving the existing failed RunOutcome values and releasing the victim container and port forward.
🟡 Minor comments (19)
plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/main.py-418-424 (1)
418-424: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick winHandle a malformed
mitigations.json.
json.loadsraisesJSONDecodeErrorand prints a traceback. The adjacent missing-file case gets a clean message. Also guarddefense_idsagainst a payload that is not the expected shape.🐛 Proposed fix
- mitigations = json.loads(path.read_text(encoding="utf-8")) + try: + mitigations = json.loads(path.read_text(encoding="utf-8")) + except (OSError, json.JSONDecodeError) as exc: + typer.secho(f"Could not read mitigations file {mitigations_file}: {exc}", fg="red") + raise typer.Exit(code=1) from exc🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/main.py` around lines 418 - 424, Update the mitigations-loading flow in the CLI around path.read_text, json.loads, and defense_ids to catch malformed JSON and report a concise red error before exiting with code 1, matching the existing missing-file handling. Validate that the decoded payload has the expected shape before passing it to defense_ids, and handle invalid payloads with the same clean failure behavior.web/packages/studio/src/routes/NewIronSwarmManifestRoute/index.tsx-147-151 (1)
147-151: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winNeither manifest path bounds the victim port. Both forms accept a port outside
1-65535and write it into the manifest.
web/packages/studio/src/routes/NewIronSwarmManifestRoute/index.tsx#L147-L151: extend the check to rejectport < 1andport > 65535.web/packages/studio/src/components/ironSwarm/ProjectManifestWizard.tsx#L131-L131: replacepositive()with.min(1).max(65535)and attach a message to the type check so non-numeric input reports a clear error.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/routes/NewIronSwarmManifestRoute/index.tsx` around lines 147 - 151, Bound port validation in NewIronSwarmManifestRoute/index.tsx lines 147-151 to reject values below 1 or above 65535 while preserving the whole-number check. In ProjectManifestWizard.tsx line 131, replace positive() with min/max validation for 1–65535 and provide a clear message for non-numeric input.web/packages/studio/src/components/ironSwarm/ModelGroupFields.tsx-186-208 (1)
186-208: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winClear
testResultwhen the model or endpoint changes.The verdict text stays on screen after the operator edits
model,base_url, or the secret. The stale "Connection OK." then describes a configuration that no longer exists. Reset the result in eachonChange.🐛 Proposed fix
+ const update = (patch: Partial<ModelChoice>) => { + setTestResult(null); + onChange(withGroup(value, group, patch)); + };Then call
update({ model: e.target.value || undefined })and the equivalent forbase_urlandapi_key_secret.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/ironSwarm/ModelGroupFields.tsx` around lines 186 - 208, Update the model, base_url, and api_key_secret change handlers in ModelGroupFields to clear testResult whenever any of those values changes. Preserve the existing normalized values and onChange/update behavior while resetting the stale connection verdict in each handler.web/packages/studio/src/constants/routes.ts-113-117 (1)
113-117: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winNamespace resource detail routes.
React Router resolves
/iron-swarm/manifeststoironSwarmManifestList, so a run namedmanifestsis unreachable. TheNAME_PATTERNacceptsnew, so a manifest namednewis also unreachable because/iron-swarm/manifests/newresolves toironSwarmManifestNew. Namespace runs under/iron-swarm/runs/:ironSwarmRunNameand reserve or namespace the manifestnewroute.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/constants/routes.ts` around lines 113 - 117, Update the ironSwarm route definitions so run details use the `/iron-swarm/runs/:ironSwarmRunName` namespace, preventing the `manifests` run name from colliding with manifest routes. Also adjust the manifest detail/new route structure to reserve or namespace the `new` endpoint, while preserving manifest list and creation navigation.web/packages/studio/src/components/ironSwarm/eventTypes.ts-4-6 (1)
4-6: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winStale transport reference: SSE was replaced by JSON polling.
Lines 4-6 and 91-92 describe an "SSE relay" and "plugin SSE endpoint". The PR changed event delivery to JSON polling. Update both comments so readers do not look for an SSE endpoint.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/ironSwarm/eventTypes.ts` around lines 4 - 6, Update the comments near the event catalog and the plugin endpoint reference in eventTypes.ts to describe JSON polling instead of SSE, including removing references to the SSE relay and plugin SSE endpoint while preserving the existing event-source and rendering context.web/packages/studio/src/routes/IronSwarmManifestDetailRoute/index.tsx-270-286 (1)
270-286: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick winEnvironment values containing a comma are silently truncated.
saveEnvsplits the draft on,before it finds=. A value such asHOSTS=a,bis parsed asHOSTS=aplus a discarded fragment. The user gets a success toast and loses data.Reject entries that fail to parse, or switch the dialog to newline-separated pairs.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/routes/IronSwarmManifestDetailRoute/index.tsx` around lines 270 - 286, Update saveEnv so environment values containing commas are not silently truncated: either parse a format that preserves commas, such as newline-separated key/value pairs, or validate each comma-separated entry and reject malformed fragments before calling clearManifest.mutateAsync. Only show the success toast after all entries are valid and saved; retain the existing error-toast path for rejected input or mutation failures.web/packages/studio/src/routes/IronSwarmManifestDetailRoute/index.tsx-303-311 (1)
303-311: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winRevoking the object URL synchronously can cancel the download.
URL.revokeObjectURLruns in the same tick asanchor.click(). Some browsers have not started reading the blob yet, so the download fails. Defer the revoke.Proposed fix
anchor.click(); - URL.revokeObjectURL(url); + setTimeout(() => URL.revokeObjectURL(url), 0);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/routes/IronSwarmManifestDetailRoute/index.tsx` around lines 303 - 311, Update downloadCsv to defer URL.revokeObjectURL until after the browser has initiated the anchor download, rather than revoking it synchronously after anchor.click(). Keep the existing blob creation, filename, and click behavior unchanged.web/packages/studio/src/components/ironSwarm/InterviewPanel.tsx-33-36 (1)
33-36: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winDefaults do not re-seed when
promptchanges.
useStateruns its initializer once. If the parent renders a second prompt at the same position, the component stays mounted andanswerskeeps the previous gaps. Questions then show no selection, and submit sends empty answers.Give the panel a key derived from the prompt at the call site, or re-seed on prompt change.
Proposed fix at the call site
- <InterviewPanel + <InterviewPanel + key={gen.interview.questions.map((q) => q.gap).join('|')} prompt={gen.interview}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/ironSwarm/InterviewPanel.tsx` around lines 33 - 36, Update InterviewPanel so its answers state is re-seeded whenever prompt changes, preserving defaults for the new prompt’s questions; alternatively, at the component’s call site provide a key derived from the prompt so React remounts it. Use the existing defaultAnswer initialization path and ensure submissions reflect the current prompt rather than prior gaps.web/packages/studio/src/components/ironSwarm/hitlTypes.ts-46-64 (1)
46-64: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick winPrompts are cast without validating their array payloads.
pendingInterviewandpendingReviewcheck onlyround.questionsandsuiteare asserted, not verified. If the job writes a prompt without those keys, consumers that map over them (for exampleReviewPanelrenderingreview.suite) throw at render.Guard the arrays and normalize to
[].🛡️ Proposed fix
if (!interview || typeof interview.round !== 'number') return null; if (response?.round === interview.round) return null; - return interview; + return { round: interview.round, questions: Array.isArray(interview.questions) ? interview.questions : [] }; @@ if (!review || typeof review.round !== 'number') return null; if (response?.round === review.round) return null; - return review; + return { round: review.round, suite: Array.isArray(review.suite) ? review.suite : [] };🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/ironSwarm/hitlTypes.ts` around lines 46 - 64, Update pendingInterview and pendingReview to validate their prompt array payloads before returning them, requiring interview.questions and review.suite to be arrays alongside the existing round checks. Normalize missing or invalid arrays to [] as appropriate so consumers can safely iterate them, while preserving the existing null response-round behavior.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/events.py-109-119 (1)
109-119: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winRouter description still advertises SSE.
This module now implements polling only.
service.pyregisters this router with the description "Live run-event ingest (from the run) + SSE stream (to Studio)". That text reaches the generated OpenAPI tag description, so API consumers see a transport that no longer exists. Update the description inservice.pyto describe the poll endpoint.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/events.py` around lines 109 - 119, Update the router description supplied by service.py when registering this events router so it describes the polling endpoint and no longer mentions an SSE stream. Keep the existing ingest-run context and ensure the generated OpenAPI tag description matches the module’s polling-only behavior.plugins/nemo-iron-swarm/tests/unit/test_compose_defense.py-97-106 (1)
97-106: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winDo not preserve success for a nonexistent run.
This test creates no
run-1entity but asserts200. It locks the route contract where/runs/{name}/compose-defenseignores{name}. Look up the run in the handler, return404when absent, and change this test to assert that result.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/tests/unit/test_compose_defense.py` around lines 97 - 106, Update the compose-defense handler to resolve the run identified by the route’s name parameter and return 404 when it does not exist, rather than composing a successful response unconditionally. Revise test_compose_defense_endpoint_composes_selection to create or reference an existing run for the success case, and add or update the nonexistent run assertion to expect 404.plugins/nemo-iron-swarm/README.md-300-312 (1)
300-312: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winUse one manifest lifecycle contract.
Lines 300-312 say a manifest freezes its resolved target until
refresh. Lines 441-447 say an agent-source manifest re-resolves on every run. These rules give opposite results for run comparability and refresh behavior. Document the implemented contract consistently.As per PR objectives, manifests freeze resolved targets and explicit refresh incorporates agent changes.
Also applies to: 441-447
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/README.md` around lines 300 - 312, Update the manifest lifecycle documentation around the frozen-target guidance and the agent-source manifest section to consistently state that manifests preserve their resolved targets across runs. Document explicit refresh as the mechanism that incorporates agent changes, and remove or revise the claim that agent-source manifests re-resolve on every run.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/model_preflight.py-72-76 (1)
72-76: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick winHandle non-object JSON responses.
If
resp.json()returns an array or scalar,.get()raises an uncaughtAttributeError. Validate the decoded object before accessing"data"and return the existing soft-pass result.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/model_preflight.py` around lines 72 - 76, Update the JSON handling in the model preflight probe to validate that resp.json() returns an object with mapping behavior before calling .get("data"). For arrays or scalar JSON responses, return the existing reachable/authenticated soft-pass result with list_supported=False and the current detail, while preserving normal processing for valid objects.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/synth_client.py-60-70 (1)
60-70: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick winClassify malformed synth responses as synth-service failures.
Catch
ValueErrorfrom invalid JSON and reject non-object JSON before returning from_post.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/synth_client.py` around lines 60 - 70, Update _post to catch ValueError from resp.json() and classify it as CATEGORY_SYNTH_SERVICE via IronSwarmRunError. Validate that the decoded JSON is an object/dict before returning it, and raise the same synth-service failure for any other JSON shape while preserving existing HTTPError handling.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/service.py-69-74 (1)
69-74: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winStale "SSE stream" wording; events are polled JSON.
Event delivery moved from SSE to cursor-based JSON polling.
GET /runs/{name}/eventsreturnsEventsResponsewith anaftercursor. ThisRouterSpecdescription surfaces in the generated OpenAPI tag, so the wrong wording reaches API consumers. The module docstring at Line 9 has the same problem.📝 Proposed fix
RouterSpec( router=events.router, tag="Iron Swarm Events", - description="Live run-event ingest (from the run) + SSE stream (to Studio).", + description="Run-event ingest (from the run) + polled event reads (to Studio).", prefix="/v2/workspaces/{workspace}", ),🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/service.py` around lines 69 - 74, Update the events RouterSpec description to describe cursor-based JSON polling rather than an SSE stream, and revise the module docstring to remove the stale SSE wording. Keep the description accurate for GET /runs/{name}/events returning EventsResponse with an after cursor.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/manifests.py-114-132 (1)
114-132: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winReturn the serialized filter, not the model.
Line 114 computes
filter_dict, but line 131 returns the rawfiltermodel. The response then includes every unset field asnull, which disagrees with the filter actually applied.♻️ Proposed fix
- "filter": filter or None, + "filter": filter_dict or None,🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/manifests.py` around lines 114 - 132, Update the response construction in the manifest-listing function to return the computed serialized filter_dict instead of the raw filter model. Preserve the existing filter or None behavior when no filter values are provided, while ensuring unset fields remain excluded consistently with the filter passed to entity_client.list.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/manifest.py-165-168 (1)
165-168: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick winGuard against non-mapping
backendsentries.
manifest.get("backends")comes from stored YAML. If any entry is not a mapping,b.get("name")raisesAttributeErrorand the run fails with an unclassified error instead of a manifest error.🛡️ Proposed fix
- others = [b for b in (manifest.get("backends") or []) if b.get("name") != gw_backend.get("name")] + others = [ + b + for b in (manifest.get("backends") or []) + if isinstance(b, dict) and b.get("name") != gw_backend.get("name") + ]🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/manifest.py` around lines 165 - 168, Update the backends filtering logic in the manifest handling block to validate each entry is a mapping before calling b.get("name"). Preserve valid mappings, exclude or reject non-mapping entries using the existing manifest-error handling path, and ensure malformed YAML produces a classified manifest error instead of an AttributeError.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/runs.py-142-158 (1)
142-158: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick winReturn 409 for optimistic-lock conflicts.
NemoEntitiesClient.update()sendsexpected_db_versionand prevents lost updates, but this handler mapsNemoEntityConflictErrorto 500. Catch it explicitly and return 409.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/runs.py` around lines 142 - 158, Update the exception handling around entity_client.update(agent) to catch NemoEntityConflictError explicitly and raise an HTTPException with status 409, preserving the conflict as the cause; keep the existing generic exception path returning 500 for other update failures.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/tasks/war_game/__main__.py-25-38 (1)
25-38: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick winMove sandbox teardown to cover
up. If SIGTERM arrives afterupstarts the sandbox but before it returns,_shutdown_handlerexits before_teardown_sandboxis registered.subprocess.runalso does not clean up descendant processes. Wrapupin thetry/finallyblock and terminate the subprocess group during cancellation.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/tasks/war_game/__main__.py` around lines 25 - 38, Update the war-game task’s up/teardown flow so sandbox cleanup is registered before invoking `up`, ensuring SIGTERM during `up` still reaches teardown. In the teardown logic, terminate the subprocess process group rather than only the direct process, while preserving normal cleanup and cancellation behavior.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: d58ccc27-2434-466b-b2b6-71af8e07923b
⛔ Files ignored due to path filters (3)
uv.lockis excluded by!**/*.lockweb/packages/sdk/generated/iron-swarm/api.tsis excluded by!**/generated/**web/pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (116)
.gitignoredocs/iron-swarm-review/findings.mdplugins/nemo-iron-swarm/.gitignoreplugins/nemo-iron-swarm/README.mdplugins/nemo-iron-swarm/openapi/openapi.yamlplugins/nemo-iron-swarm/pyproject.tomlplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/_perms.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/agent_resolver.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/_filters.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/events.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/jobs.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/manifests.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/runs.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/schemas.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/authz.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/checks.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/client.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/credentials.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/main.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/provisioning.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/config.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/entities.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/filesets.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/_common.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/artifacts.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/benign_suite.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/defenses.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/errors.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/execution.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/hitl.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/manifest.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/records.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/run.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/spec.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/synth_benign.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/synth_client.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/model_config.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/model_preflight.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/sdk.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/service.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/skills.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/skills/iron-swarm/SKILL.mdplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/tasks/synth_benign/__main__.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/tasks/war_game/__main__.pyplugins/nemo-iron-swarm/tests/unit/_doubles.pyplugins/nemo-iron-swarm/tests/unit/test_agent_resolver.pyplugins/nemo-iron-swarm/tests/unit/test_api_manifests.pyplugins/nemo-iron-swarm/tests/unit/test_api_runs.pyplugins/nemo-iron-swarm/tests/unit/test_apply_mitigation.pyplugins/nemo-iron-swarm/tests/unit/test_artifacts.pyplugins/nemo-iron-swarm/tests/unit/test_benign_suite.pyplugins/nemo-iron-swarm/tests/unit/test_compose_defense.pyplugins/nemo-iron-swarm/tests/unit/test_errors.pyplugins/nemo-iron-swarm/tests/unit/test_events.pyplugins/nemo-iron-swarm/tests/unit/test_filesets.pyplugins/nemo-iron-swarm/tests/unit/test_garak_provision.pyplugins/nemo-iron-swarm/tests/unit/test_model_config.pyplugins/nemo-iron-swarm/tests/unit/test_model_preflight.pyplugins/nemo-iron-swarm/tests/unit/test_operator_env.pyplugins/nemo-iron-swarm/tests/unit/test_preflight.pyplugins/nemo-iron-swarm/tests/unit/test_run_cli.pyplugins/nemo-iron-swarm/tests/unit/test_run_record.pyplugins/nemo-iron-swarm/tests/unit/test_run_service.pyplugins/nemo-iron-swarm/tests/unit/test_sanity_check_cli.pyplugins/nemo-iron-swarm/tests/unit/test_sdk_resources.pyplugins/nemo-iron-swarm/tests/unit/test_service.pyplugins/nemo-iron-swarm/tests/unit/test_synth_benign.pyplugins/nemo-iron-swarm/tests/unit/test_synth_hitl.pypyproject.tomlservices/studio/src/nmp/studio/env_mappings.pyweb/packages/sdk/orval/constants.tsweb/packages/sdk/package.jsonweb/packages/studio/env/.env.fastapiweb/packages/studio/package.jsonweb/packages/studio/src/api/ironSwarm.tsweb/packages/studio/src/components/dataViews/IronSwarmManifestsDataView/index.tsxweb/packages/studio/src/components/dataViews/IronSwarmRunsDataView/index.tsxweb/packages/studio/src/components/ironSwarm/BenignInterviewCard.tsxweb/packages/studio/src/components/ironSwarm/BenignSuiteEditor.tsxweb/packages/studio/src/components/ironSwarm/BenignSuiteTable.tsxweb/packages/studio/src/components/ironSwarm/HardenPanel.tsxweb/packages/studio/src/components/ironSwarm/InterviewPanel.tsxweb/packages/studio/src/components/ironSwarm/ModelGroupFields.tsxweb/packages/studio/src/components/ironSwarm/ProjectManifestWizard.tsxweb/packages/studio/src/components/ironSwarm/ReconChecklist.tsxweb/packages/studio/src/components/ironSwarm/ReviewPanel.tsxweb/packages/studio/src/components/ironSwarm/SanityCheckReport.tsxweb/packages/studio/src/components/ironSwarm/TargetPanel.tsxweb/packages/studio/src/components/ironSwarm/YamlDiff.tsxweb/packages/studio/src/components/ironSwarm/eventTypes.tsweb/packages/studio/src/components/ironSwarm/hitlTypes.tsweb/packages/studio/src/components/ironSwarm/swarm/MessageFeed.tsxweb/packages/studio/src/components/ironSwarm/swarm/NodeDetail.tsxweb/packages/studio/src/components/ironSwarm/swarm/SwarmGraph.tsxweb/packages/studio/src/components/ironSwarm/swarm/swarmModel.test.tsweb/packages/studio/src/components/ironSwarm/swarm/swarmModel.tsweb/packages/studio/src/components/ironSwarm/swarm/useSwarmEvents.tsweb/packages/studio/src/components/ironSwarm/useGenerateBenignSuite.tsweb/packages/studio/src/components/ironSwarm/useMitigations.test.tsweb/packages/studio/src/components/ironSwarm/useMitigations.tsweb/packages/studio/src/components/ironSwarm/useRunWarGame.tsweb/packages/studio/src/components/ironSwarm/useSanityCheck.tsweb/packages/studio/src/constants/environment.tsweb/packages/studio/src/constants/featureFlags/featureFlags.tsweb/packages/studio/src/constants/routes.tsweb/packages/studio/src/routes/IronSwarmManifestDetailRoute/index.tsxweb/packages/studio/src/routes/IronSwarmManifestListRoute/index.tsxweb/packages/studio/src/routes/IronSwarmRunDetailsRoute/index.tsxweb/packages/studio/src/routes/IronSwarmRunListRoute/index.tsxweb/packages/studio/src/routes/NewIronSwarmManifestRoute/index.tsxweb/packages/studio/src/routes/WorkspaceLayout/WorkspaceSideNav.tsxweb/packages/studio/src/routes/groups/index.tsweb/packages/studio/src/routes/groups/ironSwarmRoutes.tsxweb/packages/studio/src/routes/index.tsxweb/packages/studio/src/routes/utils.tsweb/packages/studio/src/tests/title-change.test.tsx
Contributor
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
koralchapnik
changed the title
Contributor
There was a problem hiding this comment.
Actionable comments posted: 8
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
plugins/nemo-iron-swarm/README.md (1)
1-448: 📐 Maintainability & Code Quality | 🟠 Major | 🏗️ Heavy liftSplit this README by documentation purpose.
This page mixes tutorial, how-to, reference, troubleshooting, and architecture content. Split these into Diataxis-specific pages. Put prerequisites before procedural steps. Use Sphinx substitutions for product names.
As per coding guidelines, “Each documentation page should fit ONE Diataxis quadrant” and product names must use Sphinx substitutions.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/README.md` around lines 1 - 448, Split the README into Diataxis-focused pages: a tutorial for Quickstart, how-to pages for UI/CLI execution and setup, a reference page for environment variables and command behavior, a troubleshooting page for failure scenarios, and an explanation page for architecture/how it works. Put prerequisites before procedural instructions, link the pages together, and replace literal product names throughout with the repository’s Sphinx product substitutions.Source: Coding guidelines
🧹 Nitpick comments (5)
web/packages/studio/src/components/ironSwarm/BenignInterviewCard.tsx (1)
6-6: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueUse a type-only import for
FC.
FCis used only as a type. Replace the value import withimport type.Proposed change
-import { FC } from 'react'; +import type { FC } from 'react';Verify with
pnpm lint:fixfromweb/. As per coding guidelines, useimport typefor type-only imports.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/ironSwarm/BenignInterviewCard.tsx` at line 6, Update the FC import in BenignInterviewCard to use a type-only import, since FC is referenced only as a type; leave the component implementation unchanged and verify the formatting with pnpm lint:fix from web/.Source: Coding guidelines
plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/manifests.py (1)
555-566: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick winUnhandled entity errors in
refresh_manifest.entity_client.update(existing)at Line 562 is not wrapped, unlikeupdate_manifest. A concurrent modification surfaces as an unhandled exception instead of 409, andapply-mitigationthen reports the manifest as unrefreshable without a reason. Mirror theNemoEntityNotFoundError/NemoEntityConflictErrorhandling used at Lines 516-521.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/manifests.py` around lines 555 - 566, Update refresh_manifest around entity_client.update(existing) to catch NemoEntityNotFoundError and NemoEntityConflictError, matching the handling already used by update_manifest. Translate these exceptions into the established 404/409 responses so concurrent modifications and missing entities are reported with their reasons instead of escaping as unhandled errors.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/synth_client.py (1)
110-117: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick winReap the process after
kill().
proc.kill()without a followingwait()leaves a zombie for the life of the job process.Proposed fix
except subprocess.TimeoutExpired: proc.kill() + with contextlib.suppress(subprocess.TimeoutExpired): + proc.wait(timeout=5)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/synth_client.py` around lines 110 - 117, Update _terminate so that after proc.kill() handles the subprocess timeout, it also waits for the process to exit and reap it before returning.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/events.py (1)
55-73: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value
_seqis write-only state.Ids come from line numbers in
history()._seqis never read. Drop it and the docstring claim about it.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/events.py` around lines 55 - 73, Remove the unused _seq state from the event store, including its initialization in __init__ and increment in publish. Delete the docstring claim that _seq is seeded for monotonic IDs, while preserving line-number-based IDs and existing history persistence behavior.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/_common.py (1)
78-84: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick winWrap the secret lookup so a bad secret name gets classified.
sdk.secrets.accessraises on a missing or unauthorized secret. That exception escapes_apply_groupunclassified, so the run fails without themissing_credentialcategory or remediation text.Proposed fix
def _resolve_secret(sdk: Any, name: str, workspace: str) -> str | None: """Fetch a Secret's plaintext value via the platform SDK; None if unavailable (caller warns/fails).""" if sdk is None: return None - secret = sdk.secrets.access(name, workspace=workspace) + try: + secret = sdk.secrets.access(name, workspace=workspace) + except Exception as exc: + raise IronSwarmRunError( + CATEGORY_MISSING_CREDENTIAL, + f"could not read secret {name!r} from workspace {workspace!r}: {exc}", + ) from exc value = getattr(secret, "value", None) return str(value) if value else None🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/_common.py` around lines 78 - 84, Update _resolve_secret to catch exceptions from sdk.secrets.access and classify missing or unauthorized secret lookups as missing_credential, including the existing remediation text expected by _apply_group. Preserve the current None return for an unavailable SDK or absent/empty secret value, and ensure the classified error propagates through _apply_group’s normal failure handling.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@plugins/nemo-iron-swarm/README.md`:
- Line 160: Replace the unpinned install.sh command in the README with a pinned
release artifact or commit-specific URL, and require verification of its
published checksum or signature before executing it. Do not pipe mutable remote
content directly to sh; document the download, verification, and execution
steps.
- Around line 441-447: Update the manifest behavior description near the
agent-source manifest explanation to state that manifests use frozen target
filesets and are not re-resolved automatically on each run. Explain that changes
to the registered agent take effect only after an explicit refresh, while
preserving the distinction between persisted settings and rendered YAML.
In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/events.py`:
- Around line 140-167: The fileset fallback reads the run-specific path from
_events_path, but download_fileset extracts events.jsonl into its parent
directory, so the re-read misses the downloaded log. In
plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/events.py:140-167,
update _fileset_fallback to download into a temporary directory and read the
extracted member, or copy events.jsonl to stream._path before calling history;
in plugins/nemo-iron-swarm/tests/unit/test_events.py:88-120, patch _events_path
to return a run-specific path such as tmp_path / "missing" / "my-run.jsonl" so
the test validates the real filename contract.
In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/manifests.py`:
- Around line 588-590: The delete_manifest cleanup loop must not unconditionally
delete the client-owned project_fileset referenced by other manifests. Update
the manifest lifecycle around _build_project_manifest and delete_manifest to
either copy project_fileset into a service-owned fileset during creation, or
check for remaining manifest references before deleting it; retain unconditional
cleanup for the service-created agent_fileset.
In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/main.py`:
- Around line 278-285: The local manifest messaging around the manifest
rendering in the CLI must match the behavior of run --config and both source
types. Update the comments and rendered header to define a consistent contract:
local manifests generated by init remain valid runnable inputs, and avoid
claiming that every source has an agent ref because --project-dir sources do
not.
- Around line 387-434: Update the sanity_check command after
ctx.sdk.iron_swarm.sanity_check returns its result: continue printing the JSON
result, then raise a nonzero Typer exit unless result status equals "completed".
Preserve normal success behavior for completed results.
In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/provisioning.py`:
- Around line 71-73: Update the logging in the provisioning flow around
config.index_url to pass the URL through the existing redact_index_url helper
before typer.echo displays it, while continuing to use the original URL in
install_cmd for installation.
In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/synth_client.py`:
- Around line 84-87: Update the docstring for the server-spawning function
around _await_ready to state that it raises IronSwarmRunError instead of
RuntimeError, preserving the existing description of early exit and readiness
timeout.
---
Outside diff comments:
In `@plugins/nemo-iron-swarm/README.md`:
- Around line 1-448: Split the README into Diataxis-focused pages: a tutorial
for Quickstart, how-to pages for UI/CLI execution and setup, a reference page
for environment variables and command behavior, a troubleshooting page for
failure scenarios, and an explanation page for architecture/how it works. Put
prerequisites before procedural instructions, link the pages together, and
replace literal product names throughout with the repository’s Sphinx product
substitutions.
---
Nitpick comments:
In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/events.py`:
- Around line 55-73: Remove the unused _seq state from the event store,
including its initialization in __init__ and increment in publish. Delete the
docstring claim that _seq is seeded for monotonic IDs, while preserving
line-number-based IDs and existing history persistence behavior.
In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/manifests.py`:
- Around line 555-566: Update refresh_manifest around
entity_client.update(existing) to catch NemoEntityNotFoundError and
NemoEntityConflictError, matching the handling already used by update_manifest.
Translate these exceptions into the established 404/409 responses so concurrent
modifications and missing entities are reported with their reasons instead of
escaping as unhandled errors.
In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/_common.py`:
- Around line 78-84: Update _resolve_secret to catch exceptions from
sdk.secrets.access and classify missing or unauthorized secret lookups as
missing_credential, including the existing remediation text expected by
_apply_group. Preserve the current None return for an unavailable SDK or
absent/empty secret value, and ensure the classified error propagates through
_apply_group’s normal failure handling.
In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/synth_client.py`:
- Around line 110-117: Update _terminate so that after proc.kill() handles the
subprocess timeout, it also waits for the process to exit and reap it before
returning.
In `@web/packages/studio/src/components/ironSwarm/BenignInterviewCard.tsx`:
- Line 6: Update the FC import in BenignInterviewCard to use a type-only import,
since FC is referenced only as a type; leave the component implementation
unchanged and verify the formatting with pnpm lint:fix from web/.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: af40b6bc-cba0-488a-b643-813cf84f9b42
⛔ Files ignored due to path filters (3)
uv.lockis excluded by!**/*.lockweb/packages/sdk/generated/iron-swarm/api.tsis excluded by!**/generated/**web/pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (116)
.gitignoreplugins/nemo-iron-swarm/.gitignoreplugins/nemo-iron-swarm/README.mdplugins/nemo-iron-swarm/openapi/openapi.yamlplugins/nemo-iron-swarm/pyproject.tomlplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/_perms.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/agent_resolver.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/_filters.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/events.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/jobs.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/manifests.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/runs.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/schemas.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/authz.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/checks.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/client.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/credentials.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/main.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/provisioning.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/config.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/entities.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/filesets.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/_common.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/artifacts.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/benign_suite.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/defenses.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/errors.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/execution.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/hitl.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/manifest.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/records.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/run.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/spec.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/synth_benign.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/synth_client.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/model_config.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/model_preflight.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/sdk.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/service.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/skills.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/skills/iron-swarm/SKILL.mdplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/tasks/synth_benign/__main__.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/tasks/war_game/__main__.pyplugins/nemo-iron-swarm/tests/unit/_doubles.pyplugins/nemo-iron-swarm/tests/unit/test_agent_resolver.pyplugins/nemo-iron-swarm/tests/unit/test_api_manifests.pyplugins/nemo-iron-swarm/tests/unit/test_api_runs.pyplugins/nemo-iron-swarm/tests/unit/test_apply_mitigation.pyplugins/nemo-iron-swarm/tests/unit/test_artifacts.pyplugins/nemo-iron-swarm/tests/unit/test_benign_suite.pyplugins/nemo-iron-swarm/tests/unit/test_compose_defense.pyplugins/nemo-iron-swarm/tests/unit/test_errors.pyplugins/nemo-iron-swarm/tests/unit/test_events.pyplugins/nemo-iron-swarm/tests/unit/test_filesets.pyplugins/nemo-iron-swarm/tests/unit/test_garak_provision.pyplugins/nemo-iron-swarm/tests/unit/test_model_config.pyplugins/nemo-iron-swarm/tests/unit/test_model_preflight.pyplugins/nemo-iron-swarm/tests/unit/test_operator_env.pyplugins/nemo-iron-swarm/tests/unit/test_preflight.pyplugins/nemo-iron-swarm/tests/unit/test_run_cli.pyplugins/nemo-iron-swarm/tests/unit/test_run_record.pyplugins/nemo-iron-swarm/tests/unit/test_run_service.pyplugins/nemo-iron-swarm/tests/unit/test_sanity_check_cli.pyplugins/nemo-iron-swarm/tests/unit/test_sdk_resources.pyplugins/nemo-iron-swarm/tests/unit/test_service.pyplugins/nemo-iron-swarm/tests/unit/test_synth_benign.pyplugins/nemo-iron-swarm/tests/unit/test_synth_hitl.pypyproject.tomlpytest.iniservices/studio/src/nmp/studio/env_mappings.pyweb/packages/sdk/orval/constants.tsweb/packages/sdk/package.jsonweb/packages/studio/env/.env.fastapiweb/packages/studio/package.jsonweb/packages/studio/src/api/ironSwarm.tsweb/packages/studio/src/components/dataViews/IronSwarmManifestsDataView/index.tsxweb/packages/studio/src/components/dataViews/IronSwarmRunsDataView/index.tsxweb/packages/studio/src/components/ironSwarm/BenignInterviewCard.tsxweb/packages/studio/src/components/ironSwarm/BenignSuiteEditor.tsxweb/packages/studio/src/components/ironSwarm/BenignSuiteTable.tsxweb/packages/studio/src/components/ironSwarm/HardenPanel.tsxweb/packages/studio/src/components/ironSwarm/InterviewPanel.tsxweb/packages/studio/src/components/ironSwarm/ModelGroupFields.tsxweb/packages/studio/src/components/ironSwarm/ProjectManifestWizard.tsxweb/packages/studio/src/components/ironSwarm/ReconChecklist.tsxweb/packages/studio/src/components/ironSwarm/ReviewPanel.tsxweb/packages/studio/src/components/ironSwarm/SanityCheckReport.tsxweb/packages/studio/src/components/ironSwarm/TargetPanel.tsxweb/packages/studio/src/components/ironSwarm/YamlDiff.tsxweb/packages/studio/src/components/ironSwarm/eventTypes.tsweb/packages/studio/src/components/ironSwarm/hitlTypes.tsweb/packages/studio/src/components/ironSwarm/swarm/MessageFeed.tsxweb/packages/studio/src/components/ironSwarm/swarm/NodeDetail.tsxweb/packages/studio/src/components/ironSwarm/swarm/SwarmGraph.tsxweb/packages/studio/src/components/ironSwarm/swarm/swarmModel.test.tsweb/packages/studio/src/components/ironSwarm/swarm/swarmModel.tsweb/packages/studio/src/components/ironSwarm/swarm/useSwarmEvents.tsweb/packages/studio/src/components/ironSwarm/useGenerateBenignSuite.tsweb/packages/studio/src/components/ironSwarm/useMitigations.test.tsweb/packages/studio/src/components/ironSwarm/useMitigations.tsweb/packages/studio/src/components/ironSwarm/useRunWarGame.tsweb/packages/studio/src/components/ironSwarm/useSanityCheck.tsweb/packages/studio/src/constants/environment.tsweb/packages/studio/src/constants/featureFlags/featureFlags.tsweb/packages/studio/src/constants/routes.tsweb/packages/studio/src/routes/IronSwarmManifestDetailRoute/index.tsxweb/packages/studio/src/routes/IronSwarmManifestListRoute/index.tsxweb/packages/studio/src/routes/IronSwarmRunDetailsRoute/index.tsxweb/packages/studio/src/routes/IronSwarmRunListRoute/index.tsxweb/packages/studio/src/routes/NewIronSwarmManifestRoute/index.tsxweb/packages/studio/src/routes/WorkspaceLayout/WorkspaceSideNav.tsxweb/packages/studio/src/routes/groups/index.tsweb/packages/studio/src/routes/groups/ironSwarmRoutes.tsxweb/packages/studio/src/routes/index.tsxweb/packages/studio/src/routes/utils.tsweb/packages/studio/src/tests/title-change.test.tsx
🚧 Files skipped from review as they are similar to previous changes (86)
- web/packages/studio/env/.env.fastapi
- web/packages/studio/src/constants/routes.ts
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/tasks/war_game/main.py
- plugins/nemo-iron-swarm/tests/unit/test_model_preflight.py
- web/packages/studio/src/components/ironSwarm/swarm/useSwarmEvents.ts
- plugins/nemo-iron-swarm/tests/unit/test_service.py
- web/packages/studio/src/components/ironSwarm/HardenPanel.tsx
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/client.py
- pyproject.toml
- web/packages/studio/src/components/ironSwarm/useGenerateBenignSuite.ts
- plugins/nemo-iron-swarm/tests/unit/test_model_config.py
- web/packages/studio/src/components/ironSwarm/useRunWarGame.ts
- plugins/nemo-iron-swarm/.gitignore
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/hitl.py
- web/packages/studio/src/components/ironSwarm/swarm/swarmModel.test.ts
- web/packages/studio/src/components/ironSwarm/hitlTypes.ts
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/credentials.py
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/defenses.py
- web/packages/studio/src/routes/groups/index.ts
- web/packages/studio/src/components/dataViews/IronSwarmManifestsDataView/index.tsx
- plugins/nemo-iron-swarm/tests/unit/test_sdk_resources.py
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/service.py
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/spec.py
- web/packages/studio/src/components/ironSwarm/swarm/NodeDetail.tsx
- plugins/nemo-iron-swarm/tests/unit/test_run_record.py
- plugins/nemo-iron-swarm/tests/unit/test_apply_mitigation.py
- web/packages/studio/src/components/ironSwarm/BenignSuiteEditor.tsx
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/entities.py
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/tasks/synth_benign/main.py
- web/packages/studio/package.json
- plugins/nemo-iron-swarm/tests/unit/test_preflight.py
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/execution.py
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/errors.py
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/config.py
- web/packages/studio/src/components/ironSwarm/ReviewPanel.tsx
- web/packages/studio/src/constants/featureFlags/featureFlags.ts
- web/packages/studio/src/components/ironSwarm/YamlDiff.tsx
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/run.py
- plugins/nemo-iron-swarm/tests/unit/test_benign_suite.py
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/schemas.py
- web/packages/studio/src/components/ironSwarm/BenignSuiteTable.tsx
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/artifacts.py
- web/packages/studio/src/components/ironSwarm/swarm/swarmModel.ts
- web/packages/studio/src/components/ironSwarm/swarm/MessageFeed.tsx
- web/packages/studio/src/components/ironSwarm/ModelGroupFields.tsx
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/records.py
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/sdk.py
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/model_preflight.py
- web/packages/sdk/orval/constants.ts
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/agent_resolver.py
- web/packages/studio/src/routes/NewIronSwarmManifestRoute/index.tsx
- web/packages/studio/src/components/ironSwarm/useSanityCheck.ts
- web/packages/studio/src/components/ironSwarm/InterviewPanel.tsx
- web/packages/studio/src/components/ironSwarm/TargetPanel.tsx
- web/packages/studio/src/components/ironSwarm/ProjectManifestWizard.tsx
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/skills.py
- web/packages/studio/src/components/ironSwarm/ReconChecklist.tsx
- web/packages/studio/src/components/ironSwarm/swarm/SwarmGraph.tsx
- web/packages/studio/src/routes/IronSwarmRunListRoute/index.tsx
- web/packages/studio/src/components/ironSwarm/useMitigations.test.ts
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/model_config.py
- web/packages/studio/src/routes/IronSwarmManifestListRoute/index.tsx
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/jobs.py
- web/packages/studio/src/routes/index.tsx
- web/packages/sdk/package.json
- web/packages/studio/src/constants/environment.ts
- web/packages/studio/src/tests/title-change.test.tsx
- plugins/nemo-iron-swarm/tests/unit/test_artifacts.py
- web/packages/studio/src/routes/groups/ironSwarmRoutes.tsx
- plugins/nemo-iron-swarm/tests/unit/_doubles.py
- plugins/nemo-iron-swarm/tests/unit/test_api_runs.py
- web/packages/studio/src/routes/IronSwarmManifestDetailRoute/index.tsx
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/_perms.py
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/authz.py
- services/studio/src/nmp/studio/env_mappings.py
- web/packages/studio/src/routes/IronSwarmRunDetailsRoute/index.tsx
- plugins/nemo-iron-swarm/tests/unit/test_compose_defense.py
- web/packages/studio/src/components/dataViews/IronSwarmRunsDataView/index.tsx
- plugins/nemo-iron-swarm/tests/unit/test_operator_env.py
- web/packages/studio/src/components/ironSwarm/useMitigations.ts
- web/packages/studio/src/api/ironSwarm.ts
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/manifest.py
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/_filters.py
- web/packages/studio/src/components/ironSwarm/SanityCheckReport.tsx
- web/packages/studio/src/routes/utils.ts
- web/packages/studio/src/components/ironSwarm/eventTypes.ts
Contributor
|
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (1)
plugins/nemo-iron-swarm/README.md (1)
183-189: 📐 Maintainability & Code Quality | 🔵 Trivial | 🏗️ Heavy liftKeep this page in one Diataxis quadrant.
This block adds tutorial steps to a README that also contains configuration reference, troubleshooting, and architecture explanation. Move those sections to linked pages, or make this README one tutorial or how-to.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/README.md` around lines 183 - 189, Restructure the README so it stays within a single Diataxis quadrant: move configuration reference, troubleshooting, and architecture explanation out of this page into linked documentation pages, or convert the entire README into a focused tutorial/how-to and remove the unrelated sections. Keep the bootstrap, setup, and doctor commands within the chosen scope.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@plugins/nemo-iron-swarm/README.md`:
- Around line 183-189: Restructure the README so it stays within a single
Diataxis quadrant: move configuration reference, troubleshooting, and
architecture explanation out of this page into linked documentation pages, or
convert the entire README into a focused tutorial/how-to and remove the
unrelated sections. Keep the bootstrap, setup, and doctor commands within the
chosen scope.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 48ef44b7-ea9d-4e84-aef6-b4220be037c3
📒 Files selected for processing (5)
plugins/nemo-iron-swarm/README.mdplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/manifests.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/runs.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/provisioning.pyweb/packages/studio/src/components/ironSwarm/swarm/useSwarmEvents.ts
🚧 Files skipped from review as they are similar to previous changes (4)
- web/packages/studio/src/components/ironSwarm/swarm/useSwarmEvents.ts
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/provisioning.py
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/runs.py
- plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/manifests.py
parkanzky
approved these changes
Aug 7, 2026
parkanzky
left a comment
parkanzky
left a comment
Contributor
There was a problem hiding this comment.
Plugin looks idiomatic enough to me. When tests pass I think we should just merge this to get it in and allow people to interact with it. However, I am not familiar enough with typescript to review the studio pieces. Finding someone to review that now.
marcusds
reviewed
Aug 7, 2026
Contributor
There was a problem hiding this comment.
Rather than including changes directly into studio I think this would benefit from using the new support studio has for plugins.
This PR has details: #594
Reach out if would like support on this. There some agent instructions that should be helpful.
Edit: since this would be the first real consumer of the studio plugin architecture I took a look at what moving it over would look like. For the most part it'll be straightforward but there are some changes I am going to make on our end to make it better. Edit: This is in #1180 and #1192
8 tasks
marcusds
requested changes
Aug 7, 2026
8 tasks
Author
|
Working on this — moving the whole Studio UI into the plugin, as suggested. Thanks for #1180 and #1192! they're what made it possible. The port is done locally and verified against a running platform: the UI now ships in the plugin wheel as a Studio bundle ( It sits on top of #1192, so I'll push once that merges and I can rebase .. pushing before then would pull its file moves into this PR's diff. Two plugin-API gaps came up while testing in the browser - commented on #1192. |
Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Events without a numeric id fell back to Date.now(), and that value then advanced the poll cursor. An epoch-millisecond cursor sits far above any real event id, so `after` matched nothing and the feed stopped for the rest of the run. It also produced duplicate React keys in MessageFeed. Skip events with no numeric id and advance the cursor from the highest real id seen. Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
- secrets_file was a client-supplied path handed straight to `init`, which reads it on the platform host. A caller could name any readable file (/proc/self/environ, another tenant's dotenv) and fold it into their manifest. Constrain it to the uploaded project directory. - Log entries interpolated user-controlled names and workspaces, so a value containing newlines could forge log records (CodeQL log injection, 7 hits). Route them through nemo_platform_plugin's existing sanitize_for_log, as nemo-evaluator already does. - `setup` echoed index_url verbatim; Artifactory embeds an access token in that URL. Reuse redact_index_url, which checks.py already applies. Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
The install line fetched install.sh from the mutable main branch and piped it to sh, so the code executed could change between readings. Pin it to v0.0.92 — the version the deployments plugin already requires — matching how the root README and CONTRIBUTING pin the uv installer. Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Wrapping the manifest id in sanitize_for_log pushed the warning past the configured width. `ruff check` was clean, so only `ruff format --check` caught it. Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
- launch_synth_service documented RuntimeError; _await_ready raises
IronSwarmRunError.
- The README said agent-source manifests are re-resolved on every run.
They are frozen targets; an agent edit reaches one only through
POST /manifests/{name}/refresh.
- `init`'s rendered file was described as inspection-only and as re-rendered
from the agent ref. `run --config` accepts it, and project sources have no
agent ref at all.
Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Every other Studio feature flag is in the sample, so a developer copying it had no way to discover this one. Defaults to false, matching the flag's off-by-default definition. Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
The test named both the local log and the downloaded member `events.jsonl`, so it passed no matter where the fallback wrote. The log is per-run (`_events_path` -> `<safe-run-name>.jsonl`) and `_save_events_fileset` uploads that exact file, so the member carries the same basename. Name both after the run so the test covers the contract rather than a coincidence. Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
`agent_fileset` is uploaded by the service, but `project_fileset` is passed in by the caller and stored verbatim, and nothing stops two manifests naming the same bundle. Deleting either manifest destroyed the other's target. Delete only the fileset we created; the uploader owns the project bundle. Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Main promoted AccessibleTitle, AccordionSection, FileUpload, ExpandableMessage, ErrorPanel, CancelJobButton, DeleteConfirmationModal, ConfirmationModal, and QuickActionsMenuRoot out of @studio/components into @nemo/common, and dropped react-router-dom in favour of react-router. The iron-swarm files still referenced the old specifiers, so they failed to resolve once the branch moved onto current main. Signed-off-by: mschwab <mschwab@nvidia.com>
The Iron Swarm UI lived in Studio: routes, dataViews, components, a feature flag, side-nav entry, and a generated client in the shared SDK. Studio carried all of it whether or not the plugin was installed. It now ships with the plugin, loaded at runtime through the Studio plugin system. Studio keeps no iron-swarm-specific code — the side-nav entry comes from the plugin's navItems() joining the existing "Governance" group, and routing from the plugin's own <Routes> under its splat mount. Plugin web bundle: - Shared singletons stay external (react, react-dom, react-router, KUI, react-query, @nemo/common), verified against the built artifact. - The iron-swarm client is generated by orval from the plugin's own openapi.yaml, replacing the entry in the Studio SDK. Its mutator reads the access token the host hands the plugin rather than Studio's OIDC axios interceptor, which does not cross the plugin boundary. - Platform calls go through host.sdk.platform. CreateSecretModal and the agent picker are reimplemented here: Studio's copies call the SDK directly and live outside @nemo/common, and promoting them would pull the SDK into the shared vendor bundle every plugin loads. - useWorkspaceFromPath/useToast/useBreadcrumbs are shimmed over the host handle so the moved components keep their call shapes. - eslint, vitest, and typecheck mirror Studio's setup. eslint needs the TS 6 API (typescript-eslint does not support TS 7), which is why this root pins the TS 6 build the way web/'s root does. Adds host.apiBaseUrl to the plugin contract. A plugin calling its own service needs it: Studio's dev-server /apis proxy is opt-in, so a relative request hits the dev server whenever VITE_PLATFORM_BASE_URL is set. Adds ErrorPanel, CancelJobButton, ControlledTextArea, ENTITY_NAME_HELP, and entityNameSchema to the @nemo/common plugin surface. Signed-off-by: mschwab <mschwab@nvidia.com>
Studio's Tailwind scans web/packages/** and nothing else, so a utility class Studio does not already emit has no CSS once the code lives under plugins/. Nothing catches this: the build, typecheck, and tests all pass, and it surfaces only as unstyled UI. 28 of the 32 palette classes the migrated UI used were in that state, along with ~20 layout utilities (pr-1, top-2, w-32, leading-5, divide-y, lg:grid-cols-2, and the arbitrary sizes). Neutral greys map onto Studio's semantic tokens (bg-surface-*, border-base, text-subtle/primary). Categorical and status colours move to src/theme.ts, which binds Studio's --text-color-accent-* / --text-color-feedback-* properties — these are global and theme-aware, where a palette class is neither. Studio writes the same values as text-[color:var(--...)] utilities, but a plugin must not copy that form: those classes exist only while some file under web/packages/** still uses them. Layout follows the same rule — KUI Grid replaces lg:grid-cols-2, borders replace divide-y, and fixed sizes become styles. The live-feed panel is no longer a KUI Card: bounding .nv-card-content needs a descendant selector, and an arbitrary-variant class is never emitted for a plugin. Every class the plugin now uses is verified present in Studio's scanner input. Inline style is no longer banned by the plugin's eslint config, since binding theme custom properties is the only theme-aware option a plugin has. Signed-off-by: mschwab <mschwab@nvidia.com>
react-diff-viewer-continued moved to the iron-swarm plugin's own pnpm root, but only Studio's package.json was updated — the lockfile still carried it and its transitives (emotion, refractor, prismjs types, hastscript). Regenerated with the pinned pnpm; `--frozen-lockfile` now passes. Signed-off-by: mschwab <mschwab@nvidia.com>
Adding ErrorPanel and CancelJobButton to @nemo/common's plugin surface pulled axios into the vendor bundle — ErrorPanel through api/common/utils, CancelJobButton through @nemo/sdk/generated/platform/api. axios resolves to its Node build there, so vendor/common.js emitted bare imports of crypto, http, https, url, events, stream and zlib. The browser cannot resolve those, so the dynamic import of the plugin threw: [plugins] Failed to load plugin "iron-swarm": TypeError: Failed to resolve module specifier "crypto" loadPlugin catches that, logs a warning and returns null, so the only symptom is a missing nav item. This broke every plugin's bundle, not just iron-swarm — common.js is shared. Both exports are removed, with comments recording why. ErrorPanel was dead weight anyway: its only caller was ironSwarmRoutes.tsx, deleted when the UI moved. CancelJobButton is reimplemented in the plugin on host.sdk.platform, the same way CreateSecretModal already was and for the same reason. Every served vendor bundle now imports only import-map specifiers. Signed-off-by: mschwab <mschwab@nvidia.com>
The host.apiBaseUrl paragraph landed in #1270 without a worked example, because the plugin it describes was not on main yet. Signed-off-by: mschwab <mschwab@nvidia.com>
Both of these existed in the plugin only because Studio had no shareable version when the UI moved. Main now has both. CreateSecretModal moved into @nemo/common in #1263, kept SDK-free by taking an onCreate callback — so the caller owns the mutation and the barrel stays clear of axios. The plugin's ~150-line copy is deleted; the mutation now comes off host.sdk.platform and result messages route through host.notifications.notify via onNotify, since ToastProvider is not shared across the plugin boundary. host.sdk gained an `agents` client, so the hand-rolled paginating fetch against this plugin's own fetcher is replaced by useAgentsForSelect, which walks pagination on Studio's authenticated axios and shared cache. The plugin no longer reimplements auth for a service it does not own. Bundle drops to 814 kB. Verified the plugin bundle still imports only import-map specifiers, and that no vendor bundle emits a Node builtin. Signed-off-by: mschwab <mschwab@nvidia.com>
Asserts get_studio_spec() returns the iron-swarm spec and that the built bundle it points at actually ships, so a missing dist/index.js fails the suite rather than the browser. Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
The UI ships with the plugin now, so studio.feature_flags.iron_swarm_enabled no longer exists — the README still told users to set it. Also documents the web bundle: how to rebuild it, and the Tailwind constraint that only shows up as unstyled UI. Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
The five ConfirmationModal/DeleteConfirmationModal call sites did not pass onNotify, so every success and error message was dropped with a console warning instead of reaching Studio's toaster — useNotify's fallback made it silent rather than fatal. Also points the typecheck script at tsc6, the binary the pinned @typescript/typescript6 actually installs. Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
koralchapnik
force-pushed
the
iron-swarm-plugin/koralchapnik
branch
from
August 13, 2026 07:21
635d205 to
377db23
Compare
…t docstring change 335bd4d reworded delete_manifest's docstring but left the committed spec stale, so lint-openapi — which regenerates and diffs — failed. Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
A plugin's Studio UI ships as a minified bundle committed next to its Python package so the wheel can install it. Scanning it surfaced five high-severity alerts (ReDoS, unanchored regexes, double escaping) that all belong to bundled third-party code — prismjs/refractor via react-diff-viewer-continued — which we cannot patch. The sources that produce the bundle are still scanned. Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
marcusds
approved these changes
Aug 13, 2026
| import { FC, useCallback, useEffect, useRef, useState } from 'react'; | ||
| import { useParams } from 'react-router'; | ||
|
|
||
| type AttackIntensity = 'light' | 'standard' | 'thorough'; |
Contributor
There was a problem hiding this comment.
Since this file is so large, I would split the types into a types.ts and the consts in a consts.ts
| } | ||
| }; | ||
|
|
||
| const downloadCsv = () => { |
Contributor
There was a problem hiding this comment.
Similarly to the consts and types, I would move some of these functions out to a utils.ts file to make this component more maintainable. Would require a little work on to move some items to arguments, but an agent should be able to do it really easily for you.
Contributor
There was a problem hiding this comment.
Though in this specific example we actually have a shared triggerDownload you should be able to use.
| } | ||
| /> | ||
|
|
||
| {gen.active ? ( |
Contributor
There was a problem hiding this comment.
I would personally look into how you could split this component up into sub components (in a dir like ./components), its better for performance and maintability.
|
|
||
| const NAME_PATTERN = /^[a-z0-9][a-z0-9-]*$/; | ||
|
|
||
| const schema = z.object({ |
Contributor
There was a problem hiding this comment.
We usually by practice put schema's in their own files.
marcusds
reviewed
Aug 13, 2026
| : !lastHitlogFileset)) || | ||
| (benignSource === 'upload' && (!benignFileset || uploadBenign.isPending)) | ||
| } | ||
| onSubmit={() => start()} |
Contributor
There was a problem hiding this comment.
Should have a e.preventDefault() here
onSubmit={(e) => {
e.preventDefault();
start();
}
marcusds
reviewed
Aug 13, 2026
|
|
||
| // Re-seed the launch dialog's config from the manifest default each time it opens, so per-run tweaks | ||
| // are ephemeral (they never persist unless the user hits "Save as default"). | ||
| useEffect(() => { |
Contributor
There was a problem hiding this comment.
Any manifest refetch while the dialog is could wipe user's entries. I'd move this logic into openRunDialog
marcusds
reviewed
Aug 13, 2026
| <Button color="brand" type="submit" disabled={isCreating}> | ||
| {isCreating ? 'Creating…' : 'Create Manifest'} | ||
| </Button> | ||
| <Button kind="tertiary" onClick={onReset}> |
Contributor
There was a problem hiding this comment.
add a type="button" so we don't submit the form.
marcusds
reviewed
Aug 13, 2026
| // so the job can resume benign-suite synthesis. | ||
| export const InterviewPanel: FC<InterviewPanelProps> = ({ prompt, loading, onSubmit }) => { | ||
| const [answers, setAnswers] = useState<Record<string, string>>(() => | ||
| Object.fromEntries(prompt.questions.map((q) => [q.gap, defaultAnswer(q.options)])) |
Contributor
There was a problem hiding this comment.
Could this cause state to get reused in additional rounds?
marcusds
reviewed
Aug 13, 2026
| }, | ||
| }); | ||
|
|
||
| return { report: query.data, isLoading: Boolean(jobName) && !hasReport, hasReport }; |
Contributor
There was a problem hiding this comment.
If a job fails, will it have a report? If so, will this always be stuck loading if failed?
marcusds
reviewed
Aug 13, 2026
| query: { | ||
| enabled: Boolean(jobName), | ||
| refetchInterval: (query) => | ||
| query.state.data?.data?.some((result) => result.name === MITIGATIONS_RESULT) |
Contributor
There was a problem hiding this comment.
Perhaps add a job status related escape hatch?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the
nemo-iron-swarmplugin: a security war-game that red-teams and hardens NAT agents. garakattackers probe a sandboxed copy of the agent, defenders generate guardrails and OpenShell network
policy, and validators replay the attacks plus a benign suite to confirm the fix blocked the
attack without breaking ordinary behaviour. Before this, there was no way to measure whether a
hardening change actually helped; after it,
harden → apply → re-rungives a comparable answer.The plugin contributes a service, a job, a CLI, an SDK namespace, two entities and an agent skill.
It never imports iron-swarm: garak pulls
litellm → httpx>=0.28andtorch, which conflict withnvidia-nat'shttpx~=0.27, so iron-swarm is provisioned into its own venv bynemo iron-swarm setupand driven by subprocess. Nothing breaks when it isn't installed, and theStudio tab is behind
studio.feature_flags.iron_swarm_enabled, off by default.Changes
The feature
init --agent <name>for any registered agent (it need not bedeployed), or
init --project-dir <path>for a local NAT project with nothing registered. Bothsave the same
IronSwarmManifest, so--manifest-idis the single handle afterwards. CLI andStudio go through the same
POST /manifests.runis a pure consumer of the benign suite. It never synthesises;synth-benignis aseparate command and job, and the reviewed suite is cached on the manifest. A missing suite fails
fast rather than doing something surprising.
initresolves once and stores the resulting scaffold as afileset; runs download it instead of re-resolving, so two runs are comparable — which is what a
"did the hardening help?" answer depends on. Agent edits land via an explicit
POST /manifests/{name}/refresh, whichapply-mitigationcalls automatically.git ls-files --exclude-standardand drop.env*/*.pem/*.key, reusingnemo-agents'DOCKERIGNORE_TEMPLATE. Victim secrets come from the platform Secrets store by name.refresh and an editor for non-secret env vars.
Integration with
mainThe branch was 251 commits behind, so this now includes a merge of
mainplus the fixes thatrequired:
main's anonymizer landed in the same slots (routes,route params, gate helpers, nav, route groups).
intakeEnabledkeepsmain's newtruedefault,and the dead
traceIdroute param is dropped —mainreworked Intake to be session-based.react-router-domtoreact-router;mainmoved Studio toreact-router@8.3.0and the old specifier is in no lockfile.pnpm install --frozen-lockfileaborted type generation and the image build failed.nemo-iron-swarm-pluginis installed in thedevdependency group. CI runsuv run --frozen,which syncs only default groups, so workspace membership alone left the package unimportable —
the plugin's tests could not import it and the OpenAPI generator could not build its app.
plugins/nemo-iron-swarm/tests/unitregistered on the pytestpythonpath, so the sibling_doubleshelper resolves when pytest runs from the repo root.main(db_versionfor optimistic locking,output_location, and theWarGameSpecOutput→WarGameSpecrename).Review fixes
Date.now(), which then advanced the poll cursor far above any real id so the server returnednothing for the rest of the run.
secrets_fileis constrained to the uploaded project directory. It is caller-supplied and read byiniton the platform host, so it could previously name any readable file.sanitize_for_logbefore logging (7 CodeQL log-injectionalerts, now cleared).
setupredactsindex_url; Artifactory embeds an access token in that URL.delete_manifestno longer deletesproject_fileset. That fileset is supplied by the caller andtwo manifests can name the same bundle, so deleting either used to destroy the other's target.
agent_filesetis service-created and is still cleaned up.usePluginInstalled('iron-swarm'), matching thenemo-agentspattern, and fail open so an unresolved manifest never hides the feature.v0.0.92instead of fetchinginstall.shfrommain.initoutput, and adocstring naming the wrong exception).
Type of Change
Quality Gates
281 unit tests in
plugins/nemo-iron-swarm/tests. Two were added or corrected for the review fixes:a regression test asserting
delete_manifestleaves a caller-suppliedproject_filesetalone(verified to fail against the previous behaviour), and the events-fallback test now names the log
after the run instead of
events.jsonl, so it exercises the real_events_pathcontract rather thanpassing by coincidence.
Verification
Signed-off-by:traileruv run pre-commit run -apasses, or any blocked checks are identified belowTargeted validation:
uv run pre-commit run -auv run --frozen pytest plugins/nemo-iron-swarm/testsbash tools/lint/lint-python-style.sh(ruff check+ruff format --check)bash tools/lint/lint-python-types.sh(ty)pnpm --filter nemo-studio-ui typecheckpnpm --filter nemo-studio-ui lint--max-warnings 0pnpm --filter nemo-studio-ui testpnpm run format(Prettier)cd web && pnpm gen:checkuv lock --checkNot verified locally, called out rather than claimed:
nmp-studio-uiDocker build could not be run here (buildxunavailable, and the legacybuilder rejects
$BUILDPLATFORM). The fix was validated by checking that every plugin spec orvalreads has a matching
COPY, and CI's Build CPU smoke images subsequently passed.Service Unavailable/Bad Gateway/Failed to resolve action download infoduring Set up job). Onelint-web-sdkfailure occurredin that window and could not be reproduced locally under Node 26, Node 22.23.1, or a fresh
--frozen-lockfileinstall with forced regeneration. It needs a clean run to confirm.Security notes for review
overrides.defenders[].implementationis aPython import path that iron-swarm loads in the job process (it logs
importing user-configured module … ensure the source is trusted), so it executes on the platform host with the job'senvironment and Docker access — not inside the victim sandbox. A free-text YAML editor would turn
manifest-write permission into code execution.
envon a manifest is stored in plaintext and documented as non-secret; credentials usesecrets,which stores only names and resolves values from the Secrets store at run time.
Correct for local/dev, explicitly not a production deployment — containerising the orchestrator
and decoupling the sandbox from a host Docker daemon is the Phase-2 item.
apply-mitigationwrites another plugin's entity (Agent.config) while holding onlyiron-swarm.runs.apply, because the platform fail-closes on permission ids outside a service's ownnamespace. Treat that permission as an agent-write grant when assigning it.
Known issues, logged rather than hidden
then fails on a name collision, so one fault presents as a different fault each time. Cancellation
tears down correctly — the failure paths do not.
completedwhile persisting nothing, and a cachedbenign_interviewis not cleared when a new suite lands without one.sanity_checksubmits a job and returns immediately, so a failed validation cannot yet be gated onin CI. A
--waitflag that polls to a terminal status is the follow-up.unitmarker, somake test-unit's-m unitfilter deselects them.This is repo-wide (
nemo-anonymizerandnemo-agentsare the same) and is left alone here.Manual verification
Real war-games against the bundled
react-agentand theresearchagent fromagents-labuploadedas a project. The second is the interesting result — attacks 11/11 blocked, benign 8/12 passed
(3 refused, 1 error): the defenses stopped everything but broke a third of normal functionality,
which is exactly what the benign suite exists to catch.
Studio flows were driven end-to-end with Playwright: manifest creation, the target view, refresh,
the env editor, and a full benign-suite generation including the HITL interview.
Not in this PR
Routing attack and detection through the NeMo Auditor (NAIS-356/357) is prototyped on a separate
branch and deliberately excluded here.
Summary by CodeRabbit
New Features
Documentation
Tests