feat(nemo-agents): add Fabric (spec-v1) email-phishing example - #1117
Merged
Conversation
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds a Fabric email-phishing agent example with DeepAgents orchestration, MCP-based IOC extraction, dataset generation, evaluation settings, tests, documentation, packaging, and workspace registration. ChangesEmail phishing Fabric example
Sequence Diagram(s)sequenceDiagram
participant DatasetBuilder
participant DeepAgentsOrchestrator
participant PhishingAnalyzer
participant IOCMCP
participant Evaluator
DatasetBuilder->>DeepAgentsOrchestrator: assembled email
DeepAgentsOrchestrator->>PhishingAnalyzer: analysis request
PhishingAnalyzer->>IOCMCP: extract_iocs(text)
IOCMCP-->>PhishingAnalyzer: URLs and domains
PhishingAnalyzer-->>DeepAgentsOrchestrator: YAML verdict
DeepAgentsOrchestrator-->>Evaluator: generated verdict
Evaluator->>Evaluator: compare verdict with label
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@plugins/nemo-agents/examples/email-phishing-fabric/data/build_dataset.py`:
- Around line 42-47: Update assemble_email to validate the trimmed sender value
and raise an error when the sender is missing or blank, before constructing or
writing the message. Preserve the existing subject and body handling while
ensuring invalid rows cannot be reported as successfully processed.
In `@plugins/nemo-agents/examples/email-phishing-fabric/email-phishing-eval.yml`:
- Line 34: Update dataset generation in build_dataset.py to validate that every
subject used by id_key is unique, failing generation when duplicates are
detected; alternatively, configure id_key to use an existing stable unique
identifier instead of subject. Ensure the configured id_key and generated
dataset preserve uniqueness.
In `@plugins/nemo-agents/examples/email-phishing-fabric/README.md`:
- Around line 12-16: Update the diagram code fence in the README to specify the
text language, changing the untyped fence around the orchestrator diagram to a
text fence while leaving its contents unchanged.
- Around line 1-3: Update the README with a Prerequisites section before the
description, listing uv, the nemo CLI, Platform access, the deepagents adapter,
NVIDIA_API_KEY, and the evaluation model variable; add a Next Steps section
after Status with links for dataset generation, evaluation, and deployment
guidance.
- Around line 57-59: Update the README’s multiline --input example for
email-phishing-fabric-deployment to use Bash ANSI-C quoting with $'...', so each
\n sequence becomes an actual newline before the CLI receives the input.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: c80b94c1-1896-4dcf-bea9-5f3bf0d54fe0
⛔ Files ignored due to path filters (2)
plugins/nemo-agents/examples/email-phishing-fabric/data/smaller_test.csvis excluded by!**/*.csvuv.lockis excluded by!**/*.lock
📒 Files selected for processing (9)
plugins/nemo-agents/examples/email-phishing-fabric/README.mdplugins/nemo-agents/examples/email-phishing-fabric/agent.yamlplugins/nemo-agents/examples/email-phishing-fabric/data/build_dataset.pyplugins/nemo-agents/examples/email-phishing-fabric/email-phishing-eval.ymlplugins/nemo-agents/examples/email-phishing-fabric/pyproject.tomlplugins/nemo-agents/examples/email-phishing-fabric/src/email_phishing_fabric/iocs.pyplugins/nemo-agents/examples/email-phishing-fabric/src/email_phishing_fabric/mcp_server.pyplugins/nemo-agents/examples/email-phishing-fabric/tests/test_extract_iocs.pypyproject.toml
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
plugins/nemo-agents/examples/email-phishing-fabric/README.md (1)
42-48: 📐 Maintainability & Code Quality | 🔵 Trivial | 🏗️ Heavy liftKeep this page in one Diátaxis quadrant.
ShapeandTuneexplain architecture.Runis a how-to. Move the architecture content to an explanation page, or keep this README as one how-to with cross-links.As per coding guidelines, each documentation page must fit one Diátaxis quadrant.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-agents/examples/email-phishing-fabric/README.md` around lines 42 - 48, Move the architecture-oriented prose in the README’s Run section out of this how-to page and keep this page focused only on running the example. Update the section around extract_iocs so it no longer explains the deepagents adapter, command resolution, or deployment-mode behavior, and instead cross-link that material to the appropriate explanation page while preserving the run instructions here.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@plugins/nemo-agents/examples/email-phishing-fabric/README.md`:
- Around line 57-93: The README’s usage documentation must provide parallel CLI
and Python SDK workflows in tabbed local and container alternatives. Update the
local and container sections around the shown `nemo agents` commands to use the
project’s tab-set documentation convention, adding equivalent Python SDK
examples for each variant while preserving the existing CLI commands and
deployment details.
- Around line 50-66: Update the local CLI instructions before the `nemo agents
create` commands to direct users to follow `SETUP.md`, set
`NMP_BASE_URL=http://localhost:8080`, verify Platform readiness, and run
commands via `uv run nemo` or an activated `.venv`. Apply the same setup and
invocation guidance to the additional local command blocks, ensuring the MCP
console script is available on PATH.
---
Nitpick comments:
In `@plugins/nemo-agents/examples/email-phishing-fabric/README.md`:
- Around line 42-48: Move the architecture-oriented prose in the README’s Run
section out of this how-to page and keep this page focused only on running the
example. Update the section around extract_iocs so it no longer explains the
deepagents adapter, command resolution, or deployment-mode behavior, and instead
cross-link that material to the appropriate explanation page while preserving
the run instructions here.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 98592611-ae81-48b8-b795-d33728b112f2
📒 Files selected for processing (2)
plugins/nemo-agents/examples/email-phishing-fabric/README.mdplugins/nemo-agents/examples/email-phishing-fabric/agent.yaml
🚧 Files skipped from review as they are similar to previous changes (1)
- plugins/nemo-agents/examples/email-phishing-fabric/agent.yaml
Contributor
Author
|
Live-validated |
walston
force-pushed
the
astd-370-372-fabric-phishing-example/nwalston
branch
from
August 6, 2026 16:07
216b003 to
f6f054b
Compare
Contributor
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@plugins/nemo-agents/examples/email-phishing-fabric/README.md`:
- Around line 68-109: Update the README’s Status section to state that the
subprocess path, subagent delegation, and extract_iocs were live-validated,
while Docker/Kubernetes packaging remains unvalidated. Clearly label the
container deployment commands under the Container section as unvalidated, unless
those workflows are actually tested before publishing.
In
`@plugins/nemo-agents/examples/email-phishing-fabric/src/email_phishing_fabric/iocs.py`:
- Around line 27-29: Update the URL extraction logic using _URL_RE and urlsplit
to catch ValueError from malformed candidates, including fullwidth-slash cases,
so untrusted content cannot fail the tool. Require a non-empty hostname before
adding either the URL or its domain, and add a regression test covering the
malformed candidate.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 4532af6e-33a8-4f66-b9f3-160c4121913d
⛔ Files ignored due to path filters (2)
plugins/nemo-agents/examples/email-phishing-fabric/data/smaller_test.csvis excluded by!**/*.csvuv.lockis excluded by!**/*.lock
📒 Files selected for processing (9)
plugins/nemo-agents/examples/email-phishing-fabric/README.mdplugins/nemo-agents/examples/email-phishing-fabric/agent.yamlplugins/nemo-agents/examples/email-phishing-fabric/data/build_dataset.pyplugins/nemo-agents/examples/email-phishing-fabric/email-phishing-eval.ymlplugins/nemo-agents/examples/email-phishing-fabric/pyproject.tomlplugins/nemo-agents/examples/email-phishing-fabric/src/email_phishing_fabric/iocs.pyplugins/nemo-agents/examples/email-phishing-fabric/src/email_phishing_fabric/mcp_server.pyplugins/nemo-agents/examples/email-phishing-fabric/tests/test_extract_iocs.pypyproject.toml
🚧 Files skipped from review as they are similar to previous changes (6)
- pyproject.toml
- plugins/nemo-agents/examples/email-phishing-fabric/email-phishing-eval.yml
- plugins/nemo-agents/examples/email-phishing-fabric/data/build_dataset.py
- plugins/nemo-agents/examples/email-phishing-fabric/agent.yaml
- plugins/nemo-agents/examples/email-phishing-fabric/pyproject.toml
- plugins/nemo-agents/examples/email-phishing-fabric/src/email_phishing_fabric/mcp_server.py
Contributor
Author
|
Relocated the example to |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
plugins/nemo-agents/examples/nemo-agent-config/email-phishing-agent/mcps/iocs.py (1)
38-57:⚠️ Potential issue | 🟠 MajorHandle malformed URL candidates before calling
urlsplit.The URL regex accepts candidates such as
https://exa／mple.com.urlsplit()raisesValueErrorfor this input, so one crafted email can make the MCP tool fail. CatchValueErroraround parsing and hostname access, skip candidates without a hostname, and add a regression test.Proposed fix
- parsed = urlsplit(url) - hostname = parsed.hostname + try: + parsed = urlsplit(url) + hostname = parsed.hostname + except ValueError: + continue + if not hostname: + continueThis repeats the unresolved malformed-URL finding from the previous review.
#!/bin/bash set -euo pipefail target="plugins/nemo-agents/examples/nemo-agent-config/email-phishing-agent/mcps/iocs.py" tests="plugins/nemo-agents/examples/nemo-agent-config/email-phishing-agent/tests/test_extract_iocs.py" rg -n -C 6 'urlsplit|_URL_RE|hostname|def extract_iocs' "$target" rg -n -i 'malformed|fullwidth|ValueError|exa' "$tests" || true uv run python - <<'PY' from urllib.parse import urlsplit try: urlsplit("https://exa／mple.com") except ValueError: pass else: raise SystemExit("expected ValueError") PY🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-agents/examples/nemo-agent-config/email-phishing-agent/mcps/iocs.py` around lines 38 - 57, Update extract_iocs so each URL candidate is parsed with urlsplit inside ValueError handling, skipping malformed candidates and any result without a hostname while preserving valid URL and domain extraction. Add a regression test in test_extract_iocs.py covering a candidate such as https://exa／mple.com and confirming the tool does not fail.
🧹 Nitpick comments (2)
plugins/nemo-agents/examples/nemo-agent-config/email-phishing-agent/agent.yaml (1)
69-79: 🩺 Stability & Availability | 🔵 TrivialVerify the MCP command in packaged deployments.
agent.yamlresolvesemail-phishing-iocsfromPATH. The reported validation covers subprocess mode, but it does not prove that Docker or Kubernetes images install the console script and preserve the MCP handshake. Run a packaged smoke test that invokesextract_iocs.Based on the PR objectives, Docker/Kubernetes packaging remains unvalidated.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-agents/examples/nemo-agent-config/email-phishing-agent/agent.yaml` around lines 69 - 79, Add a packaged-deployment smoke test for the MCP server configured by the iocs entry in agent.yaml, covering Docker or Kubernetes packaging rather than only subprocess mode. Build/package the image, verify the email-phishing-iocs console script is available on PATH, and invoke extract_iocs through the stdio MCP handshake to confirm the tool works end to end.plugins/nemo-agents/examples/nemo-agent-config/email-phishing-agent/README.md (1)
1-3: 📐 Maintainability & Code Quality | 🔵 Trivial | 🏗️ Heavy liftKeep the README within one Diataxis quadrant.
The page combines architecture explanation, tuning reference, and how-to instructions. Choose one primary quadrant, move other material to linked pages, add
Prerequisitesbefore the overview, and addNext StepsafterStatus.As per coding guidelines, each documentation page must use one Diataxis quadrant, list prerequisites at the top, and include Next Steps at the end.
Also applies to: 35-41, 109-117
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-agents/examples/nemo-agent-config/email-phishing-agent/README.md` around lines 1 - 3, Refocus the README on a single Diataxis quadrant, moving architecture and tuning reference material to linked documentation pages while retaining only the chosen page type’s content. Add a Prerequisites section before the overview, and add a Next Steps section after Status with links to the relocated material or follow-up guidance.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@plugins/nemo-agents/examples/nemo-agent-config/email-phishing-agent/email-phishing-eval.yml`:
- Around line 5-7: Update the commented evaluation command to use the active
email-phishing-agent example path and agent name instead of
email-phishing-fabric, and rename the associated output directory from the stale
identifier to email-phishing-agent to keep generated artifacts consistent.
---
Outside diff comments:
In
`@plugins/nemo-agents/examples/nemo-agent-config/email-phishing-agent/mcps/iocs.py`:
- Around line 38-57: Update extract_iocs so each URL candidate is parsed with
urlsplit inside ValueError handling, skipping malformed candidates and any
result without a hostname while preserving valid URL and domain extraction. Add
a regression test in test_extract_iocs.py covering a candidate such as
https://exa／mple.com and confirming the tool does not fail.
---
Nitpick comments:
In
`@plugins/nemo-agents/examples/nemo-agent-config/email-phishing-agent/agent.yaml`:
- Around line 69-79: Add a packaged-deployment smoke test for the MCP server
configured by the iocs entry in agent.yaml, covering Docker or Kubernetes
packaging rather than only subprocess mode. Build/package the image, verify the
email-phishing-iocs console script is available on PATH, and invoke extract_iocs
through the stdio MCP handshake to confirm the tool works end to end.
In
`@plugins/nemo-agents/examples/nemo-agent-config/email-phishing-agent/README.md`:
- Around line 1-3: Refocus the README on a single Diataxis quadrant, moving
architecture and tuning reference material to linked documentation pages while
retaining only the chosen page type’s content. Add a Prerequisites section
before the overview, and add a Next Steps section after Status with links to the
relocated material or follow-up guidance.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 184d58cd-1942-4a96-b45a-32592020baeb
⛔ Files ignored due to path filters (2)
plugins/nemo-agents/examples/nemo-agent-config/email-phishing-agent/data/smaller_test.csvis excluded by!**/*.csvuv.lockis excluded by!**/*.lock
📒 Files selected for processing (8)
plugins/nemo-agents/examples/nemo-agent-config/email-phishing-agent/README.mdplugins/nemo-agents/examples/nemo-agent-config/email-phishing-agent/agent.yamlplugins/nemo-agents/examples/nemo-agent-config/email-phishing-agent/data/build_dataset.pyplugins/nemo-agents/examples/nemo-agent-config/email-phishing-agent/email-phishing-eval.ymlplugins/nemo-agents/examples/nemo-agent-config/email-phishing-agent/mcps/iocs.pyplugins/nemo-agents/examples/nemo-agent-config/email-phishing-agent/pyproject.tomlplugins/nemo-agents/examples/nemo-agent-config/email-phishing-agent/tests/test_extract_iocs.pypyproject.toml
Contributor
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@plugins/nemo-agents/examples/nemo-agent-config/email-phishing-agent/README.md`:
- Around line 51-53: Add a “Next Steps” section immediately after the Status
section in the email-phishing-agent README, including cross-links to the related
registration, evaluation, and trace-inspection documentation. Ensure this
section is the final content on the page.
- Around line 31-33: Move the “Prereqs” line from the “Use it in Platform (CLI)”
section to directly after the README introduction, before all other
documentation sections, while preserving its existing prerequisite details.
- Around line 12-30: Keep this README focused on a single Diataxis documentation
type by retaining either the component reference or the porting guidance, not
both. Move the other section to a dedicated documentation page and add a clear
cross-link between the pages, preserving the existing details and examples.
- Around line 36-39: Validate the documented Docker/Kubernetes deployment,
Studio workflow, and evaluation-judge tuning steps in the email-phishing-agent
README before presenting them as supported usage. If they cannot be tested, move
the affected instructions near the untested-workflows note into an explicitly
marked draft or unverified section, including the blocks around Deploy, Tune &
evaluate, and the corresponding Studio/evaluation steps.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 363bb4a3-9503-4d28-b7ca-2140db67067e
📒 Files selected for processing (1)
plugins/nemo-agents/examples/nemo-agent-config/email-phishing-agent/README.md
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Contributor
|
walston
marked this pull request as draft
August 7, 2026 19:28
walston
marked this pull request as ready for review
August 7, 2026 23:01
Port the email-phishing analyzer to a Platform-native nemo-agents-spec-v1 agent: a deepagents orchestrator that delegates classification to a phishing subagent and calls a deterministic extract_iocs MCP tool. The prompt and model live in agent.yaml (tunable) and each step emits a trace span, replacing the opaque MCP-proxy classifier. - extract_iocs ported as a stdio MCP console tool (pure regex, unit-tested) - sender-inclusive input: assembled From:/Subject:/body 'email' column so the sender (a top phishing tell) reaches the model and extract_iocs - eval config uses question_key: email - registered as a workspace member so its package resolves ASTD-370, ASTD-371, ASTD-372 Signed-off-by: Nathan Walston <nwalston@nvidia.com>
The extract_iocs stdio MCP tool is a console script that Fabric launches as a parallel child process, resolving the command on PATH. The runtime that must contain it differs by deploy mode: - subprocess (default): runs locally from the repo .venv (sys.executable, inherits PATH); the example is a workspace member so uv sync --all-packages already provides the console script — no image needed. - docker/k8s: the container lacks the package; bake it in with nemo agents package --pyproject (uv pip install .), then deploy --mode docker/k8s --image (--publish --registry for k8s). Replaces the misleading local 'uv pip install + bare deploy' instruction and the agent.yaml comment. Addresses review P1 (deployed agent could not start its MCP server under container modes). Signed-off-by: Nathan Walston <nwalston@nvidia.com>
…nt-config/ Move the Fabric (spec-v1) email-phishing example from the top-level examples dir into nemo-agent-config/email-phishing-agent/, alongside calculator-agent — that directory is where nemo-agents-spec-v1 (Fabric) examples live; the top-level dir holds NAT examples. Adopt the sibling's conventions: - flat mcps/ package (mcps.iocs) instead of src/ layout; extract_iocs util + FastMCP server combined; console script email-phishing-iocs - package nemo-agent-config-example-email-phishing; mcp>=1.28.1,<2 - agent renamed email-phishing-agent; mcp url email-phishing-iocs (default harness_native exposure); ATOF telemetry like the sibling - build_dataset.py source path fixed for the new depth Addresses review: the -fabric suffix was redundant and the example was misfiled at the top level. Signed-off-by: Nathan Walston <nwalston@nvidia.com>
Parts table, port-it steps, and mirrored Platform-CLI / Studio usage sections (same five beats: register, deploy, invoke, observe, tune & evaluate). Studio create/deploy beats marked pending the gallery tile (ASTD-08). Signed-off-by: Nathan Walston <nwalston@nvidia.com>
Lead with a Parts table that pairs each file with what to change, an explicit cross-file 'keep in sync' note (console name; workspace member), and a compact mirrored Platform/Studio run to validate the swap. Signed-off-by: Nathan Walston <nwalston@nvidia.com>
…torial Single-quadrant tutorial (prerequisites-first, numbered steps with expected outcomes, Next Steps) mirroring the calculator-agent sibling; plain markdown (these example READMEs render on GitHub, not Sphinx, so no MyST tab-sets). Move the 'swap it for your own' how-to into CUSTOMIZE.md, cross-linked from Next Steps. Addresses CodeRabbit Diataxis/prereqs/next-steps/fence/quoting findings. Signed-off-by: Nathan Walston <nwalston@nvidia.com>
- iocs.py: guard urlsplit() against unparseable netlocs - build_dataset.py: reject blank sender; fail on duplicate subjects (id_key) - email-phishing-eval.yml: replace stale email-phishing-fabric identifiers Signed-off-by: Nathan Walston <nwalston@nvidia.com>
…g-agent WIP toward ASTD-385. Adds an atif http-storage sink pointing at the local Intake ingest endpoint, alongside the existing atof file sink. Note: this does not yet produce Intake spans for the in-process DeepAgents harness (only local ATOF is emitted today); tracked in ASTD-385. Endpoint is hardcoded to the local Intake (127.0.0.1:8080), matching the sibling agent-relay-intake.yaml. Signed-off-by: Nathan Walston <nwalston@nvidia.com>
walston
force-pushed
the
astd-370-372-fabric-phishing-example/nwalston
branch
from
August 10, 2026 20:27
3aeb1b8 to
854ca84
Compare
CodeQL flagged three `"host" in result["domains"]` assertions as *Incomplete URL substring sanitization*. They are list-membership checks, not sanitization — `extract_iocs` returns sorted lists — but CodeQL cannot infer the dict value type. Assert exact list equality instead, matching the other tests in this file. The assertions get stronger (order + full contents) and the flagged pattern is gone. Signed-off-by: Nathan Walston <nwalston@nvidia.com>
steramae-nvidia
approved these changes
Aug 10, 2026
This was referenced Aug 10, 2026
15 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
demo.mov
Summary
Ports the email-phishing analyzer to a Platform-native
nemo-agents-spec-v1example atplugins/nemo-agents/examples/nemo-agent-config/email-phishing-agent/(alongside thecalculator-agentsibling — that directory is where spec-v1 / Fabric examples live; thetop level holds NAT examples).
The NAT example proxy-passes classification to an opaque MCP server — beyond the reach of
trace views, prompt/hyperparameter tuning, and evaluation. This example restructures it as a
deepagents orchestrator that delegates the verdict to a phishing sub-agent and calls a
deterministic
extract_iocsMCP tool. The prompt and model live inagent.yaml(tunable),and the sub-agent task + tool call each emit a trace span. The proxy-pass classifier becomes a
Platform-visible, tunable, evaluable step.
Changes
agent.yaml— deepagents orchestrator + adeclarative_subagent(phishing-analyzer)with its own
system_promptand a YAML verdict(
is_likely_phishing/confidence/indicators/explanation) the top-level model parses;extract_iocswired as aharness_nativestdio MCP server; relay ATIF → Intake telemetry.extract_iocsMCP tool — pure-regex URL/domain extraction (ported fromemail-security-analyst), served over stdio by theemail-phishing-iocsconsole script(flat
mcps/package, matchingcalculator-agent); unit-tested, incl. malformed-URL guard.data/build_dataset.pyassembles anemailcolumn(
From:/Subject:/body) so the sender (a top phishing tell, and anextract_iocsinput)reaches the model; the NAT eval fed
bodyonly. Eval config usesquestion_key: email.Guards against blank senders and duplicate
id_keysubjects.README.mdis a single-quadrant Diataxis tutorial (deploy → invoke → observe →evaluate);
CUSTOMIZE.mdis the how-to for swapping in your own agent.Verification
--mode subprocess, realNVIDIA_API_KEY,model
nvidia-nemotron-3-nano-30b-a3b):create→deploy→invokereturns a correctverdict (
is_likely_phishing: true,confidence: 0.96), catching the typo-squattedpaypa1-secure.examplesender domain.the LangGraph checkpointer show orchestrator →
phishing-analyzersub-agent →extract_iocstool call (with its
urls/domainsresult), routed through the Inference Gateway.release, FABRIC-181) plus the Intake fix in fix(intake): accept ATIF observation on all step types #1189 (merged).
mainis currently pinned toFabric 0.1.1, so the trace beat lights up once that release lands — the example itself needs
no change.
agent.yamlvalidates againstAgentConfig(nemo-agents-spec-v1,extra="forbid") andtranslates to a typed Fabric config;
extract_iocstests 8/8;ruff/tyclean.Not exercised: container (
--mode docker/k8s) packaging, the Studio Create-Example tile(ASTD-407), and eval judge weights/prompt (starters — tune per the evaluator plugin). The README
scopes itself to the live-validated path and links the rest.
Tickets
Resolves ASTD-370 (restructure to a tunable/traceable sub-agent), ASTD-371 (sender in the
analyzed input), and ASTD-372 (author the spec-v1
agent.yaml).Follow-ups filed: ASTD-407 (Studio Create-Example entry), ASTD-408 (specialist sub-agents),
ASTD-409 (mock-mailbox MCP). Trace wiring tracked in ASTD-385.