Skip to content

feat(insights): enable profile-driven analysis - #718

Merged
callingmedic911 merged 17 commits into
mainfrom
feat/insights-profile-dx
Jul 16, 2026
Merged

feat(insights): enable profile-driven analysis#718
callingmedic911 merged 17 commits into
mainfrom
feat/insights-profile-dx

Conversation

@callingmedic911

@callingmedic911 callingmedic911 commented Jul 15, 2026

Copy link
Copy Markdown
Member

Summary

  • Run Insights analysis and doctor from optimizer.yaml with local .env support.
  • Share a validated local Insights handoff with Optimizer.
  • Restore a captured testbed into a fresh workspace with --into.

Test plan

  • uv run --group insights pytest plugins/nemo-insights/tests/ -q
  • uv run ruff check plugins/nemo-insights/
  • uv run --frozen ty check plugins/nemo-insights/src/nemo_insights_plugin/

Summary by CodeRabbit

  • New Features
    • Added Insights profile discovery/loading with .env support, base-URL precedence, and grouped preflight readiness checks.
    • Enhanced nemo insights analysis/doctor flows with runtime resolution of agent/workspace/base-URL and improved target insights output handling.
    • Added non-idempotent testbed restore --into for fresh/empty single-workspace restores, plus direct-restore validation and fidelity guard workflow behavior.
  • Bug Fixes
    • Enforced UTF-8 for Insights YAML reads/writes and improved client-construction error reporting.
  • Documentation
    • Expanded CLI, Insights file, and restore documentation with precedence, defaults, and target constraints.
  • Tests
    • Added/extended unit, CLI, and testbed coverage for contracts, preflight, encoding, restore, and release/workflow behavior.

@github-actions

Copy link
Copy Markdown
Contributor

@github-actions

github-actions Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor
Suite Lines Covered Line Rate Branch Rate
Unit Tests 25533/32734 78.0% 62.6%
Integration Tests 14732/31383 46.9% 19.3%

@callingmedic911
callingmedic911 force-pushed the feat/insights-profile-dx branch from 97b09f2 to 04d6dc0 Compare July 15, 2026 22:08
@callingmedic911
callingmedic911 marked this pull request as ready for review July 16, 2026 17:38
@callingmedic911
callingmedic911 requested review from a team as code owners July 16, 2026 17:38
@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The change adds profile-driven Insights analysis and readiness checks, direct empty-target testbed restoration, and repository-scoped release workflows with updated documentation and tests.

Changes

Insights analysis workflow

Layer / File(s) Summary
Profile and file contracts
plugins/nemo-insights/src/nemo_insights_plugin/contracts/*, plugins/nemo-insights/src/nemo_insights_plugin/profile.py, plugins/nemo-insights/tests/contracts/*, plugins/nemo-insights/tests/test_profile.py
Adds profile discovery/loading, .env parsing, base-URL resolution, agent-spec lookup, and Insights YAML validation contracts.
Readiness checks and reports
plugins/nemo-insights/src/nemo_insights_plugin/contracts/checks.py, plugins/nemo-insights/src/nemo_insights_plugin/preflight.py, plugins/nemo-insights/tests/test_preflight.py, plugins/nemo-insights/tests/contracts/test_checks.py
Adds structured check results, grouped reports, profile/spec checks, credential checks, readiness probes, and advisory handling.
Analyze and doctor orchestration
plugins/nemo-insights/src/nemo_insights_plugin/cli.py, plugins/nemo-insights/src/nemo_insights_plugin/analyst/*, plugins/nemo-insights/tests/test_cli_profile.py, plugins/nemo-insights/README.md, plugins/nemo-insights/pyproject.toml
Routes analyze and doctor through profile resolution, preflight validation, output validation, UTF-8 persistence, client error handling, and consolidated command errors.

Direct testbed restore

Layer / File(s) Summary
Explicit workspace restore routing
plugins/nemo-insights/testbed/cli.py, plugins/nemo-insights/testbed/reingest.py, plugins/nemo-insights/testbed/README.md, plugins/nemo-insights/tests/testbed/test_cli.py
Adds restore --into WORKSPACE, single-workspace mapping validation, and direct-restore documentation and tests.
Empty-target ingestion guards
plugins/nemo-insights/testbed/reingest.py, plugins/nemo-insights/tests/testbed/test_reingest.py
Adds require_empty checks and pre-write rechecks for spans, annotations, and evaluator results while retaining default restore behavior.

Testbed release workflow

Layer / File(s) Summary
Repository-scoped release operations
plugins/nemo-insights/testbed/release.py, plugins/nemo-insights/tests/testbed/test_release.py
Adds configurable state-repository selection, repository-scoped GitHub operations, access checks, and precise missing-release handling.
Verified candidate publishing
plugins/nemo-insights/testbed/publish.py, plugins/nemo-insights/tests/testbed/test_publish.py
Updates release creation, immutable uploads, reason fallback, authentication handling, and publish assertions.
Review-gated CI controls
.github/workflows/insights-testbed.yml, plugins/nemo-insights/testbed/README.md
Removes CI release publication, adds round-trip candidate validation, configures the review environment, and uses read-only release credentials.

Sequence Diagram(s)

sequenceDiagram
  participant User
  participant InsightsCLI
  participant Profile
  participant Preflight
  participant Analyst
  User->>InsightsCLI: run analyze or doctor
  InsightsCLI->>Profile: discover and load optimizer.yaml
  Profile-->>InsightsCLI: resolved analysis inputs
  InsightsCLI->>Preflight: run readiness checks
  Preflight-->>InsightsCLI: required failures and advisories
  InsightsCLI->>Analyst: run analysis when required checks pass
  Analyst-->>InsightsCLI: insights result or execution error
Loading
sequenceDiagram
  participant User
  participant RestoreCLI
  participant WorkspaceMapping
  participant IngestBundle
  User->>RestoreCLI: restore with --into WORKSPACE
  RestoreCLI->>WorkspaceMapping: validate target and single-workspace bundle
  WorkspaceMapping-->>RestoreCLI: exact source-to-target mapping
  RestoreCLI->>IngestBundle: ingest with require_empty=True
  IngestBundle-->>RestoreCLI: restore outcome or empty-target failure
Loading

Suggested labels: feat

Suggested reviewers: nicot

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 15.03% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: profile-driven Insights analysis.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/insights-profile-dx

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🧹 Nitpick comments (2)
plugins/nemo-insights/README.md (2)

15-15: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Use the configured product-name substitutions.

Replace the hardcoded Insights product references with the project’s Sphinx substitutions.

As per coding guidelines, “Never hardcode product names; use substitutions in Sphinx configuration to maintain consistency across documentation.”

Also applies to: 25-25, 33-36

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@plugins/nemo-insights/README.md` at line 15, Update the product-name
references in the README, including the discovery description and the sections
around the additional referenced lines, to use the project’s configured Sphinx
substitution instead of hardcoded “Insights” text. Preserve the existing
documentation wording and behavior while applying the established substitution
consistently.

Source: Coding guidelines


15-49: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Separate the CLI how-to from the profile reference.

Lines 15-23 are HOW-TO content; Lines 25-49 are REFERENCE content. Move the contract details to a reference page and cross-link it.

As per coding guidelines, “Each documentation page should fit ONE Diataxis quadrant; do not mix tutorials with reference tables or how-tos with architecture explanations; use cross-links instead.”

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@plugins/nemo-insights/README.md` around lines 15 - 49, Separate the command
walkthrough from the profile contract in the README: retain only the user-facing
how-to steps there, move the YAML fields, environment resolution, profile
discovery, and output behavior into a dedicated reference page, and add a clear
cross-link from the README to that page. Keep the documented behavior unchanged.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@plugins/nemo-insights/src/nemo_insights_plugin/analyst/analyst_backend.py`:
- Around line 479-483: The _write_records method currently reconstructs YAML
with only the insights key, discarding other validated top-level fields. Update
analyst_backend.py lines 479-483 to preserve the original document mapping when
writing, or update validation to reject unsupported top-level fields; in either
case, update test_cli_profile.py lines 594-613 to exercise real persistence and
verify metadata preservation or the expected rejection.

In `@plugins/nemo-insights/src/nemo_insights_plugin/contracts/profile.py`:
- Around line 81-86: Update the environment assignment flow around the key/value
parsing and env population logic to catch ValueError raised when a parsed value
contains invalid characters such as NUL bytes, and re-raise it as the
established EnvFileError type so the CLI retains clean error handling. Preserve
valid assignments and duplicate-key behavior.

In `@plugins/nemo-insights/src/nemo_insights_plugin/profile.py`:
- Line 17: Update the agent field in the profile model to reject whitespace-only
values, while preserving nonblank names; implement the validation at the
field/model validation layer and add a profile test covering a whitespace-only
agent name.

In `@plugins/nemo-insights/testbed/cli.py`:
- Around line 708-710: Update the CLI flow around _resolve_bundle and
reingest.explicit_workspace_map so --into syntax is validated before resolving
or downloading the bundle. After loading the manifest, retain the workspace-map
validation needed for the single-workspace constraint. Add a test covering an
invalid --into value with a published --state, in addition to the existing
local-file coverage.

In `@plugins/nemo-insights/testbed/README.md`:
- Around line 112-126: Update the restore documentation so the HOW-TO section no
longer claims every restore targets <ws>-<ref> or cannot affect other
workspaces; retain only workflow instructions there. Move the default-restore
and direct --into mode semantics to a single REFERENCE section/page, then add
cross-links between the HOW-TO and reference locations.

In `@plugins/nemo-insights/testbed/reingest.py`:
- Around line 635-643: Make fresh-target enforcement atomic in
test_require_empty_ingests_all_nonempty_collections_in_order by replacing the
client-side annotation/result recount and later span write sequence with the
server-side workspace reservation or atomic emptiness operation. Ensure all
non-empty collections are validated and reserved together before any
direct-restore writes, so concurrent population causes the operation to fail
without persisting partial spans, annotations, or evaluator results.

---

Nitpick comments:
In `@plugins/nemo-insights/README.md`:
- Line 15: Update the product-name references in the README, including the
discovery description and the sections around the additional referenced lines,
to use the project’s configured Sphinx substitution instead of hardcoded
“Insights” text. Preserve the existing documentation wording and behavior while
applying the established substitution consistently.
- Around line 15-49: Separate the command walkthrough from the profile contract
in the README: retain only the user-facing how-to steps there, move the YAML
fields, environment resolution, profile discovery, and output behavior into a
dedicated reference page, and add a clear cross-link from the README to that
page. Keep the documented behavior unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 9172cfd5-22ae-4c2a-97aa-94964d5732f9

📥 Commits

Reviewing files that changed from the base of the PR and between 25ed758 and 04d6dc0.

📒 Files selected for processing (22)
  • plugins/nemo-insights/README.md
  • plugins/nemo-insights/pyproject.toml
  • plugins/nemo-insights/src/nemo_insights_plugin/analyst/analyst_backend.py
  • plugins/nemo-insights/src/nemo_insights_plugin/analyst/run.py
  • plugins/nemo-insights/src/nemo_insights_plugin/cli.py
  • plugins/nemo-insights/src/nemo_insights_plugin/contracts/checks.py
  • plugins/nemo-insights/src/nemo_insights_plugin/contracts/insights.py
  • plugins/nemo-insights/src/nemo_insights_plugin/contracts/profile.py
  • plugins/nemo-insights/src/nemo_insights_plugin/preflight.py
  • plugins/nemo-insights/src/nemo_insights_plugin/profile.py
  • plugins/nemo-insights/testbed/README.md
  • plugins/nemo-insights/testbed/cli.py
  • plugins/nemo-insights/testbed/reingest.py
  • plugins/nemo-insights/tests/contracts/test_checks.py
  • plugins/nemo-insights/tests/contracts/test_insights.py
  • plugins/nemo-insights/tests/contracts/test_profile_contract.py
  • plugins/nemo-insights/tests/test_cli_profile.py
  • plugins/nemo-insights/tests/test_periodic_analysis.py
  • plugins/nemo-insights/tests/test_preflight.py
  • plugins/nemo-insights/tests/test_profile.py
  • plugins/nemo-insights/tests/testbed/test_cli.py
  • plugins/nemo-insights/tests/testbed/test_reingest.py
💤 Files with no reviewable changes (1)
  • plugins/nemo-insights/pyproject.toml

Comment thread plugins/nemo-insights/src/nemo_insights_plugin/contracts/profile.py
Comment thread plugins/nemo-insights/src/nemo_insights_plugin/profile.py
Comment thread plugins/nemo-insights/testbed/cli.py
Comment thread plugins/nemo-insights/testbed/README.md
Comment thread plugins/nemo-insights/testbed/reingest.py
@callingmedic911 callingmedic911 changed the title Enable profile-driven Insights analysis feat(insights): enable profile-driven analysis Jul 16, 2026
@github-actions github-actions Bot added the feat label Jul 16, 2026
Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>
Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>
Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>
Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>
Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>
Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>
Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>
Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>
Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>
Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>
Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>
Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>
Replace a nested ternary in make_check_result with an explicit
if/elif/else, and fold load_env_file's path.is_file() probe into its
existing try/except so a permission error while checking existence is
wrapped as EnvFileError instead of leaking a raw OSError. Missing-file
no-op behavior is unchanged; adds one focused regression test.

Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>
Read environment files directly so missing files remain a no-op while
other metadata and read failures are consistently wrapped.

Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>
load_insights_document and validate_insights_file now both go through
one internal read/parse/validate helper instead of validate doing a
separate stat() before the read, closing a TOCTOU window; a file that
disappears at the read itself is treated as missing, same as before.
Generic OSError failures now say "could not be read" instead of the
misleading "not readable as UTF-8" (UnicodeError keeps that wording).
LocalAnalystBackend's file read/write now pass encoding="utf-8"
explicitly, matching the shared validator.

Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>
Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>
@callingmedic911
callingmedic911 force-pushed the feat/insights-profile-dx branch from 04d6dc0 to 9498b45 Compare July 16, 2026 19:13

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
plugins/nemo-insights/README.md (1)

15-49: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Separate the HOW-TO from the profile reference.

Lines 15-23 are a HOW-TO, while Lines 25-49 are reference material. Split them into distinct sections or pages, add prerequisites before the commands, and add a Next Steps section with cross-links.
As per coding guidelines: each documentation page should fit one Diataxis quadrant, list prerequisites at the top, and include Next Steps.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@plugins/nemo-insights/README.md` around lines 15 - 49, Restructure the README
so the command walkthrough and profile contract are separate documentation
sections or pages, keeping each focused on a single Diataxis quadrant. Add a
prerequisites section before the command examples, and finish with a Next Steps
section containing relevant cross-links to the complementary documentation.

Source: Coding guidelines

plugins/nemo-insights/tests/contracts/test_profile_contract.py (1)

73-78: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add the missing home-path assertion.

The test name promises ~ expansion coverage but never supplies a home-relative path.

Proposed test
-def test_resolve_profile_path_handles_relative_absolute_and_home(tmp_path: Path) -> None:
+def test_resolve_profile_path_handles_relative_absolute_and_home(
+    tmp_path: Path,
+    monkeypatch: pytest.MonkeyPatch,
+) -> None:
+    monkeypatch.setenv("HOME", str(tmp_path))
     relative = resolve_profile_path("./agent", tmp_path)
     absolute = resolve_profile_path(str(tmp_path / "agent"), Path("/elsewhere"))
+    home = resolve_profile_path("~/agent", Path("/elsewhere"))

     assert relative == (tmp_path / "agent").resolve()
     assert absolute == (tmp_path / "agent").resolve()
+    assert home == (tmp_path / "agent").resolve()
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@plugins/nemo-insights/tests/contracts/test_profile_contract.py` around lines
73 - 78, Extend test_resolve_profile_path_handles_relative_absolute_and_home to
call resolve_profile_path with a home-relative "~" path and assert it resolves
to the expected home-directory location, preserving the existing relative and
absolute assertions.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@plugins/nemo-insights/README.md`:
- Around line 41-44: Update the README precedence statement to match the CLI
handling in the argument-resolution logic: state that agent is required and
raises when absent from both --agent and the profile, workspace and the base URL
use their built-in defaults, and agent_spec falls back to conventional
profile-local files rather than a built-in default. Preserve the documented
precedence of explicit flags over profile values and NMP_BASE_URL, including
--base-url over NMP_BASE_URL.

---

Nitpick comments:
In `@plugins/nemo-insights/README.md`:
- Around line 15-49: Restructure the README so the command walkthrough and
profile contract are separate documentation sections or pages, keeping each
focused on a single Diataxis quadrant. Add a prerequisites section before the
command examples, and finish with a Next Steps section containing relevant
cross-links to the complementary documentation.

In `@plugins/nemo-insights/tests/contracts/test_profile_contract.py`:
- Around line 73-78: Extend
test_resolve_profile_path_handles_relative_absolute_and_home to call
resolve_profile_path with a home-relative "~" path and assert it resolves to the
expected home-directory location, preserving the existing relative and absolute
assertions.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: e4100963-cc51-4b49-98e0-b5dd2b5f68ca

📥 Commits

Reviewing files that changed from the base of the PR and between 04d6dc0 and 9498b45.

📒 Files selected for processing (22)
  • plugins/nemo-insights/README.md
  • plugins/nemo-insights/pyproject.toml
  • plugins/nemo-insights/src/nemo_insights_plugin/analyst/analyst_backend.py
  • plugins/nemo-insights/src/nemo_insights_plugin/analyst/run.py
  • plugins/nemo-insights/src/nemo_insights_plugin/cli.py
  • plugins/nemo-insights/src/nemo_insights_plugin/contracts/checks.py
  • plugins/nemo-insights/src/nemo_insights_plugin/contracts/insights.py
  • plugins/nemo-insights/src/nemo_insights_plugin/contracts/profile.py
  • plugins/nemo-insights/src/nemo_insights_plugin/preflight.py
  • plugins/nemo-insights/src/nemo_insights_plugin/profile.py
  • plugins/nemo-insights/testbed/README.md
  • plugins/nemo-insights/testbed/cli.py
  • plugins/nemo-insights/testbed/reingest.py
  • plugins/nemo-insights/tests/contracts/test_checks.py
  • plugins/nemo-insights/tests/contracts/test_insights.py
  • plugins/nemo-insights/tests/contracts/test_profile_contract.py
  • plugins/nemo-insights/tests/test_cli_profile.py
  • plugins/nemo-insights/tests/test_periodic_analysis.py
  • plugins/nemo-insights/tests/test_preflight.py
  • plugins/nemo-insights/tests/test_profile.py
  • plugins/nemo-insights/tests/testbed/test_cli.py
  • plugins/nemo-insights/tests/testbed/test_reingest.py
💤 Files with no reviewable changes (1)
  • plugins/nemo-insights/pyproject.toml
🚧 Files skipped from review as they are similar to previous changes (14)
  • plugins/nemo-insights/src/nemo_insights_plugin/profile.py
  • plugins/nemo-insights/tests/contracts/test_checks.py
  • plugins/nemo-insights/src/nemo_insights_plugin/analyst/run.py
  • plugins/nemo-insights/src/nemo_insights_plugin/contracts/checks.py
  • plugins/nemo-insights/tests/contracts/test_insights.py
  • plugins/nemo-insights/testbed/README.md
  • plugins/nemo-insights/tests/test_periodic_analysis.py
  • plugins/nemo-insights/src/nemo_insights_plugin/contracts/profile.py
  • plugins/nemo-insights/src/nemo_insights_plugin/analyst/analyst_backend.py
  • plugins/nemo-insights/src/nemo_insights_plugin/cli.py
  • plugins/nemo-insights/tests/testbed/test_cli.py
  • plugins/nemo-insights/tests/test_profile.py
  • plugins/nemo-insights/tests/test_cli_profile.py
  • plugins/nemo-insights/testbed/reingest.py

Comment thread plugins/nemo-insights/README.md
@callingmedic911
callingmedic911 requested a review from a team as a code owner July 16, 2026 22:06
Route fixture operations explicitly and require protected credentials so cross-repository state remains immutable and safely accessible.

Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>
@callingmedic911
callingmedic911 force-pushed the feat/insights-profile-dx branch from e249993 to 154c289 Compare July 16, 2026 22:14

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@plugins/nemo-insights/testbed/README.md`:
- Around line 69-76: Replace the literal NVIDIA-dev/NeMo-Optimizer references in
the README with the configured Sphinx substitution for the fixture repository,
including the additionally applied section. Preserve the existing guidance about
TESTBED_STATE_REPO, token permissions, and canonical automated publishing.
- Around line 348-353: Add a “Next Steps” section at the end of the README after
the secrets documentation, with cross-links to the related restore, publish, and
workflow documentation. Use the repository’s existing documentation paths and
link conventions.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: bc8a64bf-3487-4fd0-b925-3e587f343a17

📥 Commits

Reviewing files that changed from the base of the PR and between 9498b45 and 154c289.

📒 Files selected for processing (6)
  • .github/workflows/insights-testbed.yml
  • plugins/nemo-insights/testbed/README.md
  • plugins/nemo-insights/testbed/publish.py
  • plugins/nemo-insights/testbed/release.py
  • plugins/nemo-insights/tests/testbed/test_publish.py
  • plugins/nemo-insights/tests/testbed/test_release.py

Comment thread plugins/nemo-insights/testbed/README.md
Comment thread plugins/nemo-insights/testbed/README.md
@callingmedic911
callingmedic911 added this pull request to the merge queue Jul 16, 2026
Merged via the queue into main with commit c264d66 Jul 16, 2026
63 checks passed
@callingmedic911
callingmedic911 deleted the feat/insights-profile-dx branch July 16, 2026 22:45
soluwalana pushed a commit that referenced this pull request Jul 17, 2026
* feat(insights): add profile-driven analysis workflow

Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>

* Add shared readiness result contracts

Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>

* Add shared agent profile mechanics

Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>

* Add shared Insights document contracts

Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>

* Keep Insights YAML errors single-line

Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>

* Use shared contracts in Insights preflight

Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>

* Keep Insights CLI imports working

Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>

* Use shared contracts in Insights profiles

Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>

* Use unique profile contract test name

Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>

* Render Insights environment errors directly

Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>

* Route Insights CLI through shared contracts

Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>

* Cover doctor base URL resolution

Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>

* Clarify check-status branching and harden env-file existence check

Replace a nested ternary in make_check_result with an explicit
if/elif/else, and fold load_env_file's path.is_file() probe into its
existing try/except so a permission error while checking existence is
wrapped as EnvFileError instead of leaking a raw OSError. Missing-file
no-op behavior is unchanged; adds one focused regression test.

Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>

* Use one read boundary for environment files

Read environment files directly so missing files remain a no-op while
other metadata and read failures are consistently wrapped.

Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>

* Share one read boundary for Insights files

load_insights_document and validate_insights_file now both go through
one internal read/parse/validate helper instead of validate doing a
separate stat() before the read, closing a TOCTOU window; a file that
disappears at the read itself is treated as missing, same as before.
Generic OSError failures now say "could not be read" instead of the
misleading "not readable as UTF-8" (UnicodeError keeps that wording).
LocalAnalystBackend's file read/write now pass encoding="utf-8"
explicitly, matching the shared validator.

Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>

* Keep Insights Git package sources caller-owned

Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>

* Harden testbed fixture publishing

Route fixture operations explicitly and require protected credentials so cross-repository state remains immutable and safely accessible.

Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>

---------

Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>
callingmedic911 added a commit that referenced this pull request Jul 17, 2026
Post-merge review follow-up to #718. Analyze keeps other top level
keys when writing the shared insights file, .env values keep quotes
that are part of the secret, --agent alone bypasses a broken profile,
local file errors are no longer reported as platform errors, doctor
runs its checks even without a profile, analyze drops the duplicate
network probes, restore drops the mid-run recheck, the download cache
is split per repo, and the first-span guard compares timestamps
chronologically. Restores --clobber and substring release matching.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>
callingmedic911 added a commit that referenced this pull request Jul 17, 2026
Post-merge review follow-up to #718. Analyze keeps other top level
keys when writing the shared insights file, .env values keep quotes
that are part of the secret, --agent alone bypasses a broken profile,
local file errors are no longer reported as platform errors, doctor
runs its checks even without a profile, analyze drops the duplicate
network probes, restore drops the mid-run recheck, the download cache
is split per repo, and the first-span guard compares timestamps
chronologically. Restores --clobber and substring release matching.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>
fork-the-planet-automation Bot pushed a commit to fork-the-planet/NVIDIA-NeMo___nemo-platform that referenced this pull request Jul 18, 2026
…Mo#752)

Post-merge review follow-up to NVIDIA-NeMo#718. Analyze keeps other top level
keys when writing the shared insights file, .env values keep quotes
that are part of the secret, --agent alone bypasses a broken profile,
local file errors are no longer reported as platform errors, doctor
runs its checks even without a profile, analyze drops the duplicate
network probes, restore drops the mid-run recheck, the download cache
is split per repo, and the first-span guard compares timestamps
chronologically. Restores --clobber and substring release matching.

Signed-off-by: Aditya Pandey <aditypandey@nvidia.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants