fix(CI): Issue 116 clean up dataviewer tests by akhanattentive · Pull Request #184 · microsoft/physical-ai-toolchain

akhanattentive · 2026-03-12T23:28:42Z

Description

Add two CI workflows establishing automated quality gates for the dataviewer application. The repository previously had zero CI coverage for every file in src/dataviewer/.

Frontend (dataviewer-frontend-tests.yml): Rewrote the existing soft-fail placeholder workflow into a hard-gated pipeline that runs npm ci → ESLint → tsc --noEmit → vite build on src/dataviewer/frontend/** changes.

Backend (dataviewer-backend-pytests.yml): New workflow that runs uv sync --extra dev → pytest with coverage on src/dataviewer/backend/** changes.

Both workflows follow the repository's reusable workflow_call pattern, include workflow_dispatch for manual runs, use SHA-pinned actions, persist-credentials: false, step-security/harden-runner, and path-scoped push/pull_request triggers. Both are wired into main.yml and pr-validation.yml orchestrators (including the release-please needs gate).

Closes #116

Type of Change

🐛 Bug fix (non-breaking change fixing an issue)
✨ New feature (non-breaking change adding functionality)
💥 Breaking change (fix or feature causing existing functionality to change)
📚 Documentation update
🏗️ Infrastructure change (Terraform/IaC)
♻️ Refactoring (no functional changes)

Component(s) Affected

deploy/000-prerequisites - Azure subscription setup
deploy/001-iac - Terraform infrastructure
deploy/002-setup - OSMO control plane / Helm
deploy/004-workflow - Training workflows
src/training - Python training scripts
docs/ - Documentation

Not listed above — this PR affects .github/workflows/ (CI workflows for src/dataviewer/).

Testing Performed

npm run lint:yaml — actionlint passes with 0 issues across all 23 workflows
Frontend gate validated locally: npm ci && npm run lint && npm run type-check && npm run build all pass
Terraform plan reviewed (no unexpected changes) — N/A
Terraform apply tested in dev environment — N/A
Training scripts tested locally with Isaac Sim — N/A
OSMO workflow submitted successfully — N/A
Smoke tests passed (smoke_test_azure.py) — N/A

Documentation Impact

No documentation changes needed
Documentation updated in this PR
Documentation issue filed

Bug Fix Checklist

N/A — not a bug fix PR.

Checklist

My code follows the project conventions
Commit messages follow conventional commit format
I have performed a self-review
Documentation impact assessed above
No new linting warnings introduced

github-actions · 2026-03-12T23:28:56Z

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

.github/workflows/dataviewer-frontend-tests.yml

codecov-commenter · 2026-03-12T23:30:29Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 38.37%. Comparing base (4496593) to head (751a6ef).

Additional details and impacted files

@@            Coverage Diff             @@
##            main     #184       +/-   ##
==========================================
+ Coverage   9.79%   38.37%   +28.58%     
==========================================
  Files         29       73       +44     
  Lines       3881     8579     +4698     
  Branches     497      497               
==========================================
+ Hits         380     3292     +2912     
- Misses      3491     5277     +1786     
  Partials      10       10

Flag	Coverage Δ	*Carryforward flag
pester	`79.87% <ø> (ø)`
pytest	`6.89% <ø> (ø)`	Carriedforward from e78a558
pytest-dataviewer	`61.98% <ø> (?)`

*This pull request uses carry forward flags. Click here to find out more.
see 44 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

## Description Resolved two CI failures blocking the release-please 0.5.0 PR (#173): **Spell Check** and **Markdown Link Check**. The Spell Check failure was caused by three unrecognized terms in the new 0.5.0 CHANGELOG section — *MLflow*, *Dependabot*, and *CSRF* — that were not present in the cspell custom dictionaries. Added each term to the appropriate dictionary file. The Markdown Link Check failure was caused by legacy repository URLs in *CHANGELOG.md* pointing to the archived `Azure-Samples/azure-nvidia-robotics-reference-architecture` location. Since all tags and history were migrated to `microsoft/physical-ai-toolchain`, replaced all 99 legacy URLs with the current repository path. The updated URLs match the existing link checker `ignorePattern`, resolving the CI failure. > These fixes are intentionally minimal and surgical — only the specific CI blockers were addressed, with no refactoring or restructuring of existing content. Related to PR #173 ## Type of Change - [x] 🐛 Bug fix (non-breaking change fixing an issue) - [ ] ✨ New feature (non-breaking change adding functionality) - [ ] 💥 Breaking change (fix or feature causing existing functionality to change) - [ ] 📚 Documentation update - [ ] 🏗️ Infrastructure change (Terraform/IaC) - [ ] ♻️ Refactoring (no functional changes) ## Component(s) Affected - [ ] `deploy/000-prerequisites` - Azure subscription setup - [ ] `deploy/001-iac` - Terraform infrastructure - [ ] `deploy/002-setup` - OSMO control plane / Helm - [ ] `deploy/004-workflow` - Training workflows - [ ] `src/training` - Python training scripts - [ ] `docs/` - Documentation ## Testing Performed - [ ] Terraform `plan` reviewed (no unexpected changes) - [ ] Terraform `apply` tested in dev environment - [ ] Training scripts tested locally with Isaac Sim - [ ] OSMO workflow submitted successfully - [ ] Smoke tests passed (`smoke_test_azure.py`) **Additional validation performed:** - `npm run spell-check` — 0 issues across 399 files after dictionary additions - `npx markdown-link-check CHANGELOG.md` — all replaced URLs confirmed IGNORED by existing pattern - Tags v0.1.0–v0.5.0 verified present on `microsoft/physical-ai-toolchain` with HTTP 200 responses ## Documentation Impact - [x] No documentation changes needed - [ ] Documentation updated in this PR - [ ] Documentation issue filed ## Bug Fix Checklist *Complete this section for bug fix PRs. Skip for other contribution types.* - [x] Linked to issue being fixed - [ ] Regression test included, OR - [x] Justification for no regression test: CI checks (Spell Check, Markdown Link Check) themselves serve as the regression mechanism — the same CI pipeline that detected these failures will validate the fix on merge. ## Checklist - [x] My code follows the [project conventions](copilot-instructions.md) - [x] Commit messages follow [conventional commit format](instructions/commit-message.instructions.md) - [x] I have performed a self-review Signed-off-by: Bill Berry <wbery@microsoft.com> Co-authored-by: Bill Berry <wbery@microsoft.com>

…ort (#180) support This PR strengthens the dataviewer's HDF5 capabilities across video generation, blob storage integration, and dataset organization. It introduces on-demand video generation from HDF5 image data, extends Azure Blob Storage support to HDF5 datasets, enables hierarchical dataset IDs up to 5 levels deep, and simplifies the frontend video playback architecture by removing the persistent frame cache in favor of native HTML5 video rendering. > The frame caching system served its purpose during initial development but added complexity that wasn't justified as HDF5 video generation matured. Native `<video>` element playback is simpler and more reliable for the generated MP4 files. The HDF5 format handler gained full video generation capabilities with an on-demand caching strategy. Videos are generated using **ffmpeg** (H.264, `ultrafast` preset, `yuv420p` pixel format) with an **OpenCV fallback** when ffmpeg is unavailable. Generated videos are cached at `meta/videos/{camera}/episode_{NNNNNN}.mp4` and uploaded to blob storage when configured. - Added `_generate_video()` and `_generate_video_cv2()` to *hdf5_handler.py* with graceful degradation between backends - Implemented `get_video_path()` with cache-first lookup and synchronous on-demand generation - Added `load_single_frame()` in *hdf5_loader.py* using h5py slice indexing for memory-efficient frame extraction - Installed **ffmpeg** system dependency in the backend *Dockerfile* Extended the blob dataset provider to discover, sync, and serve HDF5 datasets alongside existing LeRobot support. - Added `scan_all_dataset_ids()` for single-pass discovery returning both LeRobot and HDF5 dataset types - Implemented `sync_hdf5_dataset_to_local()` for metadata and placeholder sync, and `sync_hdf5_episode_to_local()` for on-demand episode downloads - Enhanced video path resolution with `_build_video_path_candidates()` supporting template-based, index-based, and fallback scan strategies - Switched from `list_blobs()` to `list_blob_names()` for faster blob enumeration - Added `upload_video()` for persisting locally generated videos back to blob storage Enabled hierarchical dataset organization using `--` separators (e.g., `project--recordings--session_1`) mapping to `/`-separated paths in storage. - Added *paths.py* utility with `dataset_id_to_blob_prefix()` centralized across storage adapters - Updated `_scan_directory()` with recursive traversal and `_validate_dataset_id()` enforcing max 5 segments - Refactored dataset grouping from `split("--", 1)[0]` to `"--".join(split("--")[:-1])` for correct multi-level grouping - Updated *DataviewerShellHeader.tsx* to display group keys with forward slashes instead of dashes Introduced a `LabelStorage` protocol in *labels.py* with **LocalLabelStorage** and **BlobLabelStorage** implementations, enabling label persistence in both local filesystem and Azure Blob Storage backends. Both implementations include path traversal protection via `realpath()` + `startswith()` validation. Removed the persistent video frame caching system and simplified the annotation workspace playback architecture. - Deleted *useVideoFrameCache.ts* (165 lines) and its test suite (244 lines) - Removed RVFC-based canvas rendering, `displayCanvasRef`, and cache state management from *useAnnotationWorkspaceVideoSync.ts* (~150 lines) - Replaced canvas-based video rendering with direct HTML5 `<video>` element in *AnnotationWorkspacePlaybackCard.tsx* - Added 200ms debounced loading overlay to prevent flicker on quick video loads - Updated *.gitignore* dataset path from `src/dataviewer/datasets/` to root-level `datasets/` - Added *.env.azure.example* template for Azure Blob Storage development setup - Enhanced *.env.example* with Azure CLI prerequisites and permission documentation - Added `HMI_DATA_PATH` environment variable to the backend dev script in *package.json* - Added Azure SDK logging suppression and lifespan-based temp directory cleanup in *main.py* - [ ] 🐛 Bug fix (non-breaking change fixing an issue) - [x] ✨ New feature (non-breaking change adding functionality) - [x] 💥 Breaking change (fix or feature causing existing functionality to change) - [ ] 📚 Documentation update - [ ] 🏗️ Infrastructure change (Terraform/IaC) - [x] ♻️ Refactoring (no functional changes) - [ ] `deploy/000-prerequisites` - Azure subscription setup - [ ] `deploy/001-iac` - Terraform infrastructure - [ ] `deploy/002-setup` - OSMO control plane / Helm - [ ] `deploy/004-workflow` - Training workflows - [ ] `src/training` - Python training scripts - [ ] `docs/` - Documentation - [ ] Terraform `plan` reviewed (no unexpected changes) - [ ] Terraform `apply` tested in dev environment - [ ] Training scripts tested locally with Isaac Sim - [ ] OSMO workflow submitted successfully - [ ] Smoke tests passed (`smoke_test_azure.py`) - [x] No documentation changes needed - [ ] Documentation updated in this PR - [ ] Documentation issue filed *Complete this section for bug fix PRs. Skip for other contribution types.* - [ ] Linked to issue being fixed - [ ] Regression test included, OR - [ ] Justification for no regression test: - [x] My code follows the [project conventions](copilot-instructions.md) - [x] Commit messages follow [conventional commit format](instructions/commit-message.instructions.md) - [x] I have performed a self-review - [x] Documentation impact assessed above - [x] No new linting warnings introduced

Reorganized the entire repository from a flat monolithic structure into a domain-driven architecture aligned with the Physical AI Toolchain lifecycle. This migration replaced numbered deployment directories (`deploy/000-*`, `deploy/001-*`, `deploy/002-*`) and a centralized `src/` directory with purpose-driven domain directories, establishing a consistent structure across all eight lifecycle domains. > The repository's directory hierarchy now reflects the conceptual architecture documented in `docs/contributing/architecture.md`, making it easier for contributors to locate code by domain rather than deployment stage. Moved 864 files across 15 commits to establish the new directory layout. Each domain follows a **consistent structure**: `README.md`, `specifications/`, `examples/`, `scripts/`, `workflows/`, and `setup/`. Core migration paths: - `deploy/000-prerequisites/` → `infrastructure/terraform/prerequisites/` - `deploy/001-iac/` → `infrastructure/terraform/` - `deploy/002-setup/` → `infrastructure/setup/` - `src/training/` → `training/rl/` - `src/dataviewer/` → `data-management/viewer/` - `src/inference/` → split across `evaluation/sil/` and `fleet-deployment/inference/` - `src/common/` → dissolved; symbols now imported from `training.rl` - `scripts/` → `shared/ci/` (CI/linting) and `shared/lib/` (shell libraries) Consolidated all infrastructure-as-code under `infrastructure/`. Terraform modules moved from *deploy/001-iac/* to *infrastructure/terraform/* with all modules preserved. Removed the **dataviewer module** from the main Terraform configuration (~120 lines of variables, outputs, and module references). Added three example tfvars files (*terraform.tfvars.dev*, *terraform.tfvars.hybrid*, *terraform.tfvars.prod*) and five infrastructure specification documents. Moved `common.sh` to *shared/lib/common.sh* as the canonical location and `terraform-outputs.sh` to *shared/lib/terraform-outputs.sh*. Created **symlinks** at `scripts/lib/terraform-outputs.sh`, `infrastructure/setup/lib/common.sh`, and within each domain's `scripts/lib/` directory for backward compatibility. CI/linting scripts and Pester tests relocated from `scripts/` to `shared/ci/`. Split training into three subdirectories: **training/rl/** (SKRL/RSL-RL with Isaac Lab), **training/il/** (LeRobot ACT/Diffusion), and **training/vla/** (scaffolded for future VLA training). Updated Python imports from `from common import cli_args` to `from training.rl import cli_args`. Updated *pyproject.toml* package sources, wheel targets, and coverage configuration to reflect the new layout. Moved the dataviewer application from *src/dataviewer/* to *data-management/viewer/*. Frontend components received formatting normalization (semicolons, Tailwind class ordering, quote style). Added **barrel export files** (`index.ts`) for each component directory, extracted new hooks (*use-export.ts*, *use-labels.ts*, *use-toast.ts*, *useTrajectoryPlotSelection.ts*), and introduced new UI primitives (*separator*, *skeleton*, *table*, *tooltip*). Created `evaluation/` by splitting *src/inference/*. Software-in-the-loop evaluation scripts moved to `evaluation/sil/`, metrics and plotting utilities to `evaluation/metrics/`, and hardware-in-the-loop scaffolded at `evaluation/hil/`. Renamed the package to **physical-ai-evaluation**. Updated submission scripts to source from new shared library paths. Introduced four new domain directories with specification documents, README files, and initial structures: - **fleet-deployment/**: GitOps bootstrapping, image automation, deployment gating, inference node - **fleet-intelligence/**: Grafana dashboards, drift detection, telemetry agents, IoT Operations setup - **synthetic-data/**: NVIDIA Cosmos integration structure (predict, reason, transfer) - **data-pipeline/**: Recording configuration and capture with schema generation Updated all 13 GitHub Actions workflow files with new directory references. Modified *package.json* npm scripts, *.cspell.json* ignore paths, Dependabot directory mappings, *.tflint.hcl*, *.vscode/tasks.json*, and *.husky/pre-commit*. Updated all Copilot artifacts (agents, skills, prompts, instructions) with new paths. Updated 130+ markdown files. Renamed `docs/deploy/` → `docs/infrastructure/` and `docs/inference/` → `docs/evaluation/`. Added documentation sections for all new domains. Fully updated *copilot-instructions.md* with the new repository structure table and all path references. - [ ] 🐛 Bug fix (non-breaking change fixing an issue) - [ ] ✨ New feature (non-breaking change adding functionality) - [x] 💥 Breaking change (fix or feature causing existing functionality to change) - [x] 📚 Documentation update - [x] 🏗️ Infrastructure change (Terraform/IaC) - [x] ♻️ Refactoring (no functional changes) - [x] `infrastructure/terraform/prerequisites/` - Azure subscription setup - [x] `infrastructure/terraform/` - Terraform infrastructure - [x] `infrastructure/setup/` - OSMO control plane / Helm - [x] `workflows/` - Training and evaluation workflows - [x] `training/` - Training pipelines and scripts - [x] `docs/` - Documentation - [ ] Terraform `plan` reviewed (no unexpected changes) - [ ] Terraform `apply` tested in dev environment - [ ] Training scripts tested locally with Isaac Sim - [ ] OSMO workflow submitted successfully - [ ] Smoke tests passed (`smoke_test_azure.py`) - [x] No documentation changes needed - [x] Documentation updated in this PR - [ ] Documentation issue filed *Complete this section for bug fix PRs. Skip for other contribution types.* - [ ] Linked to issue being fixed - [ ] Regression test included, OR - [ ] Justification for no regression test: - [x] My code follows the [project conventions](copilot-instructions.md) - [x] Commit messages follow [conventional commit format](instructions/commit-message.instructions.md) - [x] I have performed a self-review - [x] Documentation impact assessed above - [x] No new linting warnings introduced --------- Co-authored-by: katriendg <katriendg@users.noreply.github.com>

## Description Resolved two CI failures blocking the release-please 0.5.0 PR (#173): **Spell Check** and **Markdown Link Check**. The Spell Check failure was caused by three unrecognized terms in the new 0.5.0 CHANGELOG section — *MLflow*, *Dependabot*, and *CSRF* — that were not present in the cspell custom dictionaries. Added each term to the appropriate dictionary file. The Markdown Link Check failure was caused by legacy repository URLs in *CHANGELOG.md* pointing to the archived `Azure-Samples/azure-nvidia-robotics-reference-architecture` location. Since all tags and history were migrated to `microsoft/physical-ai-toolchain`, replaced all 99 legacy URLs with the current repository path. The updated URLs match the existing link checker `ignorePattern`, resolving the CI failure. > These fixes are intentionally minimal and surgical — only the specific CI blockers were addressed, with no refactoring or restructuring of existing content. Related to PR #173 ## Type of Change - [x] 🐛 Bug fix (non-breaking change fixing an issue) - [ ] ✨ New feature (non-breaking change adding functionality) - [ ] 💥 Breaking change (fix or feature causing existing functionality to change) - [ ] 📚 Documentation update - [ ] 🏗️ Infrastructure change (Terraform/IaC) - [ ] ♻️ Refactoring (no functional changes) ## Component(s) Affected - [ ] `deploy/000-prerequisites` - Azure subscription setup - [ ] `deploy/001-iac` - Terraform infrastructure - [ ] `deploy/002-setup` - OSMO control plane / Helm - [ ] `deploy/004-workflow` - Training workflows - [ ] `src/training` - Python training scripts - [ ] `docs/` - Documentation ## Testing Performed - [ ] Terraform `plan` reviewed (no unexpected changes) - [ ] Terraform `apply` tested in dev environment - [ ] Training scripts tested locally with Isaac Sim - [ ] OSMO workflow submitted successfully - [ ] Smoke tests passed (`smoke_test_azure.py`) **Additional validation performed:** - `npm run spell-check` — 0 issues across 399 files after dictionary additions - `npx markdown-link-check CHANGELOG.md` — all replaced URLs confirmed IGNORED by existing pattern - Tags v0.1.0–v0.5.0 verified present on `microsoft/physical-ai-toolchain` with HTTP 200 responses ## Documentation Impact - [x] No documentation changes needed - [ ] Documentation updated in this PR - [ ] Documentation issue filed ## Bug Fix Checklist *Complete this section for bug fix PRs. Skip for other contribution types.* - [x] Linked to issue being fixed - [ ] Regression test included, OR - [x] Justification for no regression test: CI checks (Spell Check, Markdown Link Check) themselves serve as the regression mechanism — the same CI pipeline that detected these failures will validate the fix on merge. ## Checklist - [x] My code follows the [project conventions](copilot-instructions.md) - [x] Commit messages follow [conventional commit format](instructions/commit-message.instructions.md) - [x] I have performed a self-review Signed-off-by: Bill Berry <wbery@microsoft.com> Co-authored-by: Bill Berry <wbery@microsoft.com>

…ort (#180) support This PR strengthens the dataviewer's HDF5 capabilities across video generation, blob storage integration, and dataset organization. It introduces on-demand video generation from HDF5 image data, extends Azure Blob Storage support to HDF5 datasets, enables hierarchical dataset IDs up to 5 levels deep, and simplifies the frontend video playback architecture by removing the persistent frame cache in favor of native HTML5 video rendering. > The frame caching system served its purpose during initial development but added complexity that wasn't justified as HDF5 video generation matured. Native `<video>` element playback is simpler and more reliable for the generated MP4 files. The HDF5 format handler gained full video generation capabilities with an on-demand caching strategy. Videos are generated using **ffmpeg** (H.264, `ultrafast` preset, `yuv420p` pixel format) with an **OpenCV fallback** when ffmpeg is unavailable. Generated videos are cached at `meta/videos/{camera}/episode_{NNNNNN}.mp4` and uploaded to blob storage when configured. - Added `_generate_video()` and `_generate_video_cv2()` to *hdf5_handler.py* with graceful degradation between backends - Implemented `get_video_path()` with cache-first lookup and synchronous on-demand generation - Added `load_single_frame()` in *hdf5_loader.py* using h5py slice indexing for memory-efficient frame extraction - Installed **ffmpeg** system dependency in the backend *Dockerfile* Extended the blob dataset provider to discover, sync, and serve HDF5 datasets alongside existing LeRobot support. - Added `scan_all_dataset_ids()` for single-pass discovery returning both LeRobot and HDF5 dataset types - Implemented `sync_hdf5_dataset_to_local()` for metadata and placeholder sync, and `sync_hdf5_episode_to_local()` for on-demand episode downloads - Enhanced video path resolution with `_build_video_path_candidates()` supporting template-based, index-based, and fallback scan strategies - Switched from `list_blobs()` to `list_blob_names()` for faster blob enumeration - Added `upload_video()` for persisting locally generated videos back to blob storage Enabled hierarchical dataset organization using `--` separators (e.g., `project--recordings--session_1`) mapping to `/`-separated paths in storage. - Added *paths.py* utility with `dataset_id_to_blob_prefix()` centralized across storage adapters - Updated `_scan_directory()` with recursive traversal and `_validate_dataset_id()` enforcing max 5 segments - Refactored dataset grouping from `split("--", 1)[0]` to `"--".join(split("--")[:-1])` for correct multi-level grouping - Updated *DataviewerShellHeader.tsx* to display group keys with forward slashes instead of dashes Introduced a `LabelStorage` protocol in *labels.py* with **LocalLabelStorage** and **BlobLabelStorage** implementations, enabling label persistence in both local filesystem and Azure Blob Storage backends. Both implementations include path traversal protection via `realpath()` + `startswith()` validation. Removed the persistent video frame caching system and simplified the annotation workspace playback architecture. - Deleted *useVideoFrameCache.ts* (165 lines) and its test suite (244 lines) - Removed RVFC-based canvas rendering, `displayCanvasRef`, and cache state management from *useAnnotationWorkspaceVideoSync.ts* (~150 lines) - Replaced canvas-based video rendering with direct HTML5 `<video>` element in *AnnotationWorkspacePlaybackCard.tsx* - Added 200ms debounced loading overlay to prevent flicker on quick video loads - Updated *.gitignore* dataset path from `src/dataviewer/datasets/` to root-level `datasets/` - Added *.env.azure.example* template for Azure Blob Storage development setup - Enhanced *.env.example* with Azure CLI prerequisites and permission documentation - Added `HMI_DATA_PATH` environment variable to the backend dev script in *package.json* - Added Azure SDK logging suppression and lifespan-based temp directory cleanup in *main.py* - [ ] 🐛 Bug fix (non-breaking change fixing an issue) - [x] ✨ New feature (non-breaking change adding functionality) - [x] 💥 Breaking change (fix or feature causing existing functionality to change) - [ ] 📚 Documentation update - [ ] 🏗️ Infrastructure change (Terraform/IaC) - [x] ♻️ Refactoring (no functional changes) - [ ] `deploy/000-prerequisites` - Azure subscription setup - [ ] `deploy/001-iac` - Terraform infrastructure - [ ] `deploy/002-setup` - OSMO control plane / Helm - [ ] `deploy/004-workflow` - Training workflows - [ ] `src/training` - Python training scripts - [ ] `docs/` - Documentation - [ ] Terraform `plan` reviewed (no unexpected changes) - [ ] Terraform `apply` tested in dev environment - [ ] Training scripts tested locally with Isaac Sim - [ ] OSMO workflow submitted successfully - [ ] Smoke tests passed (`smoke_test_azure.py`) - [x] No documentation changes needed - [ ] Documentation updated in this PR - [ ] Documentation issue filed *Complete this section for bug fix PRs. Skip for other contribution types.* - [ ] Linked to issue being fixed - [ ] Regression test included, OR - [ ] Justification for no regression test: - [x] My code follows the [project conventions](copilot-instructions.md) - [x] Commit messages follow [conventional commit format](instructions/commit-message.instructions.md) - [x] I have performed a self-review - [x] Documentation impact assessed above - [x] No new linting warnings introduced

Removed merge conflict markers from general-technical.txt.

Removed the Harden runner step from the workflow.

- remove dual triggers from backend and frontend reusable workflows - strip soft-fail pattern and dead steps from frontend workflow - add code-coverage input gating to backend workflow - add id-token: write and code-coverage: true in both orchestrators - update SHA pins and version comments to canonical versions - fix codecov pytest-dataviewer flag path to match actual source 🔧 - Generated by Copilot

🤖 I have created a release *beep* *boop* --- ## [0.5.0](v0.4.0...v0.5.0) (2026-03-26) ### ✨ Features * add dataviewer web application for dataset analysis and annotation ([#375](#375)) ([c44d7bb](c44d7bb)) * add return type annotations to cli_args functions ([#476](#476)) ([35523ee](35523ee)) * add YAML config schema with pydantic validation for ROS 2 recording ([#376](#376)) ([1fa5243](1fa5243)) * **agents:** Copilot agents and skills for dataviewer and OSMO training workflows. ([#444](#444)) ([8b72daf](8b72daf)) * **build:** add automated ms.date freshness checking ([#448](#448)) ([f92ddbc](f92ddbc)) * **build:** add CLA section, Dependabot security prefix, and OWASP ZAP DAST scan ([#241](#241)) ([083a8af](083a8af)) * **build:** add coverage.py configuration to pyproject.toml ([#428](#428)) ([eac7426](eac7426)) * **build:** add Go CI pipeline with golangci-lint and go test ([#351](#351)) ([b27e4fb](b27e4fb)) * **build:** add OpenSSF Scorecard workflow and badge ([#431](#431)) ([98a62e7](98a62e7)) * **build:** add release artifact signing and SBOM attestation ([#480](#480)) ([b226e96](b226e96)) * **build:** add TFLint reusable GitHub Actions workflow ([#229](#229)) ([34d5575](34d5575)) * **build:** split Go CI into separate lint and test pipelines ([#354](#354)) ([2dec155](2dec155)) * **dataviewer:** add authentication middleware and CSRF protection for mutation endpoints ([#432](#432)) ([77c8a01](77c8a01)) * **docs:** create training documentation hub with guides and migration ([#380](#380)) ([0fdccc5](0fdccc5)) * **docs:** port Docusaurus documentation site with full build validation ([#182](#182)) ([29dd640](29dd640)) * fix and deploy dataviewer ([#498](#498)) ([c922d49](c922d49)) * **inference:** add AzureML and local LeRobot inference workflows ([#438](#438)) ([f7d786a](f7d786a)) * **inference:** add MLflow trajectory plots and multi-source support to OSMO inference workflow ([#421](#421)) ([8637458](8637458)) * **infra:** add blob storage lifecycle policies and folder structure ([#179](#179)) ([101a6e8](101a6e8)) * **infrastructure:** add optional observability and compute feature flags ([#437](#437)) ([9eba0da](9eba0da)) * **infrastructure:** add private Linux Isaac Sim VM deployment option ([#348](#348)) ([3748c2d](3748c2d)) * **infrastructure:** add terraform-docs auto-generation pipeline ([#358](#358)) ([6565caa](6565caa)) * **infrastructure:** harden Isaac Sim VM deployment with encryption and spot options ([#355](#355)) ([6ebc1f2](6ebc1f2)) * **repo:** migrate to domain-driven architecture ([#270](#270)) ([a339e70](a339e70)) * **scripts:** add --config-preview and deployment summary to submission scripts ([#499](#499)) ([4069806](4069806)) * **scripts:** add Copilot attribution footer validation to frontmatter linting ([#378](#378)) ([4d595f2](4d595f2)) * **src:** add dataviewer web application with storage adapter layer ([#404](#404)) ([8a9fb70](8a9fb70)) ### 🐛 Bug Fixes * **build:** add GHSA to cspell custom dictionary ([#315](#315)) ([67db81a](67db81a)) * **build:** correct codecov report_type input for terraform test uploads ([#324](#324)) ([d90d66d](d90d66d)) * **build:** expand CODEOWNERS coverage to critical paths ([#505](#505)) ([bafade1](bafade1)) * **build:** pin Docker base image and pip dependencies with Dependabot coverage ([#497](#497)) ([d3d7ea4](d3d7ea4)) * **build:** pin pydantic version and use uv in config schema validation workflow ([#493](#493)) ([28d823f](28d823f)) * **build:** pin uv installer to versioned URL ([#495](#495)) ([8d8541b](8d8541b)) * **build:** remediate GHSA vulnerabilities flagged by OSSF Scorecard ([#271](#271)) ([49b6e58](49b6e58)) * **build:** remove README frontmatter, add FrontmatterExcludePaths, enforce Pester 5 ([#443](#443)) ([641d0f3](641d0f3)) * **build:** resolve CI failures for release 0.5.0 PR ([#174](#174)) ([62c9900](62c9900)) * **build:** resolve codecov PR comment suppression ([#523](#523)) ([5603bd7](5603bd7)) * **build:** use npm ci for deterministic frontend dependency install ([#491](#491)) ([ee8b5d3](ee8b5d3)), closes [#490](#490) * **ci:** add `wait_for_ci` to Codecov configuration ([#183](#183)) ([370cf44](370cf44)) * **CI:** Issue 116 clean up dataviewer tests ([#184](#184)) ([f466c23](f466c23)) * **ci:** pin pydantic to ==2.12.5 across all references ([#230](#230)) ([9d841d5](9d841d5)) * **dataviewer:** add HTTP Range support for blob video streaming ([#165](#165)) ([8adde50](8adde50)) * **dataviewer:** remediate CodeQL alerts and align ruff config ([#419](#419)) ([eb6fac9](eb6fac9)) * **dataviewer:** remediate path traversal and input validation vulnerabilities ([#413](#413)) ([0a1d2ca](0a1d2ca)) * **docs:** remove trailingSlash: false for GitHub Pages compatibility ([#228](#228)) ([a78cb97](a78cb97)) * **gpu:** add GPU Operator validation dependencies to GRID driver installer ([#441](#441)) ([eec42da](eec42da)) * **infrastructure:** add zone-redundant config to VPN gateway public IP ([#352](#352)) ([2d734f4](2d734f4)) * **infrastructure:** improve stdout handling for helm commands in GPU… ([#311](#311)) ([153f467](153f467)) * **infrastructure:** resolve remaining TFLint violations in SIL module and example configs ([#298](#298)) ([c0ce3e5](c0ce3e5)) * **infrastructure:** resolve TFLint violations in root and automation modules ([#287](#287)) ([b6a4604](b6a4604)), closes [#203](#203) * **infrastructure:** update deprecated bgp vng variable name ([#307](#307)) ([f530734](f530734)) * **scripts:** pin uv version in OSMO workflow templates ([#500](#500)) ([7edf13a](7edf13a)) * **scripts:** replace lambda with def in lerobot_handler to satisfy R… ([#176](#176)) ([baf9e58](baf9e58)) * **scripts:** support OSMO control-plane deploys with in-cluster Redis ([#317](#317)) ([d4b70de](d4b70de)) * **scripts:** update compute target name derivation logic ([#319](#319)) ([bb20431](bb20431)) * **settings:** update devcontainer name to match project context ([#177](#177)) ([745321e](745321e)) * **terraform:** create PostgreSQL Key Vault secret via ARM control plane ([#304](#304)) ([5d73b81](5d73b81)) * **terraform:** gate observability with feature flags ([#303](#303)) ([ea5e056](ea5e056)) * **terraform:** switch VPN gateway defaults to AZ SKUs ([#309](#309)) ([74989c5](74989c5)) * **training:** correct learning rate mapping and pin LeRobot version ([#439](#439)) ([5cf9943](5cf9943)) * **workflows:** enable SARIF upload for dependency-pinning scans ([#502](#502)) ([124cad6](124cad6)), closes [#501](#501) * **workflows:** remove redundant top-level permissions from codeql-analysis ([#489](#489)) ([1490fda](1490fda)) * **workflows:** use bash shell for uv.lock regeneration and add SARIF to dictionary ([#225](#225)) ([e6fa6ea](e6fa6ea)) ### 📚 Documentation * add chunking and compression configuration guide for Jetson edge recording ([#408](#408)) ([787a322](787a322)) * add OpenSSF Best Practices badge to README ([#282](#282)) ([01ea384](01ea384)) * add threat model cross-reference to SECURITY.md ([#235](#235)) ([88a461e](88a461e)) * add vulnerability remediation timeline to SECURITY.md ([#233](#233)) ([5ead3ee](5ead3ee)) * **contributing:** remove version-specific planning language from ownership tip ([#407](#407)) ([3191f9b](3191f9b)) * **deploy:** replace deploy/ READMEs with pointer files ([#379](#379)) ([b3c3abb](b3c3abb)) * **docs:** add bug report response timeline for OSSF report_responses criterion ([#485](#485)) ([9b26212](9b26212)) * **docs:** add component update process for OpenSSF Silver badge ([#446](#446)) ([6adc8a2](6adc8a2)) * **docs:** Add data collection and training recipes ([#343](#343)) ([9c34f86](9c34f86)) * **docs:** add deprecation policy for external interfaces ([#445](#445)) ([229d5db](229d5db)) * **docs:** add structure for recipes in repo ([#322](#322)) ([098757b](098757b)) * **docs:** add YAML frontmatter to SUPPORT.md ([#478](#478)) ([d94c15d](d94c15d)), closes [#347](#347) * **docs:** clarify issue assignment requirement before starting work ([#299](#299)) ([1534462](1534462)) * **docs:** create inference and training docs hubs ([#402](#402)) ([7a20a2e](7a20a2e)) * **docs:** create reference hub and migrate script documentation ([#503](#503)) ([03a31c6](03a31c6)) * **docs:** create training and inference documentation hubs ([#403](#403)) ([7be003b](7be003b)) * **operations:** create operations hub and troubleshooting guide ([#525](#525)) ([31c7aaa](31c7aaa)) * **reference:** add copilot artifacts documentation hub ([#170](#170)) ([9a45ca4](9a45ca4)) * simplify root README and update prerequisites ([#440](#440)) ([c0c7710](c0c7710)) ### ♻️ Code Refactoring * **build:** align Python dependency workflows with uv ([#447](#447)) ([3102e03](3102e03)) * **docs:** rename Docusaurus site to Physical AI Toolchain ([#224](#224)) ([cfdf47a](cfdf47a)) * **infrastructure:** rename boolean variables to `should_` prefix and add missing core variables ([#292](#292)) ([4496593](4496593)) * **python:** move runtime deps to workflow pyproject manifests ([#405](#405)) ([6c5fbeb](6c5fbeb)) ### 📦 Build System * **build:** add Codecov upload to pytest workflow ([#434](#434)) ([0110c17](0110c17)) * **deps-dev:** bump the npm_and_yarn group across 2 directories with 1 update ([#325](#325)) ([59cf9e6](59cf9e6)) * **workflows:** enable coverage parameters and fix Pester test infrastructure ([#435](#435)) ([528bbde](528bbde)) ### 🔧 Miscellaneous * add gomod to cspell general-technical wordlist ([#362](#362)) ([1f93f47](1f93f47)) * **build:** add codecov.yml for unified coverage reporting ([#430](#430)) ([b0faf70](b0faf70)) * **build:** add Go toolchain devcontainer feature and Dependabot gomod ([#337](#337)) ([8a36620](8a36620)) * **deps:** bump cryptography from 45.0.7 to 46.0.5 in /src/training ([#506](#506)) ([a06434e](a06434e)) * **deps:** bump minimatch in /src/dataviewer/frontend ([#416](#416)) ([38a7607](38a7607)) * **deps:** bump pyasn1 from 0.6.2 to 0.6.3 in /training/rl ([#296](#296)) ([7b42cf5](7b42cf5)) * **deps:** bump rollup in /src/dataviewer/frontend ([#417](#417)) ([6302ce4](6302ce4)) * **deps:** bump the common-dependencies group in /src/common with 3 updates ([#507](#507)) ([db05074](db05074)) * **deps:** bump the github-actions group across 1 directory with 6 updates ([#284](#284)) ([c40eff6](c40eff6)) * **deps:** bump the github-actions group across 1 directory with 6 updates ([#433](#433)) ([2d9dd4f](2d9dd4f)) * **deps:** bump the github-actions group across 1 directory with 6 updates ([#510](#510)) ([c334a64](c334a64)) * **deps:** bump the github-actions group with 2 updates ([#163](#163)) ([f25713e](f25713e)) * **deps:** bump the inference-dependencies group in /evaluation with 3 updates ([#279](#279)) ([1d2d3dc](1d2d3dc)) * **deps:** bump the inference-dependencies group in /src/inference with 5 updates ([#508](#508)) ([2852ffb](2852ffb)) * **deps:** bump the lerobot-inference-dependencies group in /workflows/azureml with 4 updates ([#511](#511)) ([b7c5773](b7c5773)) * **deps:** bump the npm_and_yarn group across 2 directories with 1 update ([#223](#223)) ([6a261ab](6a261ab)) * **deps:** bump the training-dependencies group ([#429](#429)) ([66e43f4](66e43f4)) * **deps:** bump tornado from 6.5.4 to 6.5.5 in the uv group across 1 directory ([#172](#172)) ([d6caf29](d6caf29)) * **docs:** correct ms.date tooling and refresh stale documentation ([#349](#349)) ([ccaa1e8](ccaa1e8)) * **infrastructure:** add Go module and golangci-lint config for e2e tests ([#347](#347)) ([e0e6bbf](e0e6bbf)) * **infrastructure:** add root .terraform-docs.yml configuration ([#312](#312)) ([bb73bbb](bb73bbb)) * migrate references from Azure-Samples to microsoft/physical-ai-toolchain ([f58f0ef](f58f0ef)) * **workflows:** update Dependabot, CodeQL, CODEOWNERS, and cspell for dataviewer coverage ([#231](#231)) ([6d8c2e8](6d8c2e8)) ### 🔒 Security * **deps:** bump mlflow from 3.5.0 to 3.8.0rc0 in /training/rl ([#297](#297)) ([e9929df](e9929df)) * **deps:** bump the github-actions group across 1 directory with 4 updates ([#344](#344)) ([6826929](6826929)) * **deps:** bump the inference-dependencies group in /evaluation with 2 updates ([#339](#339)) ([6804630](6804630)) * **deps:** bump the npm_and_yarn group across 3 directories with 1 update ([#361](#361)) ([6760857](6760857)) * **deps:** bump the training-dependencies group across 1 directory with 54 updates ([#286](#286)) ([d9ae04f](d9ae04f)) * **deps:** bump the uv group across 3 directories with 1 update ([#360](#360)) ([dfbda06](dfbda06)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). --------- Co-authored-by: physical-ai-toolchain-release[bot] <267194360+physical-ai-toolchain-release[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Bill Berry <wbery@microsoft.com>

akhanattentive requested a review from a team as a code owner March 12, 2026 23:28

akhanattentive changed the title ~~Issue 116 clean up dataviewer tests~~ fix(CI): Issue 116 clean up dataviewer tests Mar 12, 2026

akhanattentive commented Mar 12, 2026

View reviewed changes

Comment thread data-management/viewer/backend/tests/test_dataset_service.py

akhanattentive commented Mar 12, 2026

View reviewed changes

Comment thread codecov.yml

katriendg reviewed Mar 13, 2026

View reviewed changes

Comment thread .github/workflows/dataviewer-frontend-tests.yml

Comment thread .github/workflows/dataviewer-backend-pytests.yml Outdated

Ali Khan and others added 16 commits March 16, 2026 17:20

Issue 116 clean up dataviewer tests

08dafb9

Adding codecov and fixing failing depend

35cc81d

fixing workflow

c05ab17

fixing test assertion

89a796d

Adding ffmeg depenedency

4f7f8c0

Moving pytestcov to pyproject.toml

51c4be5

Adding continue on error

d9b2567

Issue 116 clean up dataviewer tests

f7e9752

Adding codecov and fixing failing depend

32ee322

Resolved conflict

89bd264

Adding missing commit

27fb93b

akhanattentive force-pushed the ak/issue-116 branch from 1c40915 to 27fb93b Compare March 16, 2026 22:25

akhanattentive and others added 6 commits March 16, 2026 17:28

Fix merge conflicts in general-technical.txt

0108cd5

Removed merge conflict markers from general-technical.txt.

Remove empty line from general-technical.txt

720e932

chore: remove stray files from rebase

5ae4dff

Adding missed conflict

cfe9d65

fixing conflict

e71cbd4

Updating refactored tests

5bc9a10

Ali Khan added 3 commits March 16, 2026 17:51

Adding fe test changes

8df25fa

adding dependency

0bbe4e9

Adding fixed test

e7643e0

WilliamBerryiii requested changes Mar 17, 2026

View reviewed changes

Comment thread .github/workflows/dataviewer-frontend-tests.yml Outdated

Ali Khan and others added 4 commits March 16, 2026 23:10

PR feedback

fbc6396

Merge branch 'main' into ak/issue-116

3eb719d

Remove Harden runner step from pytest workflow

1e9dbe0

Removed the Harden runner step from the workflow.

Merge branch 'main' into ak/issue-116

e093b4b

WilliamBerryiii approved these changes Mar 17, 2026

View reviewed changes

WilliamBerryiii and others added 2 commits March 17, 2026 11:40

Merge branch 'main' into ak/issue-116

e78a558

WilliamBerryiii approved these changes Mar 17, 2026

View reviewed changes

WilliamBerryiii merged commit f466c23 into main Mar 17, 2026
26 checks passed

WilliamBerryiii deleted the ak/issue-116 branch March 17, 2026 20:16

physical-ai-toolchain-release Bot mentioned this pull request Mar 17, 2026

chore(main): release 0.5.0 #173

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(CI): Issue 116 clean up dataviewer tests#184

fix(CI): Issue 116 clean up dataviewer tests#184
WilliamBerryiii merged 31 commits into
mainfrom
ak/issue-116

akhanattentive commented Mar 12, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Mar 12, 2026 •

edited

Loading

Uh oh!

codecov-commenter commented Mar 12, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

akhanattentive commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of Change

Component(s) Affected

Testing Performed

Documentation Impact

Bug Fix Checklist

Checklist

Uh oh!

github-actions Bot commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Dependency Review

Scanned Files

Uh oh!

codecov-commenter commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

akhanattentive commented Mar 12, 2026 •

edited

Loading

github-actions Bot commented Mar 12, 2026 •

edited

Loading

codecov-commenter commented Mar 12, 2026 •

edited

Loading