diff --git a/.cspell/general-technical.txt b/.cspell/general-technical.txt index 6c4df7a07..d9fd30385 100644 --- a/.cspell/general-technical.txt +++ b/.cspell/general-technical.txt @@ -8,6 +8,7 @@ acunetix adobe adrm adrs +agentic agentless agile anom @@ -160,6 +161,7 @@ chatbot chatbots chatmode chatmodes +chatlog cheatham checkbox checkboxes diff --git a/docs/contributing/architecture.md b/docs/contributing/architecture.md index 305e84afb..f3e441eb6 100644 --- a/docs/contributing/architecture.md +++ b/docs/contributing/architecture.md @@ -177,6 +177,56 @@ This repository will not maintain a separate codebase domain for simulation. Ins - Designing Isaac Lab environments using Manager-based and Direct workflows - Connecting simulation outputs to the Synthetic Data and Training domains +## Agentic Tooling + +This project uses [GitHub Copilot](https://code.visualstudio.com/docs/copilot/overview) agents, instructions, prompts, and skills to automate development workflows. Tooling comes from two sources: the HVE-Core extension (shared across Microsoft HVE projects) and project-specific artifacts defined in `.github/`. + +### HVE-Core Extension + +The [hve-core-all](https://marketplace.visualstudio.com/items?itemName=ise-hve-essentials.hve-core-all) VS Code extension provides shared agentic tooling: + +| Artifact Type | Count | Examples | +|---------------|-------|----------| +| Agents | 33 | RPI workflow, backlog management, PR creation | +| Instructions | 24 | Coding standards (Bash, C#, Python, Terraform, Bicep), commit messages, markdown | +| Prompts | 27 | ADO work items, GitHub issues, security planning, PR descriptions | +| Skills | 2 | PR reference generation, video-to-GIF conversion | + +HVE-Core artifacts are registered via the extension's `package.json` `contributes` section and loaded when the extension activates. + +### Project Copilot Artifacts + +This repository defines project-specific artifacts in `.github/` that extend HVE-Core with domain knowledge: + +| Artifact Type | Count | Purpose | +|---------------|-------|----------| +| Agents | 2 | OSMO training manager, dataviewer developer | +| Instructions | 4 | Copilot instructions, dataviewer conventions, documentation style, shell scripts | +| Prompts | 4 | OSMO training submission, LeRobot pipeline, dataviewer workflows | +| Skills | 2 | Dataviewer interaction, OSMO LeRobot training | + +Project artifacts are auto-discovered by VS Code from the `.github/` directory without explicit registration. + +Two workflow chains compose these artifacts: + +- **OSMO Training Manager**: `osmo-training-manager` agent → `osmo-lerobot-training` skill → training submission prompts +- **Dataviewer Developer**: `dataviewer-developer` agent → `dataviewer` skill → dataviewer instruction conventions + +### Artifact Types and Loading + +Each artifact type uses YAML frontmatter to declare behavior: + +| Artifact | File Pattern | Key Frontmatter | Loading | +|----------|-------------|-----------------|----------| +| Agents | `*.agent.md` | `mode`, `tools`, `description` | Auto-discovered from `.github/agents/` | +| Instructions | `*.instructions.md` | `applyTo`, `description` | Auto-discovered from `.github/instructions/` | +| Prompts | `*.prompt.md` | `mode`, `description`, `tools` | Auto-discovered from `.github/prompts/` | +| Skills | `SKILL.md` | N/A (referenced by agents) | Referenced via `copilot-skill:` URI | + +HVE-Core artifacts follow the same patterns but load through extension contribution points rather than workspace auto-discovery. + +For the detailed per-artifact inventory and workflow chain diagrams, see [Copilot Artifacts](../reference/copilot-artifacts.md). + ## Agent Skills and Specification Documents Each domain will contain specification documents alongside working examples. These specifications serve as structured inputs for [GitHub Copilot Agent Skills](https://code.visualstudio.com/docs/copilot/chat/chat-agent-mode), enabling customers to adapt this reference architecture to their own codebase and infrastructure. diff --git a/docs/contributing/prerequisites.md b/docs/contributing/prerequisites.md index 7e3899d4d..957af5573 100644 --- a/docs/contributing/prerequisites.md +++ b/docs/contributing/prerequisites.md @@ -189,6 +189,8 @@ The workspace `.vscode/settings.json` also configures Copilot Chat to load instr These paths resolve when hve-core is installed as a peer directory or via the VS Code Extension. Without hve-core, Copilot still functions but shared conventions, prompts, and chat modes are unavailable. +For a complete list of available agents, prompts, and skills, see [Copilot Artifacts](../reference/copilot-artifacts.md). + ## Related Documentation * [Contributing Guide](README.md) - Main contributing guide with all sections diff --git a/docs/reference/README.md b/docs/reference/README.md index d12ed5b3d..2424949f3 100644 --- a/docs/reference/README.md +++ b/docs/reference/README.md @@ -20,6 +20,7 @@ Technical reference documentation for submission scripts, workflow templates, en |----------------------------------------|--------------------------------------------------------------------|-------------| | [Script Reference](scripts.md) | Submission script inventory, CLI arguments, and configuration | Available | | [Script Examples](scripts-examples.md) | Detailed examples for training, inference, and pipeline submission | Available | +| [Copilot Artifacts](copilot-artifacts.md) | Agents, instructions, prompts, and skills inventory | Available | | Workflow Templates | AzureML and OSMO workflow YAML templates and configuration | Coming soon | | Terraform Variables | Terraform input variables and outputs reference | Coming soon | | Environment Variables | Environment variables for training and deployment | Coming soon | diff --git a/docs/reference/copilot-artifacts.md b/docs/reference/copilot-artifacts.md new file mode 100644 index 000000000..fbd429d22 --- /dev/null +++ b/docs/reference/copilot-artifacts.md @@ -0,0 +1,177 @@ +--- +title: Copilot Artifacts +description: >- + Inventory and reference for GitHub Copilot agents, instructions, prompts, + and skills configured in this repository. +author: Microsoft Robotics-AI Team +ms.date: 2026-03-11 +ms.topic: reference +keywords: + - copilot + - agents + - instructions + - prompts + - skills + - extensibility +--- + +GitHub Copilot extensibility artifacts provide AI-assisted workflows +for dataset analysis, training job management, and coding standards +enforcement. These artifacts are configured in `.github/` and activate +automatically in VS Code. + +## 📋 Artifact Inventory + +| Type | Name | Description | Path | +|-------------|------------------------------|------------------------------------------------------|------------------------------------------------------------------| +| Agent | Dataviewer Developer | Interactive dataset analysis and tool development | `.github/agents/dataviewer-developer.agent.md` | +| Agent | OSMO Training Manager | LeRobot training lifecycle on OSMO with Azure ML | `.github/agents/osmo-training-manager.agent.md` | +| Instruction | Commit Messages | Conventional Commits format for all commit messages | `.github/instructions/commit-message.instructions.md` | +| Instruction | Dataviewer | Coding standards for dataviewer development | `.github/instructions/dataviewer.instructions.md` | +| Instruction | Docs Style and Conventions | Writing standards for all markdown files | `.github/instructions/docs-style-and-conventions.instructions.md`| +| Instruction | Shell Scripts | Implementation standards for bash scripts | `.github/instructions/shell-scripts.instructions.md` | +| Prompt | `/chatlog` | Create and maintain conversation logs | `.github/prompts/chatlog.prompt.md` | +| Prompt | `/check-training-status` | Monitor OSMO training job progress | `.github/prompts/check-training-status.prompt.md` | +| Prompt | `/start-dataviewer` | Launch Dataset Analysis Tool | `.github/prompts/start-dataviewer.prompt.md` | +| Prompt | `/submit-lerobot-training` | Submit LeRobot training job to OSMO | `.github/prompts/submit-lerobot-training.prompt.md` | +| Skill | dataviewer | Dataset browsing, annotation, and export | `.github/skills/dataviewer/SKILL.md` | +| Skill | osmo-lerobot-training | Training submission, monitoring, and analysis | `.github/skills/osmo-lerobot-training/SKILL.md` | + +## 🔗 Quick Reference + +| Want to... | Use this artifact | +|-------------------------------------------|----------------------------------------------------------| +| Launch the Dataset Analysis Tool | `/start-dataviewer` prompt → Dataviewer Developer | +| Browse and annotate training episodes | Dataviewer Developer agent | +| Submit a LeRobot training job | `/submit-lerobot-training` prompt → OSMO Training Manager| +| Check training job status | `/check-training-status` prompt → OSMO Training Manager | +| Save a conversation log | `/chatlog` prompt | +| Enforce commit message standards | `commit-message` instruction (auto-applied) | +| Enforce coding standards in dataviewer | `dataviewer` instruction (auto-applied) | +| Enforce markdown writing standards | `docs-style-and-conventions` instruction (auto-applied) | +| Enforce shell script standards | `shell-scripts` instruction (auto-applied) | + +## 🤖 Agents + +### Dataviewer Developer + +Interactive agent for launching, browsing, annotating, and improving the +Dataset Analysis Tool. + +| Property | Value | +|----------|-----------------------------------------------------| +| Handoffs | Start Dataviewer, Browse Dataset, Annotate Episodes | +| Tools | All (no restrictions) | +| Skill | `dataviewer` | +| Prompts | `/start-dataviewer` | + +Four-phase workflow: Launch/Configure → Interactive Browsing (Playwright) → +Episode Annotation (API+UI) → Feature Development (React+FastAPI). + +### OSMO Training Manager + +Multi-turn agent for managing LeRobot imitation learning training lifecycle +on OSMO with Azure ML integration. + +| Property | Value | +|----------|---------------------------------------------------------| +| Handoffs | Submit Training Job, Check Training Status, Run Inference Evaluation | +| Tools | 11 explicit (run_in_terminal, memory, runSubagent, ...) | +| Skill | `osmo-lerobot-training` | +| Prompts | `/submit-lerobot-training`, `/check-training-status` | + +Five-phase workflow: Submit → Monitor → Analyze → Summarize → Inference +Evaluation. Handles VM eviction recovery, CUDA errors, and KeyError +failures. + +## 📝 Instructions + +Instructions activate automatically when files matching their `applyTo` +pattern appear in the chat context. + +| Name | Applies To | Purpose | +|----------------------------|---------------------|---------------------------------------------------------| +| Commit Messages | `**` | Conventional Commits format, scopes, line-length limits | +| Dataviewer | `src/dataviewer/**` | SOLID principles, test-first, validation commands | +| Docs Style and Conventions | `**/*.md` | Document hierarchy, tables, voice/tone, frontmatter | +| Shell Scripts | `**/*.sh` | Script template, library functions, deployment patterns | + +## ⚡ Prompts + +Prompts are slash commands invoked via `/` in the chat input. Each prompt +targets a specific agent. + +| Command | Agent Target | Required Inputs | +|----------------------------|-----------------------|-----------------------| +| `/chatlog` | Generic | None | +| `/check-training-status` | OSMO Training Manager | workflowId (optional) | +| `/start-dataviewer` | Dataviewer Developer | datasetPath | +| `/submit-lerobot-training` | OSMO Training Manager | dataset (required) | + +## 🛠️ Skills + +Skills provide multi-file capabilities with progressive 3-level loading: +discovery (frontmatter only) → instructions (SKILL.md body) → resources +(bundled reference files). + +### dataviewer + +| Property | Value | +|-----------|--------------------------------------------------------------------------| +| Directory | `.github/skills/dataviewer/` | +| Resources | `references/PLAYWRIGHT.md` (selectors, interaction recipes, API endpoints) | +| Used by | Dataviewer Developer agent | + +### osmo-lerobot-training + +| Property | Value | +|-----------|----------------------------------------------------------------------------------------------------------------| +| Directory | `.github/skills/osmo-lerobot-training/` | +| Resources | `references/DEFAULTS.md` (env, datasets, GPU profiles), `references/REFERENCE.md` (CLI, inference, AzureML navigation) | +| Used by | OSMO Training Manager agent | + +## 🔄 Workflow Chains + +Agents compose prompts and skills into end-to-end workflows: + +```text +OSMO Training Manager (agent) + ├── /submit-lerobot-training (prompt) + ├── /check-training-status (prompt) + └── osmo-lerobot-training (skill) + ├── references/DEFAULTS.md + └── references/REFERENCE.md + +Dataviewer Developer (agent) + ├── /start-dataviewer (prompt) + └── dataviewer (skill) + └── references/PLAYWRIGHT.md + +Standalone: + └── /chatlog (prompt, generic) +``` + +## ➕ Adding New Artifacts + +VS Code provides generator commands for scaffolding new artifacts: + +- `/create-agent` — Create a new custom agent +- `/create-instruction` — Create a new instruction file +- `/create-prompt` — Create a new prompt file +- `/create-skill` — Create a new agent skill + +Place new artifacts in the corresponding `.github/` subdirectory and update +this inventory page. + +## Related Documentation + +For broader project context, see these companion guides: + +- [Contributing Guide](../contributing/README.md) — Development workflow and coding standards +- [Architecture](../contributing/architecture.md) — System architecture and agent skills design +- [Prerequisites](../contributing/prerequisites.md) — Required tools and VS Code settings + +--- + +*Crafted with precision by Copilot following brilliant human instruction, +then carefully refined by our team of discerning human reviewers.*