Skip to content

feat(skills): add qa-testing orchestration skill#8015

Merged
Jasonnnz merged 1 commit into
feature/qa-video-automationfrom
add-qa-testing-skill
Feb 24, 2026
Merged

feat(skills): add qa-testing orchestration skill#8015
Jasonnnz merged 1 commit into
feature/qa-video-automationfrom
add-qa-testing-skill

Conversation

@Jasonnnz
Copy link
Copy Markdown
Contributor

@Jasonnnz Jasonnnz commented Feb 24, 2026

Summary

  • Add a bundled qa-testing skill that composes screen-recording and media-processing via the includes mechanism
  • This is an awareness-only skill (SKILL.md only, no TOOLS.json) that teaches the model about QA workflows
  • Covers strict focus management, recording lifecycle, QA latch behavior, post-session video analysis, and error handling

Test plan

  • Verify SKILL.md has valid YAML frontmatter with correct includes references
  • Verify no TOOLS.json is present (awareness-only skill)
  • Verify skill is picked up by the skill loader at runtime

🤖 Generated with Claude Code


Open with Devin

Add a bundled qa-testing skill that composes screen-recording and
media-processing via includes. This is an awareness-only skill (no
TOOLS.json) that teaches the model about QA workflows, strict focus
management, recording lifecycle, and post-session video analysis.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@Jasonnnz Jasonnnz merged commit 3d1527e into feature/qa-video-automation Feb 24, 2026
@Jasonnnz Jasonnnz deleted the add-qa-testing-skill branch February 24, 2026 17:44
Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

View 3 additional findings in Devin Review.

Open in Devin Review

description: "Run QA and testing workflows with automatic screen recording, strict focus management, and post-session video analysis."
user-invocable: false
disable-model-invocation: false
includes: ["screen-recording", "media-processing"]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Missing screen-recording skill causes qa-testing to fail validation at runtime

The qa-testing skill declares includes: ["screen-recording", "media-processing"] in its frontmatter, but no screen-recording skill exists anywhere in the codebase. The media-processing skill exists at assistant/src/config/bundled-skills/media-processing/, but there is no corresponding screen-recording directory.

Root Cause and Impact

When the skill_load tool attempts to load qa-testing at runtime, it calls validateIncludes() (assistant/src/tools/skills/load.ts:56), which performs a DFS traversal of the include graph. Since screen-recording is not in the skill catalog, the validation fails with a missing error (assistant/src/skills/include-graph.ts:92-100), and the tool returns an error:

Error: skill "qa-testing" includes "screen-recording" which was not found (referenced by "qa-testing" via path: qa-testing)

This means the qa-testing skill cannot be loaded at all — the fail-closed validation at load.ts:57-74 prevents the skill body from ever being returned to the model. The PR's test plan item "Verify skill is picked up by the skill loader at runtime" is unchecked, which aligns with this being untested.

Prompt for agents
The includes field at assistant/src/config/bundled-skills/qa-testing/SKILL.md:6 references "screen-recording" which does not exist as a bundled skill. Either:

1. Create a new bundled skill at assistant/src/config/bundled-skills/screen-recording/SKILL.md with appropriate frontmatter and content describing screen recording capabilities, OR
2. Remove "screen-recording" from the includes array on line 6, changing it to: includes: ["media-processing"]

Without one of these fixes, the qa-testing skill will fail to load at runtime because validateIncludes() in assistant/src/skills/include-graph.ts performs fail-closed validation that rejects skills with missing includes.
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f67119716a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

description: "Run QA and testing workflows with automatic screen recording, strict focus management, and post-session video analysis."
user-invocable: false
disable-model-invocation: false
includes: ["screen-recording", "media-processing"]
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Replace missing included skill ID

qa-testing currently includes screen-recording, but that skill ID is not present in the bundled catalog, so loading this skill fails closed: skill_load validates include graphs and returns an error when any included child is missing (assistant/src/tools/skills/load.ts, validateIncludes path). In a default install (without a separately installed managed screen-recording skill), skill_load for qa-testing will always error and the new QA workflow cannot be activated.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant