Pin the threat-detection model on every agentic workflow

[Jakub Jareš (nohwnd)]

The detection job keeps recording parse_error even when the job itself succeeds. The default
detection model alias sometimes wraps its result line in Markdown emphasis, and gh-aw's parser
slices the JSON at a fixed offset from the start of the line instead of from the index of the marker
it just located. The two leading asterisks move the cut two characters into RESULT, so it tries to
parse T:{"prompt"…:

📄 Lines containing THREAT_DETECTION_RESULT (1 of 194):
   [155] **THREAT_DETECTION_RESULT:{"prompt_injection":false,"secret_leak":false,…**
##[error]❌ Failed to parse detection result: Unexpected token 'T', "T:{"prompt"... is not valid JSON

The parser is parse_threat_detection_results.cjs inside the gh-aw actions bundle that every run
downloads, so the bug cannot be fixed in this repository. Pinning the detector to a model that does
not add the emphasis is the mitigation, and
microsoft/testfx#10684 already proved it works for
the expert-review workflows.

This applies the same pin to the 29 workflows that were still on the default alias. Where several
workflows share an import the pin goes in the shared file, so shared/address-review-shared.md,
shared/msbuild-review-shared.md, shared/parallel-safety-audit-shared.md and
shared/test-reviewer-shared.md cover seven locks between them. malicious-code-scan.md already
declared threat-detection, so it gets engine: inside its existing block. The expert-review trio
and Test Improver were already pinned and are unchanged.

The workflows README gains a section for this failure, including how to tell it apart from the
Install GitHub Copilot CLI cause that produces the same parse_error: if the log shows
Lines containing THREAT_DETECTION_RESULT (1 of N) the marker exists and this is the formatting
cause, and if it shows No THREAT_DETECTION_RESULT found it is the installer cause covered by
microsoft/testfx#10427.

Observed on:

• run 32787262866 (Test Improver, 2026-08-24)
• run 32711599038 (Expert Code Review, 2026-08-24)
• run 31328916257 (Test Reviewer on PR, 2026-08-09)

One thing to weigh: this moves every workflow's detection job onto gpt-5-mini instead of the
default alias, so there is some extra cost. The PR-triggered review workflows run most often and
already made that move in
microsoft/testfx#10684, and the rest are mostly
daily scheduled runs, so the marginal cost is small. If you would rather scope it down, the runs
above only justify Test Improver, Test Reviewer, Parallel-safety audit, Sub-Issue Closer and Build
Failure Analysis.

The local compile also rewrote the pinned github/gh-aw-actions/setup SHA in agentic_commands.yml
back to a mutable tag, which is
microsoft/testfx#10258 again, on a gh aw whose
version matches the lock headers. That file is restored to its committed state and is not in this
diff; the README note about compiling on the pinned toolchain now says a matching compiler_version
is not enough to prevent it.

Verified: compiled with gh aw compile --strict on gh-aw v0.86.2, matching the compiler_version
recorded in every existing lock. All 33 locks now report "detection_agent_model":"gpt-5-mini", no
COPILOT_MODEL: detection remains, python .github/scripts/check_action_pins.py passes over 2345
references, and no uses: line changed anywhere in the diff.

Refs microsoft/testfx#10711

🤖

[github-actions]

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review Summary

This PR pins safe-outputs.threat-detection.engine.model: gpt-5-mini across 29 agentic workflows to work around the gh-aw parser bug where the default detection alias wraps its THREAT_DETECTION_RESULT JSON in Markdown emphasis, causing parse_error. The README is updated with troubleshooting guidance.

Verdict Table

#	Dimension	Severity	Finding
1	Algorithmic Correctness	MAJOR	N/A — no logic changes
2	Public API Surface	CRITICAL	N/A — no API changes
3	Thread Safety	MAJOR	N/A
4	Resource Management	MAJOR	N/A
5	Error Handling	MAJOR	N/A
6	Backward Compatibility	CRITICAL	N/A
7	Performance	MAJOR	N/A
8	Security	CRITICAL	N/A
9	Cross-TFM	MAJOR	N/A
10	IPC/Wire Format	CRITICAL	N/A
11	Localization	MINOR	N/A
12	Test Quality	MAJOR	N/A
13	Naming & Conventions	MINOR	N/A
14	Documentation	MINOR	✅ Clean — README section is well-structured
15	Null Safety	MAJOR	N/A
16	Disposal & Lifetime	MAJOR	N/A
17	Configuration	MINOR	✅ Clean — consistent pattern across all workflows
18	Build Integration	MAJOR	✅ Clean — all locks recompiled with --strict, compiler_version matches
19	Logging & Telemetry	MINOR	N/A
20	Code Duplication	MINOR	✅ Shared files used where possible (4 shared imports cover 7 locks)
21	Scope Discipline	MINOR	✅ Single concern — model pin only
22	TODO Policy	MINOR	N/A

Assessment

No blocking issues found. The change is mechanical and consistent: every .md source gets the same threat-detection engine block (with a comment referencing #10711), every .lock.yml is regenerated with matching detection_agent_model metadata, no uses: lines changed, and the malicious-code-scan.md (which already had a threat-detection: block with continue-on-error: true) correctly gets the engine: sub-block added alongside the existing property. The README troubleshooting section clearly differentiates this failure mode from the installer-related parse_error documented in #10427.

[Copilot]

Pull request overview

Pins threat detection to gpt-5-mini across agentic workflows, mitigating malformed Markdown-wrapped detection results.

Changes:

• Configures the model directly or through shared workflow imports.
• Regenerates 29 lock files in strict mode.
• Documents diagnosis, mitigation, and action-pin validation.

Reviewed changes

Copilot reviewed 56 out of 56 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
.github/workflows/add-tests.md	Pins detection model.
.github/workflows/add-tests.lock.yml	Regenerates compiled workflow.
.github/workflows/address-review.agent.lock.yml	Applies shared model pin.
.github/workflows/adhoc-qa.md	Pins detection model.
.github/workflows/adhoc-qa.lock.yml	Regenerates compiled workflow.
.github/workflows/autofix.agent.lock.yml	Applies shared model pin.
.github/workflows/build-failure-analysis-command.md	Pins detection model.
.github/workflows/build-failure-analysis-command.lock.yml	Regenerates compiled workflow.
.github/workflows/build-failure-analysis.md	Pins detection model.
.github/workflows/build-failure-analysis.lock.yml	Regenerates compiled workflow.
.github/workflows/code-simplifier.md	Pins detection model.
.github/workflows/code-simplifier.lock.yml	Regenerates compiled workflow.
.github/workflows/daily-file-diet.md	Pins detection model.
.github/workflows/daily-file-diet.lock.yml	Regenerates compiled workflow.
.github/workflows/dependabot-issue-bundler.md	Pins detection model.
.github/workflows/dependabot-issue-bundler.lock.yml	Regenerates compiled workflow.
.github/workflows/dependabot-pr-bundler.md	Pins detection model.
.github/workflows/dependabot-pr-bundler.lock.yml	Regenerates compiled workflow.
.github/workflows/duplicate-code-detector.md	Pins detection model.
.github/workflows/duplicate-code-detector.lock.yml	Regenerates compiled workflow.
.github/workflows/efficiency-improver.md	Pins detection model.
.github/workflows/efficiency-improver.lock.yml	Regenerates compiled workflow.
.github/workflows/glossary-maintainer.md	Pins detection model.
.github/workflows/glossary-maintainer.lock.yml	Regenerates compiled workflow.
.github/workflows/link-checker.md	Pins detection model.
.github/workflows/link-checker.lock.yml	Regenerates compiled workflow.
.github/workflows/malicious-code-scan.md	Extends existing detection settings.
.github/workflows/malicious-code-scan.lock.yml	Regenerates compiled workflow.
.github/workflows/markdown-linter.md	Pins detection model.
.github/workflows/markdown-linter.lock.yml	Regenerates compiled workflow.
.github/workflows/msbuild-quality-review.lock.yml	Applies shared model pin.
.github/workflows/parallel-safety-audit-command.lock.yml	Applies shared model pin.
.github/workflows/parallel-safety-audit.lock.yml	Applies shared model pin.
.github/workflows/perf-improver.md	Pins detection model.
.github/workflows/perf-improver.lock.yml	Regenerates compiled workflow.
.github/workflows/pr-fix.md	Pins detection model.
.github/workflows/pr-fix.lock.yml	Regenerates compiled workflow.
.github/workflows/q.md	Pins detection model.
.github/workflows/q.lock.yml	Regenerates compiled workflow.
.github/workflows/README.md	Documents parse errors and pin safety.
.github/workflows/repository-quality-improver.md	Pins detection model.
.github/workflows/repository-quality-improver.lock.yml	Regenerates compiled workflow.
.github/workflows/resource-lock-refactoring.md	Pins detection model.
.github/workflows/resource-lock-refactoring.lock.yml	Regenerates compiled workflow.
.github/workflows/shared/address-review-shared.md	Pins two consuming workflows.
.github/workflows/shared/msbuild-review-shared.md	Pins its consuming workflow.
.github/workflows/shared/parallel-safety-audit-shared.md	Pins two consuming workflows.
.github/workflows/shared/test-reviewer-shared.md	Pins two consuming workflows.
.github/workflows/sub-issue-closer.md	Pins detection model.
.github/workflows/sub-issue-closer.lock.yml	Regenerates compiled workflow.
.github/workflows/test-reviewer-on-pr.agent.lock.yml	Applies shared model pin.
.github/workflows/test-reviewer.agent.lock.yml	Applies shared model pin.
.github/workflows/unskip-closed-tests.md	Pins detection model.
.github/workflows/unskip-closed-tests.lock.yml	Regenerates compiled workflow.
.github/workflows/weekly-issue-activity.md	Pins detection model.
.github/workflows/weekly-issue-activity.lock.yml	Regenerates compiled workflow.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.