Consolidate CI workflows for full integration testing across all platforms by Copilot · Pull Request #58 · demaconsulting/ReqStream

Copilot · 2026-01-10T21:06:13Z

Full Integration Testing for Builds and Releases

Implementation Complete ✅

All requirements have been successfully implemented:

Summary

This PR provides comprehensive integration testing by:

Running unit tests on both Windows and Linux (2 platforms)
Running integration tests on Windows/Linux × .NET 8.x/9.x/10.x (6 combinations)
Collecting all test results (8 TRX files total) for requirements evidence
Using filename filters to link platform-specific tests to requirements
Ensuring all builds (push and release) use the same comprehensive testing
Documentation generation uses the version being built/tested

Changes in Commits

Commit b71a8c7 - Initial consolidation:

Moved quality checks, build, integration tests, and documentation into single build.yaml
Simplified build_on_push.yaml and release.yaml to call consolidated workflow

Commit 0b5a268 - Simplified artifact downloads and filename filters:

Replaced 8 separate download-artifact steps with single step using pattern matching
Added filename filters to platform requirements (windows-latest@, ubuntu-latest@)
Added filename filters to .NET version requirements (dotnet8.x@, dotnet9.x@, dotnet10.x@)

Commit e90dcba - Use built ReqStream version:

build-docs job now downloads ReqStream package from build artifacts
Installs ReqStream globally from downloaded package
Removed ReqStream from .config/dotnet-tools.json

Commit e3b5676 - Cleanup:

Removed obsolete build_docs.yaml workflow file

Current commit - Fix command invocation:

Changed dotnet reqstream to reqstream since it's installed globally
Maintains dotnet prefix for manifest-installed tools (pandoc, weasyprint)

Security Summary

No security vulnerabilities were found by CodeQL analysis.

Original prompt

This section details on the original issue you should resolve

<issue_title>[Feature]: Full integration testing for builds and releases</issue_title>
<issue_description>### Problem Statement

The CI pipelines should be restructured so that all build types (build-on-push and release) run all the matrix tests and use the evidence to prove requirements.

Proposed Solution

The build.yaml should probably drop taking an OS input. Instead it should:

Contain the code-quality job (moved from the build-on-push.yaml)

Matrix execute the build on Windows and Linux just so the unit-testing is executed on both operating systems. As well as publishing the nuget package it should also publish the TRX test results with file-names decorated with the OS.

Matrix execute the integration tests on windows and linux with dotnet 8.x, 9.x, and 10.x - with the self-validation test results written to TRX files using names decorated with the OS and dotnet version.

Execute the documentation build job (moved from build_docs) and downloading all the TRX results from the other jobs to be used as requirements testing evidence.

Alternatives Considered

No response

Additional Context

This should also allow for linking those tests to the OS and DotNet requirements.

Priority

Medium - Would improve workflow

Contribution

I would be willing to submit a PR for this feature

Code of Conduct

I agree to follow this project's Code of Conduct</issue_description>

Comments on the Issue (you are @copilot in this section)

Custom agent used: Software Quality Enforcer
Expert agent for code quality, testing standards, code reviews, security analysis, and ensuring adherence to coding conventions

Fixes [Feature]: Full integration testing for builds and releases #57

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

- Consolidated build.yaml to be self-contained without OS input - Added quality checks job to build.yaml (moved from build_on_push.yaml) - Added matrix build for Windows and Linux with OS-decorated TRX files - Added matrix integration tests for Windows/Linux × .NET 8/9/10 - Added build-docs job to build.yaml with all TRX downloads - Updated build_on_push.yaml to call new self-contained build.yaml - Updated release.yaml to call new self-contained build.yaml - Added test mappings to requirements.yaml for OS and .NET requirements Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

.github/workflows/build.yaml

requirements.yaml

…ters - Consolidated multiple download-artifact steps into single step with pattern - Added filename filters to platform requirements for Windows/Linux - Added filename filters to .NET version requirements for 8.x/9.x/10.x - This ensures each requirement only uses test results from relevant platforms Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

- Added package download step to build-docs job - Install ReqStream from build artifacts instead of dotnet-tools.json - Removed ReqStream from .config/dotnet-tools.json - Ensures documentation uses the version being built/tested Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

Malcolmnixon · 2026-01-11T02:28:04Z

@copilot I don't believe the build_docs.yaml file is used any more and can be deleted.

- build_docs.yaml is no longer used after consolidation into build.yaml - Documentation build functionality is now part of the build.yaml workflow Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

Copilot · 2026-01-11T02:30:44Z

@copilot I don't believe the build_docs.yaml file is used any more and can be deleted.

Deleted build_docs.yaml as it's no longer used after the documentation build was consolidated into build.yaml. (e3b5676)

.github/workflows/build.yaml

- ReqStream is installed globally, so it's available as 'reqstream' command - Removed 'dotnet' prefix from reqstream invocation - Other tools (pandoc, weasyprint) remain with dotnet prefix as they're from manifest Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

Apply all missing changes from the TemplateDotNetTool template to bring ReqStream into consistency with the latest template patterns. Changes applied from PRs #58-#74: PR #58 (ReviewMark integration): - Add .reviewmark.yaml with ReqStream-specific review sets - Add pip-requirements.txt (yamllint==1.38.0) - Update lint.sh/lint.bat to use Python venv + pip pattern - Update package.json with pinned cspell and markdownlint-cli2 devDependencies - Add demaconsulting.reviewmark to .config/dotnet-tools.json - Add reviewmark entry to .versionmark.yaml PR #59 (Add Code Review Agent to AGENTS.md invocation rules): - Add code-review agent to AGENTS.md available agents list - Add new .github/agents/code-review.agent.md PR #61 (Linting modernization and agent file standardization): - Update .markdownlint-cli2.yaml with PURPOSE/DO NOT MODIFY header - Update .yamllint.yaml with PURPOSE/DO NOT MODIFY header - Remove check-keys from yamllint truthy rule (aligns with template) PR #63 (enforce cspell word list policy): - Update .cspell.yaml header comment (NEVER add misspelled word) - Add missing technical terms to cspell word list (reviewmark, ReviewMark, versionmark, VersionMark, buildmark, BuildMark, gitattributes, etc.) PR #64 (Align lint YAML configs): - Apply standardized headers to markdownlint and yamllint configs PR #65-#67, #70-#73 (Agent synchronization): - Update code-quality.agent.md to template version (comprehensive content) - Update agent frontmatter descriptions to match shorter template pattern - Update repo-consistency agent and requirements, software-developer, technical-writer, test-developer agents PR #71 (.gitattributes LF line endings): - .gitattributes already had correct content PR #72 (Simplify repo-consistency agent): - repo-consistency.agent.md already aligned PR #54 (Various improvements): - Update .versionmark.yaml regex patterns to support pre-release versions - Update .gitignore to match template structure CONTRIBUTING.md: - Rewrite to match template standards with ReqStream-specific translation - Add dotnet tool restore step, XML documentation section - Add comprehensive testing guidelines with MSTest v4 assertions - Add requirements management section

Exhaustive review and sync of all merged TemplateDotNetTool PRs: ## Folder Renames - docs/quality/ → docs/code_quality/ (matches template naming) - docs/requirements/ → docs/requirements_doc/ (matches template naming) - docs/tracematrix/ → docs/requirements_report/ (matches template naming) ## Folders Removed - docs/justifications/ (content now merged into docs/requirements_doc/) - docs/design/ (replaced by per-unit YAML requirements in docs/reqstream/) ## New Folders Created - docs/requirements_report/ (replaces tracematrix, uses trace_matrix.md) - docs/code_review_plan/ (from PR #58: ReviewMark integration) - docs/code_review_report/ (from PR #58: ReviewMark integration) ## definition.yaml Updates - docs/code_quality/definition.yaml: updated resource-path and input-file paths - docs/requirements_doc/definition.yaml: updated paths, added justifications.md input ## New Files - docs/reqstream/ots-reviewmark.yaml (from PR #58: ReviewMark OTS requirements) ## Workflow Updates (build.yaml) - Added reviewmark to versionmark capture tool list - Added ReviewMark self-validation step - Updated reqstream report paths to requirements_doc/ and requirements_report/ - Updated sarifmark/sonarmark report paths to code_quality/ - Added ReviewMark plan/report generation step - Replaced justifications/tracematrix pandoc steps with requirements_report, code_review_plan, and code_review_report steps - Replaced old Weasyprint PDF steps with new folder names - Removed Requirements Justifications PDF (merged into Requirements PDF) - Added Review Plan and Review Report PDF generation ## Other File Updates - .reviewmark.yaml: removed docs/design/ pattern, added ots-reviewmark.yaml to OTS-Dependencies review set, removed docs/design/*.md from unit reviews - .gitignore: updated generated file paths to new folder names - .github/agents/code-quality.agent.md: updated reqstream command paths - requirements.yaml: added ots-reviewmark.yaml include Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

#131) * Initial plan * Apply TemplateDotNetTool improvements (PRs #61-#74) - Add .gitattributes to enforce LF line endings (PR #71) - Rename agent files to .agent.md naming convention (PR #61/#65) - Add tools: and user-invocable: true to all agent frontmatter (PR #73) - Use kebab-case name: fields in agent frontmatter - Replace .cspell.json with .cspell.yaml (YAML format with policy comments) (PR #61/#63) - Replace .markdownlint-cli2.jsonc with .markdownlint-cli2.yaml (PR #61) - Update .yamllint.yaml ignore patterns with thirdparty dirs (PR #64) - Update lint ignore paths with better glob patterns (PR #64) - Add spell check word list policy to AGENTS.md (PR #63) - Add Spell Checking section to CONTRIBUTING.md (PR #63) - Update Defer To sections to Subagent Delegation pattern (PR #65) - Fix linting config file name references in agent files * Fix review comments: update agent file extension reference and line lengths - Update .agent.md extension references in AGENTS.md and technical-writer.agent.md - Remove 'Propagatable' from .cspell.yaml (not used in ReqStream) - Fix MD013 line length violations in AGENTS.md and technical-writer.agent.md * Remove unused spdx from cspell word list * Apply template consistency updates from TemplateDotNetTool PRs #54-#74 Apply all missing changes from the TemplateDotNetTool template to bring ReqStream into consistency with the latest template patterns. Changes applied from PRs #58-#74: PR #58 (ReviewMark integration): - Add .reviewmark.yaml with ReqStream-specific review sets - Add pip-requirements.txt (yamllint==1.38.0) - Update lint.sh/lint.bat to use Python venv + pip pattern - Update package.json with pinned cspell and markdownlint-cli2 devDependencies - Add demaconsulting.reviewmark to .config/dotnet-tools.json - Add reviewmark entry to .versionmark.yaml PR #59 (Add Code Review Agent to AGENTS.md invocation rules): - Add code-review agent to AGENTS.md available agents list - Add new .github/agents/code-review.agent.md PR #61 (Linting modernization and agent file standardization): - Update .markdownlint-cli2.yaml with PURPOSE/DO NOT MODIFY header - Update .yamllint.yaml with PURPOSE/DO NOT MODIFY header - Remove check-keys from yamllint truthy rule (aligns with template) PR #63 (enforce cspell word list policy): - Update .cspell.yaml header comment (NEVER add misspelled word) - Add missing technical terms to cspell word list (reviewmark, ReviewMark, versionmark, VersionMark, buildmark, BuildMark, gitattributes, etc.) PR #64 (Align lint YAML configs): - Apply standardized headers to markdownlint and yamllint configs PR #65-#67, #70-#73 (Agent synchronization): - Update code-quality.agent.md to template version (comprehensive content) - Update agent frontmatter descriptions to match shorter template pattern - Update repo-consistency agent and requirements, software-developer, technical-writer, test-developer agents PR #71 (.gitattributes LF line endings): - .gitattributes already had correct content PR #72 (Simplify repo-consistency agent): - repo-consistency.agent.md already aligned PR #54 (Various improvements): - Update .versionmark.yaml regex patterns to support pre-release versions - Update .gitignore to match template structure CONTRIBUTING.md: - Rewrite to match template standards with ReqStream-specific translation - Add dotnet tool restore step, XML documentation section - Add comprehensive testing guidelines with MSTest v4 assertions - Add requirements management section * Split requirements.yaml into per-unit files and update .reviewmark.yaml review sets Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com> Agent-Logs-Url: https://github.com/demaconsulting/ReqStream/sessions/e078de59-4efb-45e9-98e2-182adb75849f * Update CONTRIBUTING.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Add design documentation, move source filter comment, update reviewmark review sets Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com> Agent-Logs-Url: https://github.com/demaconsulting/ReqStream/sessions/2b194e1a-0fcd-40ab-8fbf-6062639fbb37 * Add ValidationTests.cs and update review set and requirements links Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com> Agent-Logs-Url: https://github.com/demaconsulting/ReqStream/sessions/c78e09f2-c209-4b61-9988-b016c910e62e * Fix issues identified by code-review agent review of all review sets Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com> Agent-Logs-Url: https://github.com/demaconsulting/ReqStream/sessions/c0d6ebe8-9cf3-41ce-b852-d85ca00b62f5 * Update docs/design/program.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update docs/design/validation.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update docs/design/requirements.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Remove brittle hardcoded test count assertions from ValidationTests Agent-Logs-Url: https://github.com/demaconsulting/ReqStream/sessions/64e3fd0a-6a0f-4311-ba6f-df04cc3cf436 Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com> * Fix cspell failure: rephrase 'venv' to 'virtual environment support' in CONTRIBUTING.md Agent-Logs-Url: https://github.com/demaconsulting/ReqStream/sessions/acc9755d-ede8-4ec7-a4b8-c4a4af08956c Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com> * Add venv to cspell word list as approved technical term; restore natural wording in CONTRIBUTING.md Agent-Logs-Url: https://github.com/demaconsulting/ReqStream/sessions/cacc8054-fc75-4626-8e3c-4ed488cdea4e Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com> * Apply TemplateDotNetTool PR#75: remove buildnotes move step, use direct artifact path in release.yaml Agent-Logs-Url: https://github.com/demaconsulting/ReqStream/sessions/6720fcce-c0ec-4055-9d6d-a685202142b5 Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com> * Complete PR#75 buildnotes→build_notes rename; bump buildmark to 0.4.1 (PR#70) - Rename docs/buildnotes/ directory to docs/build_notes/ - Update docs/build_notes/definition.yaml path references - Update build.yaml: rename all buildnotes.md → build_notes.md and buildnotes/ → build_notes/ paths - Update release.yaml: fix bodyFile from artifacts/docs/buildnotes.md → artifacts/docs/build_notes.md - Remove 'buildnotes' from .cspell.yaml word list (no longer a valid identifier) - Bump DemaConsulting.BuildMark from 0.4.0 to 0.4.1 in .config/dotnet-tools.json The previous PR#75 commit only removed the move step in release.yaml but did not complete the rename from buildnotes to build_notes throughout the codebase. Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com> * Align docs/ folder structure with TemplateDotNetTool template Exhaustive review and sync of all merged TemplateDotNetTool PRs: ## Folder Renames - docs/quality/ → docs/code_quality/ (matches template naming) - docs/requirements/ → docs/requirements_doc/ (matches template naming) - docs/tracematrix/ → docs/requirements_report/ (matches template naming) ## Folders Removed - docs/justifications/ (content now merged into docs/requirements_doc/) - docs/design/ (replaced by per-unit YAML requirements in docs/reqstream/) ## New Folders Created - docs/requirements_report/ (replaces tracematrix, uses trace_matrix.md) - docs/code_review_plan/ (from PR #58: ReviewMark integration) - docs/code_review_report/ (from PR #58: ReviewMark integration) ## definition.yaml Updates - docs/code_quality/definition.yaml: updated resource-path and input-file paths - docs/requirements_doc/definition.yaml: updated paths, added justifications.md input ## New Files - docs/reqstream/ots-reviewmark.yaml (from PR #58: ReviewMark OTS requirements) ## Workflow Updates (build.yaml) - Added reviewmark to versionmark capture tool list - Added ReviewMark self-validation step - Updated reqstream report paths to requirements_doc/ and requirements_report/ - Updated sarifmark/sonarmark report paths to code_quality/ - Added ReviewMark plan/report generation step - Replaced justifications/tracematrix pandoc steps with requirements_report, code_review_plan, and code_review_report steps - Replaced old Weasyprint PDF steps with new folder names - Removed Requirements Justifications PDF (merged into Requirements PDF) - Added Review Plan and Review Report PDF generation ## Other File Updates - .reviewmark.yaml: removed docs/design/ pattern, added ots-reviewmark.yaml to OTS-Dependencies review set, removed docs/design/*.md from unit reviews - .gitignore: updated generated file paths to new folder names - .github/agents/code-quality.agent.md: updated reqstream command paths - requirements.yaml: added ots-reviewmark.yaml include Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com> Co-authored-by: Malcolm Nixon <Malcolm.nixon@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Initial plan

a4561b1

Copilot AI assigned Copilot and Malcolmnixon Jan 10, 2026

Copilot started work on behalf of Malcolmnixon January 10, 2026 21:06 View session

Copilot AI changed the title ~~[WIP] Add full integration testing for builds and releases~~ Consolidate CI workflows for full integration testing across all platforms Jan 10, 2026

Copilot AI requested a review from Malcolmnixon January 10, 2026 21:14

Copilot finished work on behalf of Malcolmnixon January 10, 2026 21:14

Malcolmnixon requested changes Jan 11, 2026

View reviewed changes

Copilot started work on behalf of Malcolmnixon January 11, 2026 02:20 View session

Copilot AI and others added 2 commits January 11, 2026 02:22

Copilot AI requested a review from Malcolmnixon January 11, 2026 02:27

Copilot finished work on behalf of Malcolmnixon January 11, 2026 02:27

Copilot started work on behalf of Malcolmnixon January 11, 2026 02:28 View session

Copilot finished work on behalf of Malcolmnixon January 11, 2026 02:32

Malcolmnixon requested changes Jan 11, 2026

View reviewed changes

.github/workflows/build.yaml Outdated Show resolved Hide resolved

Copilot started work on behalf of Malcolmnixon January 11, 2026 02:42 View session

Copilot AI requested a review from Malcolmnixon January 11, 2026 02:44

Copilot finished work on behalf of Malcolmnixon January 11, 2026 02:44

Malcolmnixon approved these changes Jan 11, 2026

View reviewed changes

Malcolmnixon marked this pull request as ready for review January 11, 2026 02:50

Malcolmnixon merged commit b1ab1c3 into main Jan 11, 2026
13 checks passed

Malcolmnixon deleted the copilot/full-integration-testing-pipelines branch January 11, 2026 02:50

Copilot AI mentioned this pull request Mar 26, 2026

Update to latest repo-consistency agent and improvements from template #131

Merged

23 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Consolidate CI workflows for full integration testing across all platforms#58

Consolidate CI workflows for full integration testing across all platforms#58
Malcolmnixon merged 6 commits intomainfrom
copilot/full-integration-testing-pipelines

Copilot AI commented Jan 10, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Malcolmnixon commented Jan 11, 2026

Uh oh!

Copilot AI commented Jan 11, 2026

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Copilot AI commented Jan 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Full Integration Testing for Builds and Releases

Implementation Complete ✅

Summary

Changes in Commits

Security Summary

Proposed Solution

Alternatives Considered

Additional Context

Priority

Contribution

Code of Conduct

Comments on the Issue (you are @copilot in this section)

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Malcolmnixon commented Jan 11, 2026

Uh oh!

Copilot AI commented Jan 11, 2026

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Copilot AI commented Jan 10, 2026 •

edited

Loading