Add IntegrationTest_* and Cli_* tests; update reqstream YAML for test-linkage compliance by Copilot · Pull Request #46 · demaconsulting/ReviewMark

Copilot · 2026-04-05T01:29:57Z

Adds new system-level (IntegrationTest_*) and CLI-subsystem (Cli_*) tests and restructures ReqStream/ReviewMark YAML so requirements link to tests at the correct hierarchy level (system → integration tests, subsystem → subsystem tests, unit → unit tests).

Description

Added 7 new IntegrationTest_* tests to IntegrationTests.cs covering major CLI flows (ReviewPlanGeneration, ReviewReportGeneration, Enforce, IndexScan, WorkingDirectoryOverride, Elaborate, Lint)
Added 15 new Cli_* subsystem tests to CliTests.cs covering key CLI flags and behaviors (ResultsFlag, LogFlag, ErrorOutput, InvalidArgs, ExitCode, DefinitionFlag, PlanFlag, ReportFlag, IndexFlag, EnforceFlag, DirFlag, ElaborateFlag, LintFlag, PlanDepthFlag, ReportDepthFlag)
Added Configuration_LoadConfig_ReportGenerationSucceeds and Configuration_LoadConfig_ElaborationSucceeds tests to ConfigurationTests.cs to correctly cover report generation and elaboration requirements
Added Indexing_SafePathCombine_WithTraversalInputs_Throws test to IndexingTests.cs to validate path traversal rejection at the subsystem level
Updated review-mark.yaml (system): link only to IntegrationTest_* tests; added children: links to subsystem requirements
Updated cli.yaml (subsystem): link PlanDepth requirement to Cli_PlanDepthFlag_SetsHeadingDepth and ReportDepth requirement to Cli_ReportDepthFlag_SetsHeadingDepth; maintained children: links to unit requirements
Updated configuration.yaml (subsystem): linked ReportGeneration requirement to Configuration_LoadConfig_ReportGenerationSucceeds and Elaboration requirement to Configuration_LoadConfig_ElaborationSucceeds
Updated indexing.yaml (subsystem): added Indexing_SafePathCombine_WithTraversalInputs_Throws to SafePathCombine requirement
Updated self-test.yaml (subsystem): link only to SelfTest_* tests; added children: links
Fixed review-mark-configuration.yaml: removed Program_* test that doesn't belong at unit level
Fixed review-index.yaml: removed ReviewMarkConfiguration_* tests from ReviewMark-EvidenceSource-None
Updated .reviewmark.yaml: added Purpose review; updated Architecture/Design/AllRequirements/subsystem/unit review paths to match new standard patterns
Fixed Cli_LogFlag_WritesOutputToFile: switched to block-scoped using so the Context (and its StreamWriter log file handle) is disposed before the file is read or deleted, preventing a "file in use" failure on Windows
Fixed Cli_ErrorOutput_WritesToStderr: rewrote to invoke Program.Main via reflection and assert stderr contains Error: and the unknown argument name
Fixed all Path.GetTempFileName() + Path.ChangeExtension temp-file leaks in CliTests.cs (7 occurrences) and IntegrationTests.cs (4 occurrences) by using Path.GetRandomFileName()/Guid.NewGuid() patterns that never pre-create a file on disk

Type of Change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Documentation update
Code quality improvement

Pre-Submission Checklist

Build and Test

Code builds successfully: dotnet build --configuration Release
All unit tests pass: dotnet test --configuration Release
Self-validation tests pass
Code produces zero warnings

Code Quality

Code formatting is correct: dotnet format --verify-no-changes
New code has appropriate XML documentation comments
Static analyzer warnings have been addressed

Quality Checks

Spell checker passes: cspell "**/*.{md,cs}"
Markdown linter passes: markdownlint "**/*.md"
YAML linter passes: yamllint .

Testing

Added unit tests for new functionality
Updated existing tests if behavior changed
All tests follow the AAA (Arrange, Act, Assert) pattern
Test coverage is maintained or improved

Documentation

Updated README.md (if applicable)
Updated ARCHITECTURE.md (if applicable)
Added code examples for new features (if applicable)
Updated requirements.yaml (if applicable)

Additional Notes

All 204 tests pass across net8.0, net9.0, and net10.0. Platform requirements (platform-requirements.yaml) retain ReviewMark_* self-validation tests with source filters — this is intentional as those are the platform qualification mechanism, not regular integration tests.

Agent-Logs-Url: https://github.com/demaconsulting/ReviewMark/sessions/5fe50d02-c30f-45d3-a137-438007a8d9b6 Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

…or test-linkage compliance - Add 7 new IntegrationTest_* methods to IntegrationTests.cs covering: ReviewPlanGeneration, ReviewReportGeneration, Enforce, IndexScan, WorkingDirectoryOverride, Elaborate, Lint - Add 13 new Cli_* methods to CliTests.cs covering: ResultsFlag, LogFlag, ErrorOutput, InvalidArgs, ExitCode, DefinitionFlag, PlanFlag, ReportFlag, EnforceFlag, DirFlag, ElaborateFlag, LintFlag, IndexFlag - Update docs/reqstream/review-mark/review-mark.yaml: system requirements now use only IntegrationTest_* tests and add children: links to subsystem IDs - Update docs/reqstream/review-mark/cli/cli.yaml: subsystem requirements now use only Cli_* tests; add new requirements for all CLI flags - Update docs/reqstream/review-mark/configuration/configuration.yaml: subsystem requirements use only Configuration_* tests - Update docs/reqstream/review-mark/indexing/indexing.yaml: subsystem requirements use only Indexing_* tests - Update docs/reqstream/review-mark/self-test/self-test.yaml: subsystem requirements use only SelfTest_* tests - Update docs/reqstream/review-mark/configuration/review-mark-configuration.yaml: remove Program_* test from ReviewMark-Config-Loading - Update docs/reqstream/review-mark/indexing/review-index.yaml: remove ReviewMarkConfiguration_* tests from ReviewMark-EvidenceSource-None - Update .reviewmark.yaml: add Purpose review; update ReviewMark-Architecture, ReviewMark-Design, ReviewMark-AllRequirements; remove user_guide and TestDirectory.cs from ReviewMark-Program review Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

…eqs, reorder review-sets Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

Copilot

Pull request overview

Adds new system-level (IntegrationTest_*) and CLI-subsystem (Cli_*) tests and restructures ReqStream/ReviewMark YAML so requirements link to tests at the correct hierarchy level (system → integration tests, subsystem → subsystem tests, unit → unit tests).

Changes:

Added 7 new IntegrationTest_* tests covering major CLI flows (plan/report/enforce/index/dir/elaborate/lint).
Added 13 new Cli_* subsystem tests covering key CLI flags and behaviors.
Updated ReqStream requirements YAML and .reviewmark.yaml review-set organization to enforce test-linkage hierarchy and consistent review scoping.

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
test/DemaConsulting.ReviewMark.Tests/IntegrationTests.cs	Adds new end-to-end integration tests for core CLI flows.
test/DemaConsulting.ReviewMark.Tests/Cli/CliTests.cs	Adds CLI subsystem tests for flags, file outputs, and error handling.
docs/reqstream/review-mark/self-test/self-test.yaml	Adjusts subsystem requirement test linkages to `SelfTest_*` tests only.
docs/reqstream/review-mark/review-mark.yaml	Updates system requirements to link only to `IntegrationTest_*` and adds `children` links.
docs/reqstream/review-mark/indexing/review-index.yaml	Removes mis-scoped configuration tests from indexing unit requirement.
docs/reqstream/review-mark/indexing/indexing.yaml	Updates indexing subsystem requirements to link only to `Indexing_*` tests and adds `children`.
docs/reqstream/review-mark/configuration/review-mark-configuration.yaml	Removes a mis-scoped `Program_*` test from configuration unit requirement.
docs/reqstream/review-mark/configuration/configuration.yaml	Updates configuration subsystem requirement test linkages and `children`.
docs/reqstream/review-mark/cli/cli.yaml	Updates CLI subsystem requirement test linkages to `Cli_*` tests and trims unit-test references.
.reviewmark.yaml	Reorganizes review-sets, adds `Purpose`, and aligns hierarchy/scoping with updated standards.
.github/standards/software-items.md	Adds naming guidance for hierarchy collisions.
.github/standards/reviewmark-usage.md	Updates usage examples and documents review-set design principles/hierarchy.
.github/standards/reqstream-usage.md	Documents test-linkage hierarchy and adds `children` field guidance.
.github/standards/csharp-testing.md	Updates AAA comment style and adds hierarchical test naming standards.
.github/agents/repo-consistency.agent.md	Converts template placeholders to `{...}` format and fixes indentation.
.github/agents/quality.agent.md	Converts template placeholders to `{...}` format and updates checklist wording.
.github/agents/implementation.agent.md	Converts template placeholders to `{...}` format and fixes indentation.
.github/agents/developer.agent.md	Converts template placeholders to `{...}` format and fixes indentation.
.github/agents/code-review.agent.md	Updates elaborate syntax example and converts placeholders to `{...}` format.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

test/DemaConsulting.ReviewMark.Tests/Cli/CliTests.cs

test/DemaConsulting.ReviewMark.Tests/IntegrationTests.cs

docs/reqstream/review-mark/configuration/configuration.yaml

docs/reqstream/review-mark/cli/cli.yaml

docs/reqstream/review-mark/indexing/indexing.yaml

Malcolmnixon · 2026-04-05T02:03:05Z

@copilot The DemaConsulting.ReviewMark.Tests.Cli.CliTests.Cli_LogFlag_WritesOutputToFile test is failing on Windows because a file is still in use. I think the test isn't disposing of the context before accessing the file. Please fix this and any other tests with the same issue.

…putToFile Agent-Logs-Url: https://github.com/demaconsulting/ReviewMark/sessions/faf4bc52-4fa8-491e-a753-863b18c1e6a6 Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

Copilot · 2026-04-05T02:07:14Z

@copilot The DemaConsulting.ReviewMark.Tests.Cli.CliTests.Cli_LogFlag_WritesOutputToFile test is failing on Windows because a file is still in use. I think the test isn't disposing of the context befo...

Fixed in ebc7068. The test was using using var context (declaration-scoped), which keeps the StreamWriter log file handle open until the end of the try block — after File.ReadAllText and the finally's File.Delete had already been called on Windows.

Changed to using (var context = ...) { } (block-scoped) with exitCode captured inside the block, and the file read/delete moved outside. This matches the pattern already used in ContextTests.Context_Create_LogFlag_OpensLogFile. No other tests had the same issue — only Cli_LogFlag_WritesOutputToFile opens a file handle through the Context.

Malcolmnixon · 2026-04-05T02:12:49Z

@copilot apply changes based on the comments in this thread

…th tests, traceability tests Agent-Logs-Url: https://github.com/demaconsulting/ReviewMark/sessions/bb1deea5-108a-46ac-9875-9199ef6993e9 Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

…pth assertions Agent-Logs-Url: https://github.com/demaconsulting/ReviewMark/sessions/bb1deea5-108a-46ac-9875-9199ef6993e9 Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

Copilot · 2026-04-05T02:23:20Z

@copilot apply changes based on the comments in this thread

All review thread feedback applied in 677d91a and 752acd9:

Temp-file leaks (CliTests.cs ×7, IntegrationTests.cs ×4): replaced Path.GetTempFileName() + Path.ChangeExtension with Path.GetRandomFileName()/Guid.NewGuid() patterns that never pre-create a file
Cli_ErrorOutput_WritesToStderr: rewrote to invoke Program.Main via reflection and assert stderr contains Error: and --unknown-arg-xyz
Cli_PlanDepthFlag_SetsHeadingDepth and Cli_ReportDepthFlag_SetsHeadingDepth: added new subsystem tests that pass --plan-depth 2/--report-depth 2 and assert ## Review Coverage/## Review Status in the output
Configuration_LoadConfig_ReportGenerationSucceeds and Configuration_LoadConfig_ElaborationSucceeds: added new tests calling PublishReviewReport and ElaborateReviewSet; updated configuration.yaml to link to them
Indexing_SafePathCombine_WithTraversalInputs_Throws: added new test asserting ArgumentException on traversal and absolute path inputs; updated indexing.yaml to include it
cli.yaml: updated PlanDepth and ReportDepth requirements to link to the new dedicated tests

All 204 tests pass across net8.0, net9.0, net10.0.

Copilot AI and others added 3 commits April 5, 2026 00:40

Fix review set spelling: 'review set' -> 'review-set' in test comments

0e54369

Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

Copilot AI assigned Copilot and Malcolmnixon Apr 5, 2026

fix: address quality findings - fix cspell wdir, add missing system r…

d5fc5e6

…eqs, reorder review-sets Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

Copilot created this pull request from a session on behalf of Malcolmnixon April 5, 2026 01:52 View session

Copilot AI requested a review from Malcolmnixon April 5, 2026 01:52

Copilot finished work on behalf of Malcolmnixon April 5, 2026 01:52

Malcolmnixon marked this pull request as ready for review April 5, 2026 01:53

Copilot AI review requested due to automatic review settings April 5, 2026 01:53

Copilot started reviewing on behalf of Malcolmnixon April 5, 2026 01:54 View session

Copilot AI reviewed Apr 5, 2026

View reviewed changes

Copilot started work on behalf of Malcolmnixon April 5, 2026 02:03 View session

Copilot finished work on behalf of Malcolmnixon April 5, 2026 02:07

Copilot started work on behalf of Malcolmnixon April 5, 2026 02:12 View session

Copilot AI and others added 2 commits April 5, 2026 02:20

Copilot finished work on behalf of Malcolmnixon April 5, 2026 02:24

Malcolmnixon approved these changes Apr 5, 2026

View reviewed changes

Malcolmnixon merged commit 6b61889 into main Apr 5, 2026
15 checks passed

Malcolmnixon deleted the copilot/copy-updated-github-files branch April 5, 2026 12:32

This was referenced Apr 6, 2026

Bump the nuget-dependencies group with 7 updates demaconsulting/NuGetCaching#33

Merged

Bump the nuget-dependencies group with 8 updates demaconsulting/SarifMark#92

Merged

Bump the nuget-dependencies group with 6 updates demaconsulting/NuGetCacheTool#29

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add IntegrationTest_* and Cli_* tests; update reqstream YAML for test-linkage compliance#46

Add IntegrationTest_* and Cli_* tests; update reqstream YAML for test-linkage compliance#46
Malcolmnixon merged 7 commits intomainfrom
copilot/copy-updated-github-files

Copilot AI commented Apr 5, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Malcolmnixon commented Apr 5, 2026

Uh oh!

Copilot AI commented Apr 5, 2026

Uh oh!

Malcolmnixon commented Apr 5, 2026

Uh oh!

Copilot AI commented Apr 5, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Copilot AI commented Apr 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of Change

Pre-Submission Checklist

Build and Test

Code Quality

Quality Checks

Testing

Documentation

Additional Notes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Malcolmnixon commented Apr 5, 2026

Uh oh!

Copilot AI commented Apr 5, 2026

Uh oh!

Malcolmnixon commented Apr 5, 2026

Uh oh!

Copilot AI commented Apr 5, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Apr 5, 2026 •

edited

Loading