Skip to content

Fix documentation accuracy and test parallelization issues#31

Merged
Malcolmnixon merged 6 commits intomainfrom
copilot/fix-report-format-issue
Jan 26, 2026
Merged

Fix documentation accuracy and test parallelization issues#31
Malcolmnixon merged 6 commits intomainfrom
copilot/fix-report-format-issue

Conversation

Copy link
Contributor

Copilot AI commented Jan 26, 2026

Pull Request

Description

This PR addresses technical documentation inaccuracies and fixes intermittent test failures caused by parallel execution.

Changes to README.md:

  • Removed **Results:** 2 line that never appears in generated markdown reports (only in console output)
  • Fixed rule ID format from warning TEST001: to warning [TEST001] to match actual code behavior

Changes to docs/guide/guide.md:

  • Removed inaccurate claims that reports include "code snippets" and "remediation guidance"
  • Updated "Report Format" section to accurately reflect what the tool actually outputs:
    • File location and line number
    • Severity level
    • Rule ID
    • Issue message

Changes to test/DemaConsulting.SarifMark.Tests/AssemblyInfo.cs:

  • Disabled test parallelization to fix intermittent failures when capturing console output
  • Changed from [assembly: Parallelize(Scope = ExecutionScope.ClassLevel)] to [assembly: DoNotParallelize]
  • Multiple tests capture Console.Out (a global resource), causing race conditions when run in parallel

Before (README.md):

**Tool:** MockTool 1.0.0

**Results:** 2

## Results

Found 2 results

src/Program.cs(42): warning TEST001: Test issue 1

After (README.md):

**Tool:** MockTool 1.0.0

## Results

Found 2 results

src/Program.cs(42): warning [TEST001] Test issue 1

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Code quality improvement

Related Issues

Pre-Submission Checklist

Before submitting this pull request, ensure you have completed the following:

Build and Test

  • Code builds successfully: dotnet build --configuration Release
  • All tests pass: dotnet test --configuration Release
  • Code produces zero warnings

Code Quality

  • Code formatting is correct: dotnet format --verify-no-changes
  • New code has appropriate XML documentation comments
  • Static analyzer warnings have been addressed

Quality Checks

Please run the following checks before submitting:

  • Spell checker passes: cspell "**/*.{md,cs}"
  • Markdown linter passes: markdownlint "**/*.md"
  • YAML linter passes: yamllint '**/*.{yml,yaml}'

Testing

  • Added unit tests for new functionality
  • Updated existing tests if behavior changed
  • All tests follow the AAA (Arrange, Act, Assert) pattern
  • Test coverage is maintained or improved

Documentation

  • Updated README.md (if applicable)
  • Updated ARCHITECTURE.md (if applicable)
  • Added code examples for new features (if applicable)
  • Updated requirements.yaml (if applicable)

Additional Notes

Verified with actual tool execution that generated markdown matches the updated documentation. Tested all command-line options mentioned in the guide (--report-depth, --heading, --enforce, --validate, --results) to ensure documentation accuracy.

Test Stability: Disabled test parallelization using [assembly: DoNotParallelize] attribute. Ran tests 5 consecutive times to verify elimination of intermittent failures. All 62 tests pass consistently with sequential execution.

Custom agent used: Documentation Writer
Expert agent for creating, updating, and maintaining project documentation including README, guides, and API documentation

Original prompt

Could you check the report format example in the README.md? It indicates the main section has a **Results:** 2 but I'm not seeing that in the code or generated reports. Please ensure technical accuracy in the documentation.

Custom agent used: Documentation Writer
Expert agent for creating, updating, and maintaining project documentation including README, guides, and API documentation


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix technical accuracy in documentation for report format Fix README report format example to match actual output Jan 26, 2026
Copilot AI requested a review from Malcolmnixon January 26, 2026 18:17
@Malcolmnixon
Copy link
Contributor

@copilot Did you check the guide for accuracy - it mentions the results may include code snippets. Is that true?

Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Jan 26, 2026

@copilot Did you check the guide for accuracy - it mentions the results may include code snippets. Is that true?

You're correct - the guide incorrectly claimed results include code snippets and remediation guidance. The tool only extracts and displays: file location, line number, severity level, rule ID, and issue message. Fixed in commit db1fe8c.

Copilot AI changed the title Fix README report format example to match actual output Fix README and guide documentation to match actual report output Jan 26, 2026
@Malcolmnixon
Copy link
Contributor

@copilot The PR is encountering intermittent failures. I don't believe the tests are safe to run in parallel if they're capturing console output. Please disable all parallelization if stdout capturing is being used.

…e output capturing

Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Jan 26, 2026

@copilot The PR is encountering intermittent failures. I don't believe the tests are safe to run in parallel if they're capturing console output. Please disable all parallelization if stdout capturing...

Fixed in commit 525a654. Changed test parallelization from 4 workers to 1 worker (sequential execution) to prevent race conditions when capturing console output. Verified with 5 consecutive test runs - all passed.

Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>
Copilot AI changed the title Fix README and guide documentation to match actual report output Fix documentation accuracy and test parallelization issues Jan 26, 2026
Copilot AI requested a review from Malcolmnixon January 26, 2026 18:42
Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>
@Malcolmnixon Malcolmnixon marked this pull request as ready for review January 26, 2026 18:45
@Malcolmnixon Malcolmnixon merged commit a2dc03d into main Jan 26, 2026
16 of 22 checks passed
@Malcolmnixon Malcolmnixon deleted the copilot/fix-report-format-issue branch January 26, 2026 18:55
Copilot AI added a commit that referenced this pull request Mar 4, 2026
Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>
Malcolmnixon added a commit that referenced this pull request Mar 4, 2026
* Initial plan

* Apply template updates from PRs #31-#51

Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

* Apply semantic requirement IDs from template PR #50

Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants