Add ReviewMark integration (review plan, review report, multi-set config, ValidationTests)#58
Conversation
- Add demaconsulting.reviewmark 0.1.0-rc.3 to .config/dotnet-tools.json - Create .reviewmark.yaml with three review sets (CLI, Utilities, Tests) - Create docs/reviewplan/ with definition.yaml, title.txt, introduction.md - Create docs/reviewreport/ with definition.yaml, title.txt, introduction.md - Add Template-OTS-ReviewMark requirement to requirements.yaml - Add reviewmark/ReviewMark/reviewplan/reviewreport words to .cspell.json - Update build.yaml: add reviewmark to tool versions capture, add self- validation step, add ReviewMark report generation, add HTML/PDF generation steps for review plan and review report
Malcolmnixon
left a comment
There was a problem hiding this comment.
You seem to have forgotten some changes - such as the new code-review-agent.md file.
Please review demaconsulting/TemplateDotNetLibrary#37 and make sure you picked up everything.
…ersionmark Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>
Done in commit |
…eviewmark review set Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Introduces ReviewMark into the TemplateDotNetTool repository to generate and publish review plan/report documentation as part of the existing “build-docs” pipeline, and links the new self-validation + CLI validation behaviors into the requirements trace.
Changes:
- Add ReviewMark configuration (
.reviewmark.yaml) with three review-sets aligned to CLI, utilities, and test infrastructure. - Extend the build-docs GitHub Actions workflow to run ReviewMark (self-validation + plan/report generation) and to produce HTML/PDF outputs for the new documents.
- Add
ValidationTestsand wire positive validation tests + ReviewMark OTS requirement intorequirements.yaml.
Reviewed changes
Copilot reviewed 14 out of 15 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| test/DemaConsulting.TemplateDotNetTool.Tests/ValidationTests.cs | Adds unit coverage for Validation.Run summary output, exit code behavior, and results file generation. |
| requirements.yaml | Links the new validation tests to CLI requirements and adds an OTS requirement for ReviewMark plan/report generation. |
| docs/reviewplan/title.txt | Adds Pandoc title metadata for the generated Review Plan document. |
| docs/reviewplan/introduction.md | Adds Review Plan introduction content used in generated documentation. |
| docs/reviewplan/definition.yaml | Adds Pandoc defaults for rendering the Review Plan to HTML/PDF. |
| docs/reviewreport/title.txt | Adds Pandoc title metadata for the generated Review Report document. |
| docs/reviewreport/introduction.md | Adds Review Report introduction content used in generated documentation. |
| docs/reviewreport/definition.yaml | Adds Pandoc defaults for rendering the Review Report to HTML/PDF. |
| .reviewmark.yaml | Defines review scope patterns, evidence source, and three review-sets for ReviewMark. |
| .github/agents/code-review-agent.md | Adds repo-specific instructions for performing formal review-set reviews using ReviewMark. |
| .github/workflows/build.yaml | Integrates ReviewMark into build-docs: version capture, self-validation, plan/report generation, and HTML/PDF publishing. |
| .config/dotnet-tools.json | Adds the demaconsulting.reviewmark local tool to the tool manifest. |
| .versionmark.yaml | Updates tool version regexes to support pre-release versions and adds ReviewMark version capture. |
| .cspell.json | Adds ReviewMark-related words to the repository spelling dictionary. |
| .gitignore | Ignores generated review plan/report markdown outputs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
|
@copilot The new tests are failing on Windws: These tests run concurrently multitargeting and so need to be safe from collisions. |
…ndows file lock Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>
... Fixed in commit |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 14 out of 15 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
@copilot Please run the TemplateTool-CLI-Review code-review |
…e default-context check Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>
Done in commit
No other issues found — all other code paths, error handling, file lifecycle, and test assertions look correct. |
|
@copilot Please run the code-review agent to perform the formal review of the TemplateTool-Utilities-Review review-set |
Formal review of Files reviewed (fingerprint
2.3 Code Checks
2.4 Testing Checks
2.8 Code vs Documentation Checks
Overall Outcome: ✅ PassNo failures. Four non-blocking observations:
|
|
@copilot Please use the code-review agent to run the formal review of the TemplateTool-Tests-Review review-set. |
….tmp files (OBS-01 from TemplateTool-Tests-Review) Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>
Formal review of Files reviewed (fingerprint
2.3 Code Checks
2.4 Testing Checks
2.8 Code vs Documentation Checks
Overall Outcome: ✅ PassOBS-01 (CODE-04) — fixed in commit Two remaining non-blocking observations (no source changes required):
|
Brings in ReviewMark integration, adds
ValidationTests.cs, and performs formal code reviews of all three review-sets (TemplateTool-CLI-Review,TemplateTool-Utilities-Review,TemplateTool-Tests-Review) per the code-review-agent checklist.ReviewMark integration
.reviewmark.yaml— three review sets, each pairing source with tests:TemplateTool-CLI-Review(Program.cs,Context.cs+ProgramTests.cs,ContextTests.cs),TemplateTool-Utilities-Review(PathHelpers.cs,Validation.cs+PathHelpersTests.cs,ValidationTests.cs),TemplateTool-Tests-Review(IntegrationTests.cs,Runner.cs,AssemblyInfo.cs).config/dotnet-tools.json— addsdemaconsulting.reviewmarkv0.1.0-rc.3.versionmark.yaml— addsreviewmarkentry; pre-release regex ((?:-[a-zA-Z0-9.]+)?) applied to all tool patterns.gitignore— excludes generatedreview-plan.md,review-report.md, andAGENT_REPORT_*.mdfiles.github/agents/code-review-agent.md— agent instructions for performing formal reviews against the review-set IDsReview plan / report documents
docs/reviewplan/,docs/reviewreport/— Pandoc definition, title, and introduction for each document; introduction wording corrected to "This plan" instead of "This report".github/workflows/build.yaml— ReviewMark self-validation, plan and report generation (markdown → HTML → PDF) added tobuild-docsjobValidationTests
test/.../ValidationTests.cs— 6 tests covering null guard, summary output, exit code, TRX output, JUnit XML output, and unsupported format; usesGuid-based temp paths and reads log files only afterContextis disposed (prevents Windows file-lock errors under concurrent multi-target runs)requirements.yaml—Template-OTS-ReviewMarkOTS requirement linked to ReviewMark self-validation tests; positiveValidationTestslinked toTemplate-Cmd-ValidateandTemplate-Cmd-ResultsTemplateTool-CLI-Review findings (formal checklist — fingerprint
4045175d…)All checks pass. Two non-blocking observations addressed:
ProgramTests:Program_Run_WithVersionFlag_DisplaysVersionOnlylacked a positive assertion — addedAssert.Contains(Program.Version, output)ContextTests:Context_Create_NoArguments_ReturnsDefaultContextmissingResultsFilecheck — addedAssert.IsNull(context.ResultsFile)TemplateTool-Utilities-Review findings (formal checklist — fingerprint
df3d06d7…)All checks pass (CODE-01–06, TEST-01–04, CODEDOC-01–02). Four non-blocking observations recorded in
AGENT_REPORT_review-TemplateTool-Utilities-Review.md(excluded from git). No source changes required.TemplateTool-Tests-Review findings (formal checklist — fingerprint
22353d9a…)All checks pass (CODE-01–06, TEST-01–04, CODEDOC-01–02). Three non-blocking observations recorded in
AGENT_REPORT_review-TemplateTool-Tests-Review.md(excluded from git). One observation addressed:IntegrationTests:GeneratesTrxFileandGeneratesJUnitFileusedPath.GetTempFileName()+Path.ChangeExtension(), leaving orphaned.tmpfiles on disk — switched toGuid-based paths (consistent withValidationTests.cs)Type of Change
Related Issues
Pre-Submission Checklist
Before submitting this pull request, ensure you have completed the following:
Build and Test
dotnet build --configuration Releasedotnet test --configuration Releasedotnet run --project src/DemaConsulting.TemplateDotNetTool --configuration Release --framework net10.0--no-build -- --validateCode Quality
dotnet format --verify-no-changesQuality Checks
Please run the following checks before submitting:
cspell "**/*.{md,cs}"markdownlint "**/*.md"yamllint .Testing
Documentation
Additional Notes
The
reviewsbranch (holding review evidence PDFs andindex.json) does not yet exist. The--enforceflag on the ReviewMark step should not be added until that branch is populated.Only positive
ValidationTestsare linked to requirements; negative/guard tests are intentionally unlinked per project policy.💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.