Enhance self-validation tests for DotNet, C++, and VHDL generation

[Malcolmnixon]

This pull request significantly expands the self-validation suite for ApiMarkTool, ensuring that the tool's real DotNet, C++, and VHDL documentation generators and enforcement pipelines are exercised end-to-end during validation. It adds new functional and enforcement tests, robustly handles environments lacking clang, and updates the documentation and requirements to reflect these features. The changes also improve test reporting by distinguishing skipped tests and suppressing generator output during validation.

Self-Validation Enhancements

• The validation suite now runs functional generation tests for DotNet, C++, and VHDL generators, verifying that each generator produces expected Markdown output from embedded sample sources. [1] [2] [3] [4]
• Enforcement tests for documentation coverage are added for all three generators, checking that undocumented items are correctly detected and that documented items are not falsely flagged. [1] [2] [3] [4]
• All generator invocations during validation are run in silent, log-capturing child contexts to keep the validation transcript clean. [1] [2] [3]

Clang Availability and Skipped Tests

• The C++ generation and enforcement tests are gated on clang availability, using the new ApiMark.Cpp.CppAst.ClangDiscovery helper. When clang is not found, these tests are recorded as skipped (not failed) and do not set a non-zero exit code. [1] [2] [3] [4] [5] Ffdbf4c8L19R19, [6] [7]
• The validation summary now includes a "Skipped: N" count for visibility in CI and operator logs. [1] [2] [3]

Documentation and Requirements Updates

• User and FAQ documentation now describe the expanded validation coverage, the handling of skipped C++ tests, and the updated test result summary. (Ffdbf4c8L19R19, [1] [2]
• Requirements and verification documents are updated to reflect the new tests, their gating/skipping logic, and the use of ClangDiscovery. [1] [2]

Test Ownership and Coverage

• The .reviewmark.yaml file is updated to assign new source and test files (ClangDiscovery.cs, ClangDiscoveryTests.cs) to the correct review group, ensuring proper code ownership and review coverage.

[Copilot]

Pull request overview

This PR expands ApiMark.Tool’s in-process --validate self-test suite so it exercises the real DotNet, C++, and VHDL generation and documentation-coverage enforcement pipelines end-to-end, while handling clang-unavailable environments by marking C++ tests as skipped (NotExecuted) instead of failing.

Changes:

• Added functional generation self-tests for DotNet, C++, and VHDL, plus enforce-docs self-tests for the same generators, with generator output routed into silent/log-capturing child contexts.
• Introduced ApiMark.Cpp.CppAst.ClangDiscovery and refactored clang discovery to support a non-throwing availability probe; improved macOS xcrun discovery by probing xcrun -find clang.
• Updated test assertions, documentation, and requirements/verification artifacts to reflect the expanded (8-test) validation suite and skipped-test reporting.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
test/ApiMark.Tool.Tests/SelfTest/ValidationTests.cs	Updates expected results count to 8 and adds assertions around functional/enforcement test names and skipped reporting.
test/ApiMark.Cpp.Tests/ClangDiscoveryTests.cs	Adds unit coverage for ClangDiscovery.IsAvailable behavior.
src/ApiMark.Tool/SelfTest/Validation.cs	Extends validation to run generation + enforcement tests and prints a skipped count.
src/ApiMark.Tool/SelfTest/Validation.DotNet.cs	Adds DotNet generator functional self-test.
src/ApiMark.Tool/SelfTest/Validation.Cpp.cs	Adds C++ generator functional self-test with clang gating (skip).
src/ApiMark.Tool/SelfTest/Validation.Vhdl.cs	Adds VHDL generator functional self-test.
src/ApiMark.Tool/SelfTest/Validation.EnforceDocs.DotNet.cs	Adds DotNet enforce-docs functional self-test against the tool’s own assembly/XML docs.
src/ApiMark.Tool/SelfTest/Validation.EnforceDocs.Cpp.cs	Adds C++ enforce-docs functional self-test with clang gating (skip).
src/ApiMark.Tool/SelfTest/Validation.EnforceDocs.Vhdl.cs	Adds VHDL enforce-docs functional self-test with embedded sample source.
src/ApiMark.Cpp/CppAst/ClangDiscovery.cs	Introduces a public helper to preflight clang availability using shared discovery logic.
src/ApiMark.Cpp/CppAst/ClangAstParser.cs	Refactors discovery to a non-throwing helper and adds an xcrun probe with timeout/kill handling.
docs/verification/api-mark-tool/self-test/validation.md	Updates verification plan/criteria to reflect expanded validation coverage and skipped tests.
docs/verification/api-mark-tool/self-test.md	Updates self-test verification summary (but currently has inconsistencies noted in review comments).
docs/verification/api-mark-cpp/clang-ast-parser.md	Documents ClangDiscovery verification and its relationship to clang discovery.
docs/user_guide/faq.md	Updates FAQ to describe expanded validation and C++ skip behavior.
docs/user_guide/cli.md	Updates CLI docs for --validate/--results and skipped-count reporting (with a wording mismatch noted).
docs/reqstream/api-mark-tool/self-test/validation.yaml	Adds requirements entries covering new generation/enforcement tests, skipping logic, and output suppression.
docs/reqstream/api-mark-tool/self-test.yaml	Links new validation requirements into the self-test requirement set.
docs/design/api-mark-tool/self-test/validation.md	Updates design documentation to reflect expanded test suite and clang gating.
.reviewmark.yaml	Assigns new clang discovery source/test files to the appropriate review group.

Suppressed comments (5)

docs/verification/api-mark-tool/self-test.md:36

• The scenario description still says ExitCode=0 only when "both internal self-tests" pass, but validation now runs additional self-tests and permits skips (e.g., clang unavailable). The scenario text should match the current behavior and the test it references.

**Self-validation with valid context exits zero**: Verifies that
`Validation.Run` completes with `ExitCode = 0` when both internal self-tests
pass. Tested by `Validation_Run_WithValidContext_ExitsZero`.

docs/verification/api-mark-tool/self-test.md:41

• This section says the TRX file contains "all five" results and refers to a single C++ test, but validation now writes 8 results and includes both C++ generation and enforcement tests.

`.trx` path, the file is created and contains `"TestRun"` and all five
self-test results, including the C++ test's `Passed` or `NotExecuted`
outcome. Tested by `Validation_Run_WithResultsTrxFile_CreatesTrxFile`.

docs/verification/api-mark-tool/self-test.md:45

• This section says the XML results file contains "all five" results, but validation now writes 8 results.

`.xml` path, the file is created and contains all five self-test results.
Tested by `Validation_Run_WithResultsXmlFile_CreatesXmlFile`.

docs/verification/api-mark-tool/self-test.md:29

• Acceptance criteria refers to a single "C++ functional test", but validation now has two clang-gated C++ tests (generation and enforce-docs).

- The C++ functional test is recorded as skipped (`TestOutcome.NotExecuted`),
  not failed, when clang cannot be located, and does not set `ExitCode` to 1.

docs/verification/api-mark-tool/self-test.md:58

• This scenario is described as checking only generation test names, but the referenced test also asserts the enforce-docs test names. The scenario text should match what the test actually checks.

**Output mentions all three functional generation test names**: Verifies
that the log output contains `"ApiMark_DotNetGeneration"`,
`"ApiMark_CppGeneration"`, and `"ApiMark_VhdlGeneration"`. Tested by
`Validation_Run_WritesFunctionalGenerationTestResults`.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.