From 2e69a50a11c6e34afe3aa1eb78f9954ec9d3e31f Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Sat, 28 Mar 2026 20:21:02 +0000 Subject: [PATCH 01/18] Initial plan From 2e52e4299913dfd27c255ba41779620cbce89f17 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Sat, 28 Mar 2026 20:40:53 +0000 Subject: [PATCH 02/18] Add design documentation and update review-sets in .reviewmark.yaml Agent-Logs-Url: https://github.com/demaconsulting/ReviewMark/sessions/a5158598-c9e7-4185-a00d-0b6c7069931e Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com> --- .reviewmark.yaml | 44 +++++++++++++ docs/design/context.md | 63 +++++++++++++++++++ docs/design/definition.yaml | 18 ++++++ docs/design/glob-matcher.md | 42 +++++++++++++ docs/design/index.md | 64 +++++++++++++++++++ docs/design/introduction.md | 75 ++++++++++++++++++++++ docs/design/path-helpers.md | 39 ++++++++++++ docs/design/program.md | 54 ++++++++++++++++ docs/design/review-mark-configuration.md | 80 ++++++++++++++++++++++++ docs/design/system.md | 72 +++++++++++++++++++++ docs/design/title.txt | 13 ++++ docs/design/validation.md | 52 +++++++++++++++ 12 files changed, 616 insertions(+) create mode 100644 docs/design/context.md create mode 100644 docs/design/definition.yaml create mode 100644 docs/design/glob-matcher.md create mode 100644 docs/design/index.md create mode 100644 docs/design/introduction.md create mode 100644 docs/design/path-helpers.md create mode 100644 docs/design/program.md create mode 100644 docs/design/review-mark-configuration.md create mode 100644 docs/design/system.md create mode 100644 docs/design/title.txt create mode 100644 docs/design/validation.md diff --git a/.reviewmark.yaml b/.reviewmark.yaml index d964e3c..27bd070 100644 --- a/.reviewmark.yaml +++ b/.reviewmark.yaml @@ -7,7 +7,9 @@ # Processed in order; prefix a pattern with '!' to exclude. needs-review: - "**/*.cs" # All C# source and test files + - "requirements.yaml" # Root requirements file - "docs/reqstream/*.yaml" # Per-software-item requirements files + - "docs/design/*.md" # Software design documents - "!**/obj/**" # Exclude build output - "!**/bin/**" # Exclude build output @@ -31,12 +33,14 @@ reviews: title: Review of Context software unit (command-line argument handling) paths: - "docs/reqstream/cli-requirements.yaml" # requirements + - "docs/design/context.md" # design - "src/**/Context.cs" # implementation - "test/**/ContextTests.cs" # tests - id: ReviewMark-GlobMatcher title: Review of GlobMatcher software unit (file pattern matching) paths: + - "docs/design/glob-matcher.md" # design - "src/**/GlobMatcher.cs" # implementation - "test/**/GlobMatcherTests.cs" # tests @@ -44,12 +48,14 @@ reviews: title: Review of Index software unit (review evidence indexing) paths: - "docs/reqstream/index-requirements.yaml" # requirements + - "docs/design/index.md" # design - "src/**/Index.cs" # implementation - "test/**/IndexTests.cs" # tests - id: ReviewMark-PathHelpers title: Review of PathHelpers software unit (file path utilities) paths: + - "docs/design/path-helpers.md" # design - "src/**/PathHelpers.cs" # implementation - "test/**/PathHelpersTests.cs" # tests @@ -58,6 +64,7 @@ reviews: paths: - "docs/reqstream/cli-requirements.yaml" # requirements - "docs/reqstream/platform-requirements.yaml" # platform requirements + - "docs/design/program.md" # design - "docs/guide/guide.md" # user guide - "src/**/Program.cs" # implementation - "test/**/ProgramTests.cs" # unit tests @@ -70,6 +77,7 @@ reviews: title: Review of ReviewMarkConfiguration software unit (configuration parsing and processing) paths: - "docs/reqstream/configuration-requirements.yaml" # requirements + - "docs/design/review-mark-configuration.md" # design - "src/**/ReviewMarkConfiguration.cs" # implementation - "test/**/ReviewMarkConfigurationTests.cs" # tests @@ -77,4 +85,40 @@ reviews: title: Review of Validation software unit (self-validation test execution) paths: - "docs/reqstream/ots-requirements.yaml" # OTS requirements verified by self-validation + - "docs/design/validation.md" # design - "src/**/Validation.cs" # implementation + + # Special review-sets + - id: ReviewMark-System + title: Review of ReviewMark system-level behavior, platform support, and integration + paths: + - "requirements.yaml" # system requirements (root) + - "docs/reqstream/platform-requirements.yaml" # platform requirements + - "docs/design/introduction.md" # design introduction and architecture + - "docs/design/system.md" # system design + - "test/**/IntegrationTests.cs" # integration tests + + - id: ReviewMark-Design + title: Review of all ReviewMark design documentation + paths: + - "requirements.yaml" # system requirements (root) + - "docs/reqstream/platform-requirements.yaml" # platform requirements + - "docs/design/introduction.md" # design introduction and architecture + - "docs/design/system.md" # system design + - "docs/design/context.md" # Context design + - "docs/design/glob-matcher.md" # GlobMatcher design + - "docs/design/index.md" # Index design + - "docs/design/path-helpers.md" # PathHelpers design + - "docs/design/program.md" # Program design + - "docs/design/review-mark-configuration.md" # ReviewMarkConfiguration design + - "docs/design/validation.md" # Validation design + + - id: ReviewMark-AllRequirements + title: Review of all ReviewMark requirements files + paths: + - "requirements.yaml" # root requirements file + - "docs/reqstream/cli-requirements.yaml" # CLI subsystem requirements + - "docs/reqstream/configuration-requirements.yaml" # Configuration subsystem requirements + - "docs/reqstream/index-requirements.yaml" # Index subsystem requirements + - "docs/reqstream/platform-requirements.yaml" # Platform support requirements + - "docs/reqstream/ots-requirements.yaml" # OTS component requirements diff --git a/docs/design/context.md b/docs/design/context.md new file mode 100644 index 0000000..cd718f8 --- /dev/null +++ b/docs/design/context.md @@ -0,0 +1,63 @@ +# Context + +## Purpose + +The `Context` software unit is responsible for parsing command-line arguments and +providing a unified interface for output and logging throughout the tool. It acts as +the primary configuration carrier, passing parsed options from the CLI entry point +to all processing subsystems. + +## Properties + +The following properties are populated by `Context.Create()` from the command-line +arguments: + +| Property | Type | Description | +| -------- | ---- | ----------- | +| `Version` | bool | Requests version display | +| `Help` | bool | Requests help display | +| `Silent` | bool | Suppresses console output | +| `Validate` | bool | Requests self-validation run | +| `Lint` | bool | Requests configuration linting | +| `ResultsFile` | string? | Path for TRX/JUnit test results output | +| `DefinitionFile` | string | Path to the `.reviewmark.yaml` configuration | +| `PlanFile` | string? | Output path for the Review Plan document | +| `PlanDepth` | int | Heading depth for the Review Plan | +| `ReportFile` | string? | Output path for the Review Report document | +| `ReportDepth` | int | Heading depth for the Review Report | +| `IndexPaths` | string[]? | Paths to scan when building an evidence index | +| `WorkingDirectory` | string | Base directory for resolving relative paths | +| `Enforce` | bool | Fail if any review-set is not Current | +| `Elaborate` | bool | Expand file lists in generated documents | + +## Argument Parsing + +`Context.Create(string[] args)` is a factory method that processes the argument +array sequentially, recognizing both flag arguments (e.g., `--validate`) and +value arguments (e.g., `--plan `). Unrecognized arguments are silently +ignored. The resulting `Context` instance holds the fully parsed state. + +## Output Methods + +| Method | Description | +| ------ | ----------- | +| `WriteLine(string)` | Writes a line to the console (unless `Silent` is set) and to the log file | +| `WriteError(string)` | Writes an error line to the console and to the log file | + +## Exit Code + +`Context.ExitCode` reflects the current error status of the tool run. It is set to +a non-zero value when an error is detected. The value of `ExitCode` is returned from +`Program.Main()` as the process exit code. + +## Logging + +When a log file path is provided via the relevant CLI argument, `Context` opens and +holds the log file handle for the duration of the tool run. All output written through +`WriteLine` and `WriteError` is duplicated to the log file. + +## Resource Cleanup + +`Context` implements `IDisposable`. Disposal closes the log file handle if one is +open. `Program.Main()` constructs `Context` inside a `using` statement to guarantee +timely disposal regardless of how the tool exits. diff --git a/docs/design/definition.yaml b/docs/design/definition.yaml new file mode 100644 index 0000000..c0daf41 --- /dev/null +++ b/docs/design/definition.yaml @@ -0,0 +1,18 @@ +--- +resource-path: + - docs/design + - docs/template +input-files: + - docs/design/title.txt + - docs/design/introduction.md + - docs/design/system.md + - docs/design/context.md + - docs/design/glob-matcher.md + - docs/design/index.md + - docs/design/path-helpers.md + - docs/design/program.md + - docs/design/review-mark-configuration.md + - docs/design/validation.md +template: template.html +table-of-contents: true +number-sections: true diff --git a/docs/design/glob-matcher.md b/docs/design/glob-matcher.md new file mode 100644 index 0000000..b36b462 --- /dev/null +++ b/docs/design/glob-matcher.md @@ -0,0 +1,42 @@ +# GlobMatcher + +## Purpose + +The `GlobMatcher` software unit resolves an ordered list of glob patterns into a +concrete, sorted list of file paths relative to a base directory. It provides the +file enumeration primitive used by the Configuration subsystem to expand the +`needs-review` and `review-set` file lists defined in `.reviewmark.yaml`. + +## Algorithm + +`GlobMatcher.GetMatchingFiles(baseDirectory, patterns)` processes patterns in the +order they are declared. Each pattern is evaluated as follows: + +```mermaid +graph TD + A[For each pattern in order] --> B{Starts with '!'?} + B -->|yes| C[Remove '!' prefix → exclusion pattern] + B -->|no| D[Inclusion pattern] + C --> E[Remove matching paths from result set] + D --> F[Add matching paths to result set] + F --> A + E --> A + A --> G[Sort result set] + G --> H[Return as relative paths with forward slashes] +``` + +Patterns are evaluated against the base directory using standard glob semantics. +The `**` wildcard matches any number of path segments (recursive matching). + +## Return Value + +The method returns a sorted list of relative file paths. Path separators are +normalized to forward slashes regardless of the host operating system, ensuring +consistent fingerprint computation across platforms. + +## Usage + +`GlobMatcher.GetMatchingFiles()` is called by `ReviewMarkConfiguration` to resolve: + +- The `needs-review` file list, which represents all files subject to review +- Each `review-set` file list, which represents the files covered by a specific review record diff --git a/docs/design/index.md b/docs/design/index.md new file mode 100644 index 0000000..d83277d --- /dev/null +++ b/docs/design/index.md @@ -0,0 +1,64 @@ +# Index + +## Purpose + +The `Index` software unit manages the loading, querying, and creation of the review +evidence index. It abstracts the evidence store behind a uniform interface so that +the rest of the tool does not need to know whether evidence is stored on a fileshare, +served over HTTP, or absent entirely. + +## Evidence Index Format + +The evidence index is a JSON file (`index.json`) containing an array of review records. +Each record has the following fields: + +| Field | Type | Description | +| ----- | ---- | ----------- | +| `id` | string | Unique identifier for the review record (matches the review-set `id` in `.reviewmark.yaml`) | +| `fingerprint` | string | SHA-256 fingerprint of the file-set at time of review | +| `date` | string | Date the review was conducted | +| `result` | string | Review outcome (`pass` or `fail`) | +| `file` | string | Relative path to the PDF evidence file | + +## ReviewIndex.Load() + +`ReviewIndex.Load(EvidenceSource)` selects a loading strategy based on the evidence +source type: + +| Source Type | Behavior | +| ----------- | -------- | +| `none` | Returns an empty index (equivalent to `ReviewIndex.Empty()`) | +| `fileshare` | Reads `index.json` from the specified file path | +| `url` | Downloads `index.json` from the specified HTTP or HTTPS URL | + +## ReviewIndex.Scan() + +`ReviewIndex.Scan(directory, patterns)` scans a directory for PDF files matching +the given glob patterns. For each PDF file found, it reads embedded metadata to +extract the review record fields and writes the resulting records to `index.json` +in the scanned directory. This supports the `--index` workflow. + +## ReviewIndex.Empty() + +`ReviewIndex.Empty()` returns an index with no records. It is used when the evidence +source type is `none`, resulting in all review-sets being reported as Missing. + +## ReviewIndex.GetStatus() + +`ReviewIndex.GetStatus(id, fingerprint)` determines the review status of a +review-set by looking up the `id` in the loaded index: + +```mermaid +graph TD + A[Look up id in index] --> B{id found?} + B -->|no| C[Return Missing] + B -->|yes| D{fingerprint matches AND result is pass?} + D -->|yes| E[Return Current] + D -->|no| F[Return Stale] +``` + +| Status | Meaning | +| ------ | ------- | +| `Current` | The review record matches the current fingerprint and has a passing result | +| `Stale` | A review record exists for the id but the fingerprint does not match (files have changed since review) | +| `Missing` | No review record exists for the id | diff --git a/docs/design/introduction.md b/docs/design/introduction.md new file mode 100644 index 0000000..a2a3bed --- /dev/null +++ b/docs/design/introduction.md @@ -0,0 +1,75 @@ +# Introduction + +This document describes the software design for the ReviewMark project. + +## Purpose + +ReviewMark is a .NET command-line tool for automated file-review evidence management +in regulated environments. It computes cryptographic fingerprints of defined file-sets, +queries a review evidence store for corresponding review records, and produces compliance +documents on each CI/CD run. + +This design document describes the internal architecture, subsystems, and software units +that together implement the ReviewMark tool. It is intended to support development, +review, and maintenance activities. + +## Scope + +This design document covers: + +- The software system decomposition into subsystems and software units +- The responsibilities and interfaces of each software unit +- The algorithms and data flows used for fingerprinting, evidence lookup, and document generation +- The self-validation framework + +This document does not cover: + +- External CI/CD pipeline configuration +- Evidence store setup or administration +- Requirements traceability (see the Requirements Specification) + +## Software Architecture + +The following diagram shows the decomposition of the ReviewMark software system into +subsystems and software units. + +```mermaid +graph TD + System["ReviewMark (Software System)"] + + CLI["CLI Subsystem"] + Program["Program (Software Unit)"] + Context["Context (Software Unit)"] + + Config["Configuration Subsystem"] + RMConfig["ReviewMarkConfiguration (Software Unit)"] + Glob["GlobMatcher (Software Unit)"] + + IndexSub["Index Subsystem"] + Index["Index (Software Unit)"] + PathHelpers["PathHelpers (Software Unit)"] + + Validation["Validation (Software Unit)"] + + System --> CLI + System --> Config + System --> IndexSub + System --> Validation + + CLI --> Program + CLI --> Context + + Config --> RMConfig + Config --> Glob + + IndexSub --> Index + IndexSub --> PathHelpers +``` + +## Audience + +This document is intended for: + +- Software developers working on ReviewMark +- Quality assurance teams performing design verification +- Project stakeholders reviewing architectural decisions diff --git a/docs/design/path-helpers.md b/docs/design/path-helpers.md new file mode 100644 index 0000000..b75bc68 --- /dev/null +++ b/docs/design/path-helpers.md @@ -0,0 +1,39 @@ +# PathHelpers + +## Purpose + +The `PathHelpers` software unit provides safe path construction utilities that +prevent path traversal attacks. It is used by the Index subsystem when constructing +file system paths to evidence PDF files referenced in the evidence index. + +## SafePathCombine() + +`PathHelpers.SafePathCombine(basePath, relativePath)` combines a trusted base path +with an untrusted relative path from the evidence index, validating that the result +does not escape the base directory. + +The algorithm is: + +```mermaid +graph TD + A[Receive basePath and relativePath] --> B{relativePath contains '..' segments?} + B -->|yes| C[Throw exception: path traversal rejected] + B -->|no| D{relativePath is rooted?} + D -->|yes| E[Throw exception: absolute path rejected] + D -->|no| F[Combine basePath and relativePath] + F --> G{Combined path starts with basePath?} + G -->|no| H[Throw exception: traversal detected after combination] + G -->|yes| I[Return combined path] +``` + +The double-check strategy (pre-validation of segments plus post-combination +verification) defends against edge cases such as URL-encoded separators or +platform-specific path normalization that might otherwise bypass a single check. + +## Security Rationale + +Evidence index files may be loaded from external sources (file shares or URLs). +The `file` field in each index record is supplied by the evidence store and must +be treated as untrusted input. Without path validation, a maliciously crafted +index could direct the tool to read or reference files outside the intended +evidence directory. `SafePathCombine` eliminates this attack surface. diff --git a/docs/design/program.md b/docs/design/program.md new file mode 100644 index 0000000..d018122 --- /dev/null +++ b/docs/design/program.md @@ -0,0 +1,54 @@ +# Program + +## Purpose + +The `Program` software unit is the main entry point of the ReviewMark tool. It is +responsible for constructing the execution context, dispatching to the appropriate +processing logic based on parsed flags, and returning a meaningful exit code to the +calling process. + +## Version Property + +`Program.Version` returns the tool version string. The version is embedded at build +time from the assembly metadata and follows semantic versioning conventions. + +## Main() Method + +`Program.Main(string[] args)` is the process entry point. It: + +1. Constructs a `Context` instance via `Context.Create(args)` inside a `using` block +2. Calls `Program.Run(Context)` to perform the requested operation +3. Returns `Context.ExitCode` as the process exit code + +Any unhandled exception that escapes `Run()` is caught, written to the error output +via `Context.WriteError()`, and causes a non-zero exit code to be returned. + +## Run() Dispatch Logic + +`Program.Run(Context)` evaluates the parsed flags in the following priority order, +executing the first matching action and returning: + +```mermaid +graph TD + A[Start] --> B{--version flag?} + B -->|yes| C[Print version and return] + B -->|no| D{--help flag?} + D -->|yes| E[Print banner and return] + D -->|no| F{--validate flag?} + F -->|yes| G[Run self-validation and return] + F -->|no| H{--lint flag?} + H -->|yes| I[Run configuration lint and return] + H -->|no| J{--index paths provided?} + J -->|yes| K[Scan and write evidence index, then return] + J -->|no| L[Generate Review Plan and/or Review Report] + L --> M[Return] +``` + +Only one top-level action is performed per invocation. Actions later in the priority +order are not reached if an earlier flag is set. + +## PrintBanner() + +`Program.PrintBanner(Context)` writes the help text to the console via +`Context.WriteLine()`. The banner lists all supported flags and arguments with brief +descriptions. diff --git a/docs/design/review-mark-configuration.md b/docs/design/review-mark-configuration.md new file mode 100644 index 0000000..449d678 --- /dev/null +++ b/docs/design/review-mark-configuration.md @@ -0,0 +1,80 @@ +# ReviewMarkConfiguration + +## Purpose + +The `ReviewMarkConfiguration` software unit is responsible for parsing the +`.reviewmark.yaml` configuration file and performing all review-set processing. +It coordinates file enumeration, fingerprint computation, evidence lookup, and +the generation of the Review Plan and Review Report compliance documents. + +## Configuration Model + +The `.reviewmark.yaml` file is deserialized into the following model: + +| Class | Description | +| ----- | ----------- | +| `ReviewMarkYaml` | Root configuration object containing the evidence source and review list | +| `EvidenceSourceYaml` | Describes how to locate the evidence index (`type`, `path` or `url`) | +| `ReviewYaml` | Describes a single review-set (`id`, `title`, file patterns) | + +## ReviewMarkConfiguration.Load() + +`ReviewMarkConfiguration.Load(definitionFile, workingDirectory)` reads and +deserializes the YAML file, resolves all glob patterns relative to the working +directory, computes fingerprints for each review-set, loads the evidence index, +and returns a fully initialized configuration object ready for plan/report generation. + +## Fingerprinting Algorithm + +The fingerprint for a review-set uniquely identifies the exact content of its file-set. +The algorithm is: + +```mermaid +graph TD + A[For each file in review-set sorted list] --> B[Read file contents] + B --> C[Compute SHA-256 hash of file contents] + C --> D[Collect all per-file hashes] + D --> E[Sort hashes lexicographically] + E --> F[Concatenate sorted hashes] + F --> G[Compute SHA-256 of concatenated hashes] + G --> H[Return as hex string fingerprint] +``` + +Sorting the per-file hashes before combining them ensures that the fingerprint is +sensitive to content changes but not to the order in which files happen to be +enumerated by the operating system. + +## Review Plan Generation + +The Review Plan is generated by `ReviewMarkConfiguration.WritePlan()`. It produces +a Markdown document that lists every file in the `needs-review` file-set and, for +each file, identifies which review-sets provide coverage. + +- The `--plan-depth` argument controls the heading level used for sections +- The `--elaborate` flag expands the file list for each review-set inline + +## Review Report Generation + +The Review Report is generated by `ReviewMarkConfiguration.WriteReport()`. It +produces a Markdown document that lists every review-set with its current status. + +For each review-set the report includes: + +- The review-set `id` and `title` +- The current fingerprint of the file-set +- The review status: `Current`, `Stale`, or `Missing` + +Status is determined by querying the loaded `ReviewIndex` via `GetStatus()`. + +- The `--report-depth` argument controls the heading level used for sections +- The `--elaborate` flag expands the list of files covered by each review-set + +## Linting + +`ReviewMarkConfiguration.Lint(Context)` validates the loaded configuration for +correctness. Lint checks include: + +- All review-set `id` values are unique +- All glob patterns resolve to at least one file +- The `needs-review` file-set is non-empty +- All files in the `needs-review` set are covered by at least one review-set diff --git a/docs/design/system.md b/docs/design/system.md new file mode 100644 index 0000000..f7d59c9 --- /dev/null +++ b/docs/design/system.md @@ -0,0 +1,72 @@ +# System Design + +This section describes the high-level behavior of the ReviewMark system and the workflow +that connects its subsystems. + +## Overview + +ReviewMark automates the evidence-gathering step of software review processes used in +regulated environments. On each CI/CD run, it determines which files are subject to +review, identifies the review evidence that covers them, and generates two compliance +documents: a Review Plan and a Review Report. + +## Main Workflow + +The following diagram illustrates the end-to-end processing flow. + +```mermaid +graph TD + A[Parse CLI arguments] --> B[Load .reviewmark.yaml] + B --> C[Resolve file lists via glob patterns] + C --> D[Compute SHA-256 fingerprints] + D --> E[Load evidence index] + E --> F{Evidence source type?} + F -->|none| G[Use empty index] + F -->|fileshare| H[Load index.json from file path] + F -->|url| I[Load index.json from HTTP URL] + G --> J[Generate Review Plan / Review Report] + H --> J + I --> J + J --> K{Enforce flag set?} + K -->|yes| L{All sets Current?} + L -->|no| M[Return non-zero exit code] + L -->|yes| N[Return success] + K -->|no| N +``` + +## Evidence Source Types + +ReviewMark supports three evidence source types, configured in `.reviewmark.yaml`: + +| Source Type | Description | +| ----------- | ----------- | +| `none` | No evidence store; all review-sets are treated as missing | +| `fileshare` | Evidence index loaded from a local or network file path | +| `url` | Evidence index loaded from an HTTP or HTTPS URL | + +## Output Documents + +### Review Plan + +The Review Plan lists every file that is subject to review and identifies which +review-sets provide coverage for each file. It is generated by the `--plan` flag +and written to a configurable output path. + +### Review Report + +The Review Report lists every review-set defined in the configuration, the current +fingerprint of its file-set, and the review status (Current, Stale, or Missing). +It is generated by the `--report` flag and written to a configurable output path. + +## Enforcement + +When the `--enforce` flag is set, ReviewMark returns a non-zero exit code if any +review-set does not have Current status. This allows CI/CD pipelines to fail builds +when review coverage is incomplete or out of date. + +## Index Management + +The `--index` flag causes ReviewMark to scan a directory for PDF evidence files and +write an `index.json` file suitable for use as a fileshare evidence source. This +supports workflows where review PDFs are stored alongside source code or on a +shared network location. diff --git a/docs/design/title.txt b/docs/design/title.txt new file mode 100644 index 0000000..d140ba3 --- /dev/null +++ b/docs/design/title.txt @@ -0,0 +1,13 @@ +--- +title: ReviewMark Design +subtitle: Software Design Document for ReviewMark +author: DEMA Consulting +description: Software Design Document for ReviewMark +lang: en-US +keywords: + - ReviewMark + - Design + - Software Architecture + - .NET + - Command-Line Tool +--- diff --git a/docs/design/validation.md b/docs/design/validation.md new file mode 100644 index 0000000..d3646c8 --- /dev/null +++ b/docs/design/validation.md @@ -0,0 +1,52 @@ +# Validation + +## Purpose + +The `Validation` software unit implements the self-validation framework for +ReviewMark. Self-validation allows the tool to verify its own correct operation +in a target environment, which is a requirement for regulated deployment contexts +where the tool itself is part of a qualified software chain. + +## Validation.Run() + +`Validation.Run(Context)` orchestrates all self-validation tests. It: + +1. Creates a test suite using the `DemaConsulting.TestResults` library +2. Executes each test case in sequence +3. Writes results to the configured output file (TRX or JUnit format) if `ResultsFile` is set +4. Writes a summary table and per-test results to the console via `Context.WriteLine()` +5. Sets `Context.ExitCode` to a non-zero value if any test fails + +## Test Output Format + +Results are written using the `DemaConsulting.TestResults` library, which supports +both TRX (Visual Studio Test Results) and JUnit XML output formats. The output format +is inferred from the file extension of `ResultsFile`. + +## Test Naming Convention + +All test names follow the pattern `ReviewMark_MethodOrScenario`, where +`MethodOrScenario` identifies the behavior under test. This convention ensures +test results are identifiable in test dashboards and traceability matrices. + +## Test Coverage + +The self-validation suite covers the following scenarios: + +| Test Name | Scenario | +| --------- | -------- | +| `ReviewMark_Version` | Tool correctly reports its version | +| `ReviewMark_Help` | Tool correctly displays help text | +| `ReviewMark_Plan` | Review Plan is generated correctly for a known configuration | +| `ReviewMark_Report` | Review Report is generated correctly for a known configuration | +| `ReviewMark_IndexScan` | Evidence index is created correctly by scanning a directory | +| `ReviewMark_Enforce` | Tool returns non-zero exit code when enforce mode detects uncovered sets | +| `ReviewMark_WorkingDirectory` | Working directory override resolves paths correctly | +| `ReviewMark_Elaborate` | Elaborate mode expands file lists in generated documents | +| `ReviewMark_Lint` | Lint mode detects configuration errors correctly | + +## Console Output + +In addition to the structured results file, `Validation.Run()` writes a human-readable +summary to the console. The summary includes a table of all tests with their pass/fail +status, followed by detailed output for any failing tests to aid diagnosis. From c4b360edd8e0dbdb75a1811fc182b71588d515db Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Sat, 28 Mar 2026 20:58:42 +0000 Subject: [PATCH 03/18] Address PR review feedback: add requirements files, ValidationTests, fix review-sets and diagram Agent-Logs-Url: https://github.com/demaconsulting/ReviewMark/sessions/11aefa92-cb5d-46a4-ad14-fd7c21807456 Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com> --- .reviewmark.yaml | 49 +++-- docs/design/introduction.md | 2 +- docs/reqstream/glob-matcher-requirements.yaml | 31 +++ docs/reqstream/path-helpers-requirements.yaml | 26 +++ .../reviewmark-system-requirements.yaml | 71 ++++++ docs/reqstream/validation-requirements.yaml | 34 +++ requirements.yaml | 4 + .../ValidationTests.cs | 202 ++++++++++++++++++ 8 files changed, 396 insertions(+), 23 deletions(-) create mode 100644 docs/reqstream/glob-matcher-requirements.yaml create mode 100644 docs/reqstream/path-helpers-requirements.yaml create mode 100644 docs/reqstream/reviewmark-system-requirements.yaml create mode 100644 docs/reqstream/validation-requirements.yaml create mode 100644 test/DemaConsulting.ReviewMark.Tests/ValidationTests.cs diff --git a/.reviewmark.yaml b/.reviewmark.yaml index 27bd070..4e9e386 100644 --- a/.reviewmark.yaml +++ b/.reviewmark.yaml @@ -40,9 +40,10 @@ reviews: - id: ReviewMark-GlobMatcher title: Review of GlobMatcher software unit (file pattern matching) paths: - - "docs/design/glob-matcher.md" # design - - "src/**/GlobMatcher.cs" # implementation - - "test/**/GlobMatcherTests.cs" # tests + - "docs/reqstream/glob-matcher-requirements.yaml" # requirements + - "docs/design/glob-matcher.md" # design + - "src/**/GlobMatcher.cs" # implementation + - "test/**/GlobMatcherTests.cs" # tests - id: ReviewMark-Index title: Review of Index software unit (review evidence indexing) @@ -55,9 +56,10 @@ reviews: - id: ReviewMark-PathHelpers title: Review of PathHelpers software unit (file path utilities) paths: - - "docs/design/path-helpers.md" # design - - "src/**/PathHelpers.cs" # implementation - - "test/**/PathHelpersTests.cs" # tests + - "docs/reqstream/path-helpers-requirements.yaml" # requirements + - "docs/design/path-helpers.md" # design + - "src/**/PathHelpers.cs" # implementation + - "test/**/PathHelpersTests.cs" # tests - id: ReviewMark-Program title: Review of Program software unit (main entry point and tool orchestration) @@ -68,10 +70,7 @@ reviews: - "docs/guide/guide.md" # user guide - "src/**/Program.cs" # implementation - "test/**/ProgramTests.cs" # unit tests - - "test/**/IntegrationTests.cs" # integration tests - - "test/**/Runner.cs" # test infrastructure - "test/**/TestDirectory.cs" # test infrastructure - - "test/**/AssemblyInfo.cs" # test infrastructure - id: ReviewMark-Configuration title: Review of ReviewMarkConfiguration software unit (configuration parsing and processing) @@ -84,24 +83,26 @@ reviews: - id: ReviewMark-Validation title: Review of Validation software unit (self-validation test execution) paths: - - "docs/reqstream/ots-requirements.yaml" # OTS requirements verified by self-validation + - "docs/reqstream/validation-requirements.yaml" # requirements - "docs/design/validation.md" # design - "src/**/Validation.cs" # implementation + - "test/**/ValidationTests.cs" # tests # Special review-sets - id: ReviewMark-System title: Review of ReviewMark system-level behavior, platform support, and integration paths: - - "requirements.yaml" # system requirements (root) - - "docs/reqstream/platform-requirements.yaml" # platform requirements - - "docs/design/introduction.md" # design introduction and architecture - - "docs/design/system.md" # system design - - "test/**/IntegrationTests.cs" # integration tests + - "docs/reqstream/reviewmark-system-requirements.yaml" # system requirements + - "docs/reqstream/platform-requirements.yaml" # platform requirements + - "docs/design/introduction.md" # design introduction and architecture + - "docs/design/system.md" # system design + - "test/**/IntegrationTests.cs" # integration tests + - "test/**/Runner.cs" # test infrastructure + - "test/**/AssemblyInfo.cs" # test infrastructure - id: ReviewMark-Design title: Review of all ReviewMark design documentation paths: - - "requirements.yaml" # system requirements (root) - "docs/reqstream/platform-requirements.yaml" # platform requirements - "docs/design/introduction.md" # design introduction and architecture - "docs/design/system.md" # system design @@ -116,9 +117,13 @@ reviews: - id: ReviewMark-AllRequirements title: Review of all ReviewMark requirements files paths: - - "requirements.yaml" # root requirements file - - "docs/reqstream/cli-requirements.yaml" # CLI subsystem requirements - - "docs/reqstream/configuration-requirements.yaml" # Configuration subsystem requirements - - "docs/reqstream/index-requirements.yaml" # Index subsystem requirements - - "docs/reqstream/platform-requirements.yaml" # Platform support requirements - - "docs/reqstream/ots-requirements.yaml" # OTS component requirements + - "requirements.yaml" # root requirements file + - "docs/reqstream/reviewmark-system-requirements.yaml" # system-level requirements + - "docs/reqstream/cli-requirements.yaml" # CLI subsystem requirements + - "docs/reqstream/configuration-requirements.yaml" # Configuration subsystem requirements + - "docs/reqstream/glob-matcher-requirements.yaml" # GlobMatcher unit requirements + - "docs/reqstream/index-requirements.yaml" # Index subsystem requirements + - "docs/reqstream/path-helpers-requirements.yaml" # PathHelpers unit requirements + - "docs/reqstream/platform-requirements.yaml" # Platform support requirements + - "docs/reqstream/validation-requirements.yaml" # Validation unit requirements + - "docs/reqstream/ots-requirements.yaml" # OTS component requirements diff --git a/docs/design/introduction.md b/docs/design/introduction.md index a2a3bed..f3e1377 100644 --- a/docs/design/introduction.md +++ b/docs/design/introduction.md @@ -34,7 +34,7 @@ The following diagram shows the decomposition of the ReviewMark software system subsystems and software units. ```mermaid -graph TD +graph LR System["ReviewMark (Software System)"] CLI["CLI Subsystem"] diff --git a/docs/reqstream/glob-matcher-requirements.yaml b/docs/reqstream/glob-matcher-requirements.yaml new file mode 100644 index 0000000..74bd142 --- /dev/null +++ b/docs/reqstream/glob-matcher-requirements.yaml @@ -0,0 +1,31 @@ +--- +# GlobMatcher Software Unit Requirements +# +# PURPOSE: +# - Define requirements for the GlobMatcher software unit +# - This unit resolves ordered include/exclude glob patterns to a list of files +# - It is used by ReviewMarkConfiguration to resolve needs-review and review-set file lists + +sections: + - title: GlobMatcher Unit Requirements + requirements: + - id: ReviewMark-GlobMatcher-IncludeExclude + title: >- + The GlobMatcher shall resolve ordered include and exclude glob patterns to a sorted list of + relative file paths. + justification: | + Review-set and needs-review configurations specify files using ordered glob patterns, + where patterns prefixed with '!' are exclusions. The GlobMatcher must apply these + patterns in declaration order so that a later include can re-add files removed by an + earlier exclude, and vice versa. The result must be sorted to ensure deterministic + fingerprinting regardless of filesystem iteration order. + tests: + - GlobMatcher_GetMatchingFiles_SingleIncludePattern_ReturnsMatchingFiles + - GlobMatcher_GetMatchingFiles_ExcludePattern_RemovesMatchingFiles + - GlobMatcher_GetMatchingFiles_MultiplePatterns_AppliesInOrder + - GlobMatcher_GetMatchingFiles_NullBaseDirectory_ThrowsArgumentNullException + - GlobMatcher_GetMatchingFiles_EmptyBaseDirectory_ThrowsArgumentException + - GlobMatcher_GetMatchingFiles_NullPatterns_ThrowsArgumentNullException + - GlobMatcher_GetMatchingFiles_NoMatchingFiles_ReturnsEmptyList + - GlobMatcher_GetMatchingFiles_ResultsAreSorted + - GlobMatcher_GetMatchingFiles_ResultsUseForwardSlashes diff --git a/docs/reqstream/path-helpers-requirements.yaml b/docs/reqstream/path-helpers-requirements.yaml new file mode 100644 index 0000000..da3cafb --- /dev/null +++ b/docs/reqstream/path-helpers-requirements.yaml @@ -0,0 +1,26 @@ +--- +# PathHelpers Software Unit Requirements +# +# PURPOSE: +# - Define requirements for the PathHelpers software unit +# - This unit provides safe path operations that prevent path traversal attacks +# - It is used by Index.cs and Validation.cs when constructing file paths + +sections: + - title: PathHelpers Unit Requirements + requirements: + - id: ReviewMark-PathHelpers-SafeCombine + title: The PathHelpers shall safely combine a base path and a relative path, rejecting path traversal attempts. + justification: | + When constructing file paths from user-supplied or externally-sourced components + (such as relative paths read from an evidence index), the tool must prevent path + traversal attacks. SafePathCombine validates that the relative path does not + contain '..' sequences or absolute path components, and performs a defense-in-depth + check that the resolved combined path remains under the base directory. + tests: + - PathHelpers_SafePathCombine_ValidPaths_ReturnsCombinedPath + - PathHelpers_SafePathCombine_NullBasePath_ThrowsArgumentNullException + - PathHelpers_SafePathCombine_NullRelativePath_ThrowsArgumentNullException + - PathHelpers_SafePathCombine_PathTraversal_ThrowsArgumentException + - PathHelpers_SafePathCombine_AbsoluteRelativePath_ThrowsArgumentException + - PathHelpers_SafePathCombine_NestedPath_ReturnsCombinedPath diff --git a/docs/reqstream/reviewmark-system-requirements.yaml b/docs/reqstream/reviewmark-system-requirements.yaml new file mode 100644 index 0000000..807b362 --- /dev/null +++ b/docs/reqstream/reviewmark-system-requirements.yaml @@ -0,0 +1,71 @@ +--- +# ReviewMark System-Level Requirements +# +# PURPOSE: +# - Define system-level requirements describing what end-users need the ReviewMark tool to provide +# - These requirements capture the externally visible behavior of the complete ReviewMark system +# - Unit-level requirements (per-class behavior) are in the individual *-requirements.yaml files + +sections: + - title: System-Level Requirements + requirements: + - id: ReviewMark-System-ReviewPlan + title: >- + The tool shall generate a Review Plan document listing all files requiring review and their + review-set coverage. + justification: | + In regulated environments, auditors require evidence that every file subject to review + is covered by at least one named review-set. The Review Plan document provides this + evidence automatically on each CI/CD run, replacing manual tracking spreadsheets. + tests: + - ReviewMark_ReviewPlanGeneration + - IntegrationTest_PlanFlag_GeneratesReviewPlan + + - id: ReviewMark-System-ReviewReport + title: The tool shall generate a Review Report document listing every review-set and its current review status. + justification: | + Auditors need evidence that the review evidence for each review-set is current — + that the reviewed files have not changed since the review was conducted. The Review + Report provides this evidence automatically, showing Current, Stale, or Missing + status for each review-set. + tests: + - ReviewMark_ReviewReportGeneration + - IntegrationTest_ReportFlag_GeneratesReviewReport + + - id: ReviewMark-System-Enforce + title: The tool shall return a non-zero exit code when enforcement is enabled and any review-set is not current. + justification: | + CI/CD pipelines must be able to gate releases on review coverage. The --enforce flag + enables this by causing the tool to exit with a non-zero code when any review-set has + Stale or Missing status, making incomplete review coverage a build-breaking condition. + tests: + - ReviewMark_Enforce + - IntegrationTest_EnforceFlag_FailsWhenReviewStale + + - id: ReviewMark-System-IndexScan + title: The tool shall scan PDF evidence files and write an index.json when the --index flag is provided. + justification: | + Review evidence PDFs contain embedded metadata (id, fingerprint, date, result) in their + Keywords field. The --index command scans a directory of such PDFs and writes an + index.json, enabling the evidence store to be refreshed after new review PDFs are added + without manual maintenance of the index file. + tests: + - ReviewMark_IndexScan + - IntegrationTest_IndexFlag_ScansEvidence + + - id: ReviewMark-System-Validate + title: The tool shall execute self-validation tests when the --validate flag is provided. + justification: | + Regulated environments require tool qualification evidence to demonstrate that the tool + functions correctly in its specific deployment environment. The --validate flag triggers + a built-in test suite that exercises core tool behaviors and produces a pass/fail report. + tests: + - ReviewMark_VersionDisplay + - ReviewMark_HelpDisplay + - ReviewMark_ReviewPlanGeneration + - ReviewMark_ReviewReportGeneration + - ReviewMark_IndexScan + - ReviewMark_Enforce + - ReviewMark_WorkingDirectoryOverride + - ReviewMark_Elaborate + - ReviewMark_Lint diff --git a/docs/reqstream/validation-requirements.yaml b/docs/reqstream/validation-requirements.yaml new file mode 100644 index 0000000..622ccd1 --- /dev/null +++ b/docs/reqstream/validation-requirements.yaml @@ -0,0 +1,34 @@ +--- +# Validation Software Unit Requirements +# +# PURPOSE: +# - Define requirements for the Validation software unit +# - This unit provides self-validation test execution for regulated environments +# - Self-validation proves the tool is functioning correctly in its deployment environment + +sections: + - title: Validation Unit Requirements + requirements: + - id: ReviewMark-Validation-Run + title: The tool shall execute self-validation tests and report results when the --validate flag is provided. + justification: | + In regulated environments, tool qualification evidence is required to demonstrate + that the tool functions correctly in its deployment environment. Self-validation + runs a suite of functional tests covering core behaviors and reports pass/fail + results with a summary count, giving quality assurance teams the evidence they need. + tests: + - Validation_Run_NullContext_ThrowsArgumentNullException + - Validation_Run_WritesValidationHeader + - Validation_Run_WritesSummaryWithTotalTests + - Validation_Run_AllTestsPass_ExitCodeIsZero + + - id: ReviewMark-Validation-ResultsFile + title: The tool shall write self-validation results to a TRX or JUnit XML file when --results is provided. + justification: | + CI/CD pipelines and requirements traceability tools (such as ReqStream) consume + test result files in standard formats. By supporting both TRX (MSTest) and JUnit + XML output, the self-validation results can be fed directly into pipeline tooling + without additional conversion steps. + tests: + - Validation_Run_WithTrxResultsFile_WritesFile + - Validation_Run_WithXmlResultsFile_WritesFile diff --git a/requirements.yaml b/requirements.yaml index 3654661..395bbf7 100644 --- a/requirements.yaml +++ b/requirements.yaml @@ -24,8 +24,12 @@ # --- includes: + - docs/reqstream/reviewmark-system-requirements.yaml - docs/reqstream/cli-requirements.yaml - docs/reqstream/configuration-requirements.yaml + - docs/reqstream/glob-matcher-requirements.yaml - docs/reqstream/index-requirements.yaml + - docs/reqstream/path-helpers-requirements.yaml - docs/reqstream/platform-requirements.yaml + - docs/reqstream/validation-requirements.yaml - docs/reqstream/ots-requirements.yaml diff --git a/test/DemaConsulting.ReviewMark.Tests/ValidationTests.cs b/test/DemaConsulting.ReviewMark.Tests/ValidationTests.cs new file mode 100644 index 0000000..d7d9e03 --- /dev/null +++ b/test/DemaConsulting.ReviewMark.Tests/ValidationTests.cs @@ -0,0 +1,202 @@ +// Copyright (c) DEMA Consulting +// +// Permission is hereby granted, free of charge, to any person obtaining a copy +// of this software and associated documentation files (the "Software"), to deal +// in the Software without restriction, including without limitation the rights +// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +// copies of the Software, and to permit persons to whom the Software is +// furnished to do so, subject to the following conditions: +// +// The above copyright notice and this permission notice shall be included in all +// copies or substantial portions of the Software. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +// SOFTWARE. + +namespace DemaConsulting.ReviewMark.Tests; + +/// +/// Unit tests for the class. +/// +[TestClass] +public class ValidationTests +{ + /// + /// Test that Run throws ArgumentNullException when context is null. + /// + [TestMethod] + public void Validation_Run_NullContext_ThrowsArgumentNullException() + { + // Act & Assert + Assert.Throws(() => Validation.Run(null!)); + } + + /// + /// Test that Run writes a validation header containing system information. + /// + [TestMethod] + public void Validation_Run_WritesValidationHeader() + { + // Arrange + var originalOut = Console.Out; + try + { + using var outWriter = new StringWriter(); + Console.SetOut(outWriter); + using var context = Context.Create(["--validate"]); + + // Act + Validation.Run(context); + + // Assert — output contains the markdown header and table headings + var output = outWriter.ToString(); + Assert.Contains("DEMA Consulting ReviewMark", output); + Assert.Contains("Tool Version", output); + Assert.Contains("Machine Name", output); + } + finally + { + Console.SetOut(originalOut); + } + } + + /// + /// Test that Run writes a summary with a total test count. + /// + [TestMethod] + public void Validation_Run_WritesSummaryWithTotalTests() + { + // Arrange + var originalOut = Console.Out; + try + { + using var outWriter = new StringWriter(); + Console.SetOut(outWriter); + using var context = Context.Create(["--validate"]); + + // Act + Validation.Run(context); + + // Assert — output contains the summary section + var output = outWriter.ToString(); + Assert.Contains("Total Tests:", output); + Assert.Contains("Passed:", output); + Assert.Contains("Failed:", output); + } + finally + { + Console.SetOut(originalOut); + } + } + + /// + /// Test that Run returns a zero exit code when all tests pass. + /// + [TestMethod] + public void Validation_Run_AllTestsPass_ExitCodeIsZero() + { + // Arrange + var originalOut = Console.Out; + try + { + using var outWriter = new StringWriter(); + Console.SetOut(outWriter); + using var context = Context.Create(["--validate"]); + + // Act + Validation.Run(context); + + // Assert — exit code is zero (no errors) + Assert.AreEqual(0, context.ExitCode); + } + finally + { + Console.SetOut(originalOut); + } + } + + /// + /// Test that Run writes results to a TRX file when --results is provided with a .trx extension. + /// + [TestMethod] + public void Validation_Run_WithTrxResultsFile_WritesFile() + { + // Arrange + var resultsFile = Path.Combine(Path.GetTempPath(), $"reviewmark-validation-{Guid.NewGuid()}.trx"); + try + { + var originalOut = Console.Out; + try + { + using var outWriter = new StringWriter(); + Console.SetOut(outWriter); + using var context = Context.Create(["--validate", "--results", resultsFile]); + + // Act + Validation.Run(context); + + // Assert — results file exists and has content + Assert.IsTrue(File.Exists(resultsFile), "TRX results file was not created"); + var content = File.ReadAllText(resultsFile); + Assert.IsFalse(string.IsNullOrWhiteSpace(content), "TRX results file is empty"); + Assert.Contains("TestRun", content); + } + finally + { + Console.SetOut(originalOut); + } + } + finally + { + if (File.Exists(resultsFile)) + { + File.Delete(resultsFile); + } + } + } + + /// + /// Test that Run writes results to a JUnit XML file when --results is provided with a .xml extension. + /// + [TestMethod] + public void Validation_Run_WithXmlResultsFile_WritesFile() + { + // Arrange + var resultsFile = Path.Combine(Path.GetTempPath(), $"reviewmark-validation-{Guid.NewGuid()}.xml"); + try + { + var originalOut = Console.Out; + try + { + using var outWriter = new StringWriter(); + Console.SetOut(outWriter); + using var context = Context.Create(["--validate", "--results", resultsFile]); + + // Act + Validation.Run(context); + + // Assert — results file exists and has content + Assert.IsTrue(File.Exists(resultsFile), "XML results file was not created"); + var content = File.ReadAllText(resultsFile); + Assert.IsFalse(string.IsNullOrWhiteSpace(content), "XML results file is empty"); + Assert.Contains("testsuites", content); + } + finally + { + Console.SetOut(originalOut); + } + } + finally + { + if (File.Exists(resultsFile)) + { + File.Delete(resultsFile); + } + } + } +} From f9b92298b9d48057a2e10ec3d82750c4b85cf503 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Sat, 28 Mar 2026 21:12:10 +0000 Subject: [PATCH 04/18] Rename requirements files per convention and split OTS into per-component files Agent-Logs-Url: https://github.com/demaconsulting/ReviewMark/sessions/041a3a1a-f5b2-4bec-b0d4-135c272431e2 Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com> --- .reviewmark.yaml | 59 +++++----- docs/reqstream/ots-buildmark.yaml | 20 ++++ docs/reqstream/ots-mstest.yaml | 28 +++++ docs/reqstream/ots-reqstream.yaml | 21 ++++ docs/reqstream/ots-requirements.yaml | 102 ------------------ docs/reqstream/ots-sarifmark.yaml | 21 ++++ docs/reqstream/ots-sonarmark.yaml | 23 ++++ docs/reqstream/ots-versionmark.yaml | 21 ++++ ...quirements.yaml => reviewmark-system.yaml} | 0 ...i-requirements.yaml => subsystem-cli.yaml} | 0 ...ents.yaml => subsystem-configuration.yaml} | 0 ...requirements.yaml => subsystem-index.yaml} | 0 ...quirements.yaml => unit-glob-matcher.yaml} | 0 ...quirements.yaml => unit-path-helpers.yaml} | 0 ...requirements.yaml => unit-validation.yaml} | 0 requirements.yaml | 21 ++-- 16 files changed, 179 insertions(+), 137 deletions(-) create mode 100644 docs/reqstream/ots-buildmark.yaml create mode 100644 docs/reqstream/ots-mstest.yaml create mode 100644 docs/reqstream/ots-reqstream.yaml delete mode 100644 docs/reqstream/ots-requirements.yaml create mode 100644 docs/reqstream/ots-sarifmark.yaml create mode 100644 docs/reqstream/ots-sonarmark.yaml create mode 100644 docs/reqstream/ots-versionmark.yaml rename docs/reqstream/{reviewmark-system-requirements.yaml => reviewmark-system.yaml} (100%) rename docs/reqstream/{cli-requirements.yaml => subsystem-cli.yaml} (100%) rename docs/reqstream/{configuration-requirements.yaml => subsystem-configuration.yaml} (100%) rename docs/reqstream/{index-requirements.yaml => subsystem-index.yaml} (100%) rename docs/reqstream/{glob-matcher-requirements.yaml => unit-glob-matcher.yaml} (100%) rename docs/reqstream/{path-helpers-requirements.yaml => unit-path-helpers.yaml} (100%) rename docs/reqstream/{validation-requirements.yaml => unit-validation.yaml} (100%) diff --git a/.reviewmark.yaml b/.reviewmark.yaml index 4e9e386..b086516 100644 --- a/.reviewmark.yaml +++ b/.reviewmark.yaml @@ -32,7 +32,7 @@ reviews: - id: ReviewMark-Context title: Review of Context software unit (command-line argument handling) paths: - - "docs/reqstream/cli-requirements.yaml" # requirements + - "docs/reqstream/subsystem-cli.yaml" # requirements - "docs/design/context.md" # design - "src/**/Context.cs" # implementation - "test/**/ContextTests.cs" # tests @@ -40,7 +40,7 @@ reviews: - id: ReviewMark-GlobMatcher title: Review of GlobMatcher software unit (file pattern matching) paths: - - "docs/reqstream/glob-matcher-requirements.yaml" # requirements + - "docs/reqstream/unit-glob-matcher.yaml" # requirements - "docs/design/glob-matcher.md" # design - "src/**/GlobMatcher.cs" # implementation - "test/**/GlobMatcherTests.cs" # tests @@ -48,7 +48,7 @@ reviews: - id: ReviewMark-Index title: Review of Index software unit (review evidence indexing) paths: - - "docs/reqstream/index-requirements.yaml" # requirements + - "docs/reqstream/subsystem-index.yaml" # requirements - "docs/design/index.md" # design - "src/**/Index.cs" # implementation - "test/**/IndexTests.cs" # tests @@ -56,7 +56,7 @@ reviews: - id: ReviewMark-PathHelpers title: Review of PathHelpers software unit (file path utilities) paths: - - "docs/reqstream/path-helpers-requirements.yaml" # requirements + - "docs/reqstream/unit-path-helpers.yaml" # requirements - "docs/design/path-helpers.md" # design - "src/**/PathHelpers.cs" # implementation - "test/**/PathHelpersTests.cs" # tests @@ -64,7 +64,7 @@ reviews: - id: ReviewMark-Program title: Review of Program software unit (main entry point and tool orchestration) paths: - - "docs/reqstream/cli-requirements.yaml" # requirements + - "docs/reqstream/subsystem-cli.yaml" # requirements - "docs/reqstream/platform-requirements.yaml" # platform requirements - "docs/design/program.md" # design - "docs/guide/guide.md" # user guide @@ -75,15 +75,15 @@ reviews: - id: ReviewMark-Configuration title: Review of ReviewMarkConfiguration software unit (configuration parsing and processing) paths: - - "docs/reqstream/configuration-requirements.yaml" # requirements - - "docs/design/review-mark-configuration.md" # design - - "src/**/ReviewMarkConfiguration.cs" # implementation - - "test/**/ReviewMarkConfigurationTests.cs" # tests + - "docs/reqstream/subsystem-configuration.yaml" # requirements + - "docs/design/review-mark-configuration.md" # design + - "src/**/ReviewMarkConfiguration.cs" # implementation + - "test/**/ReviewMarkConfigurationTests.cs" # tests - id: ReviewMark-Validation title: Review of Validation software unit (self-validation test execution) paths: - - "docs/reqstream/validation-requirements.yaml" # requirements + - "docs/reqstream/unit-validation.yaml" # requirements - "docs/design/validation.md" # design - "src/**/Validation.cs" # implementation - "test/**/ValidationTests.cs" # tests @@ -92,13 +92,13 @@ reviews: - id: ReviewMark-System title: Review of ReviewMark system-level behavior, platform support, and integration paths: - - "docs/reqstream/reviewmark-system-requirements.yaml" # system requirements - - "docs/reqstream/platform-requirements.yaml" # platform requirements - - "docs/design/introduction.md" # design introduction and architecture - - "docs/design/system.md" # system design - - "test/**/IntegrationTests.cs" # integration tests - - "test/**/Runner.cs" # test infrastructure - - "test/**/AssemblyInfo.cs" # test infrastructure + - "docs/reqstream/reviewmark-system.yaml" # system requirements + - "docs/reqstream/platform-requirements.yaml" # platform requirements + - "docs/design/introduction.md" # design introduction and architecture + - "docs/design/system.md" # system design + - "test/**/IntegrationTests.cs" # integration tests + - "test/**/Runner.cs" # test infrastructure + - "test/**/AssemblyInfo.cs" # test infrastructure - id: ReviewMark-Design title: Review of all ReviewMark design documentation @@ -117,13 +117,18 @@ reviews: - id: ReviewMark-AllRequirements title: Review of all ReviewMark requirements files paths: - - "requirements.yaml" # root requirements file - - "docs/reqstream/reviewmark-system-requirements.yaml" # system-level requirements - - "docs/reqstream/cli-requirements.yaml" # CLI subsystem requirements - - "docs/reqstream/configuration-requirements.yaml" # Configuration subsystem requirements - - "docs/reqstream/glob-matcher-requirements.yaml" # GlobMatcher unit requirements - - "docs/reqstream/index-requirements.yaml" # Index subsystem requirements - - "docs/reqstream/path-helpers-requirements.yaml" # PathHelpers unit requirements - - "docs/reqstream/platform-requirements.yaml" # Platform support requirements - - "docs/reqstream/validation-requirements.yaml" # Validation unit requirements - - "docs/reqstream/ots-requirements.yaml" # OTS component requirements + - "requirements.yaml" # root requirements file + - "docs/reqstream/reviewmark-system.yaml" # system-level requirements + - "docs/reqstream/subsystem-cli.yaml" # CLI subsystem requirements + - "docs/reqstream/subsystem-configuration.yaml" # Configuration subsystem requirements + - "docs/reqstream/subsystem-index.yaml" # Index subsystem requirements + - "docs/reqstream/unit-glob-matcher.yaml" # GlobMatcher unit requirements + - "docs/reqstream/unit-path-helpers.yaml" # PathHelpers unit requirements + - "docs/reqstream/unit-validation.yaml" # Validation unit requirements + - "docs/reqstream/platform-requirements.yaml" # Platform support requirements + - "docs/reqstream/ots-mstest.yaml" # MSTest OTS requirements + - "docs/reqstream/ots-reqstream.yaml" # ReqStream OTS requirements + - "docs/reqstream/ots-buildmark.yaml" # BuildMark OTS requirements + - "docs/reqstream/ots-versionmark.yaml" # VersionMark OTS requirements + - "docs/reqstream/ots-sarifmark.yaml" # SarifMark OTS requirements + - "docs/reqstream/ots-sonarmark.yaml" # SonarMark OTS requirements diff --git a/docs/reqstream/ots-buildmark.yaml b/docs/reqstream/ots-buildmark.yaml new file mode 100644 index 0000000..d59a4a7 --- /dev/null +++ b/docs/reqstream/ots-buildmark.yaml @@ -0,0 +1,20 @@ +--- +# BuildMark OTS Requirements +# +# PURPOSE: +# - Define requirements for the BuildMark off-the-shelf documentation generation tool +# - BuildMark generates build-notes documentation from GitHub Actions metadata + +sections: + - title: BuildMark OTS Requirements + requirements: + - id: ReviewMark-OTS-BuildMark + title: BuildMark shall generate build-notes documentation from GitHub Actions metadata. + justification: | + DemaConsulting.BuildMark queries the GitHub API to capture workflow run details and + renders them as a markdown build-notes document included in the release artifacts. + It runs as part of the same CI pipeline that produces the TRX test results, so a + successful pipeline run is evidence that BuildMark executed without error. + tags: [ots] + tests: + - BuildMark_MarkdownReportGeneration diff --git a/docs/reqstream/ots-mstest.yaml b/docs/reqstream/ots-mstest.yaml new file mode 100644 index 0000000..98dd61a --- /dev/null +++ b/docs/reqstream/ots-mstest.yaml @@ -0,0 +1,28 @@ +--- +# MSTest OTS Requirements +# +# PURPOSE: +# - Define requirements for the MSTest off-the-shelf testing framework +# - MSTest is used to discover, execute, and report unit test results + +sections: + - title: MSTest OTS Requirements + requirements: + - id: ReviewMark-OTS-MSTest + title: MSTest shall execute unit tests and report results. + justification: | + MSTest (MSTest.TestFramework and MSTest.TestAdapter) is the unit-testing framework used + by the project. It discovers and runs all test methods and writes TRX result files that + feed into coverage reporting and requirements traceability. Passing tests confirm the + framework is functioning correctly. + tags: [ots] + tests: + - Context_Create_NoArguments_ReturnsDefaultContext + - Context_Create_VersionFlag_SetsVersionTrue + - Context_Create_HelpFlag_SetsHelpTrue + - Context_Create_SilentFlag_SetsSilentTrue + - Context_Create_ValidateFlag_SetsValidateTrue + - Context_Create_ResultsFlag_SetsResultsFile + - Context_Create_LogFlag_OpensLogFile + - Context_Create_UnknownArgument_ThrowsArgumentException + - Context_Create_ShortVersionFlag_SetsVersionTrue diff --git a/docs/reqstream/ots-reqstream.yaml b/docs/reqstream/ots-reqstream.yaml new file mode 100644 index 0000000..908a75f --- /dev/null +++ b/docs/reqstream/ots-reqstream.yaml @@ -0,0 +1,21 @@ +--- +# ReqStream OTS Requirements +# +# PURPOSE: +# - Define requirements for the ReqStream off-the-shelf requirements traceability tool +# - ReqStream validates that every requirement is linked to passing test evidence + +sections: + - title: ReqStream OTS Requirements + requirements: + - id: ReviewMark-OTS-ReqStream + title: ReqStream shall enforce that every requirement is linked to passing test evidence. + justification: | + DemaConsulting.ReqStream processes requirements.yaml and the TRX test-result files to + produce a requirements report, justifications document, and traceability matrix. When + run with --enforce, it exits with a non-zero code if any requirement lacks test evidence, + making unproven requirements a build-breaking condition. A successful pipeline run with + --enforce proves all requirements are covered and that ReqStream is functioning. + tags: [ots] + tests: + - ReqStream_EnforcementMode diff --git a/docs/reqstream/ots-requirements.yaml b/docs/reqstream/ots-requirements.yaml deleted file mode 100644 index b763998..0000000 --- a/docs/reqstream/ots-requirements.yaml +++ /dev/null @@ -1,102 +0,0 @@ ---- -# OTS (Off-the-Shelf) Software Requirements -# -# PURPOSE: -# - Define requirements for third-party components used by ReviewMark -# - OTS requirements document which capabilities the project depends on -# - Tests verify the OTS component provides the required behavior in this environment - -sections: - - title: OTS Software Requirements - sections: - - title: MSTest - requirements: - - id: ReviewMark-OTS-MSTest - title: MSTest shall execute unit tests and report results. - justification: | - MSTest (MSTest.TestFramework and MSTest.TestAdapter) is the unit-testing framework used - by the project. It discovers and runs all test methods and writes TRX result files that - feed into coverage reporting and requirements traceability. Passing tests confirm the - framework is functioning correctly. - tags: [ots] - tests: - - Context_Create_NoArguments_ReturnsDefaultContext - - Context_Create_VersionFlag_SetsVersionTrue - - Context_Create_HelpFlag_SetsHelpTrue - - Context_Create_SilentFlag_SetsSilentTrue - - Context_Create_ValidateFlag_SetsValidateTrue - - Context_Create_ResultsFlag_SetsResultsFile - - Context_Create_LogFlag_OpensLogFile - - Context_Create_UnknownArgument_ThrowsArgumentException - - Context_Create_ShortVersionFlag_SetsVersionTrue - - - title: ReqStream - requirements: - - id: ReviewMark-OTS-ReqStream - title: ReqStream shall enforce that every requirement is linked to passing test evidence. - justification: | - DemaConsulting.ReqStream processes requirements.yaml and the TRX test-result files to - produce a requirements report, justifications document, and traceability matrix. When - run with --enforce, it exits with a non-zero code if any requirement lacks test evidence, - making unproven requirements a build-breaking condition. A successful pipeline run with - --enforce proves all requirements are covered and that ReqStream is functioning. - tags: [ots] - tests: - - ReqStream_EnforcementMode - - - title: BuildMark - requirements: - - id: ReviewMark-OTS-BuildMark - title: BuildMark shall generate build-notes documentation from GitHub Actions metadata. - justification: | - DemaConsulting.BuildMark queries the GitHub API to capture workflow run details and - renders them as a markdown build-notes document included in the release artifacts. - It runs as part of the same CI pipeline that produces the TRX test results, so a - successful pipeline run is evidence that BuildMark executed without error. - tags: [ots] - tests: - - BuildMark_MarkdownReportGeneration - - - title: VersionMark - requirements: - - id: ReviewMark-OTS-VersionMark - title: VersionMark shall publish captured tool-version information. - justification: | - DemaConsulting.VersionMark reads version metadata for each dotnet tool used in the - pipeline and writes a versions markdown document included in the release artifacts. - It runs in the same CI pipeline that produces the TRX test results, so a successful - pipeline run is evidence that VersionMark executed without error. - tags: [ots] - tests: - - VersionMark_CapturesVersions - - VersionMark_GeneratesMarkdownReport - - - title: SarifMark - requirements: - - id: ReviewMark-OTS-SarifMark - title: SarifMark shall convert CodeQL SARIF results into a markdown report. - justification: | - DemaConsulting.SarifMark reads the SARIF output produced by CodeQL code scanning and - renders it as a human-readable markdown document included in the release artifacts. - It runs in the same CI pipeline that produces the TRX test results, so a successful - pipeline run is evidence that SarifMark executed without error. - tags: [ots] - tests: - - SarifMark_SarifReading - - SarifMark_MarkdownReportGeneration - - - title: SonarMark - requirements: - - id: ReviewMark-OTS-SonarMark - title: SonarMark shall generate a SonarCloud quality report. - justification: | - DemaConsulting.SonarMark retrieves quality-gate and metrics data from SonarCloud and - renders it as a markdown document included in the release artifacts. It runs in the - same CI pipeline that produces the TRX test results, so a successful pipeline run is - evidence that SonarMark executed without error. - tags: [ots] - tests: - - SonarMark_QualityGateRetrieval - - SonarMark_IssuesRetrieval - - SonarMark_HotSpotsRetrieval - - SonarMark_MarkdownReportGeneration diff --git a/docs/reqstream/ots-sarifmark.yaml b/docs/reqstream/ots-sarifmark.yaml new file mode 100644 index 0000000..c49a525 --- /dev/null +++ b/docs/reqstream/ots-sarifmark.yaml @@ -0,0 +1,21 @@ +--- +# SarifMark OTS Requirements +# +# PURPOSE: +# - Define requirements for the SarifMark off-the-shelf SARIF reporting tool +# - SarifMark converts CodeQL SARIF results into a human-readable markdown report + +sections: + - title: SarifMark OTS Requirements + requirements: + - id: ReviewMark-OTS-SarifMark + title: SarifMark shall convert CodeQL SARIF results into a markdown report. + justification: | + DemaConsulting.SarifMark reads the SARIF output produced by CodeQL code scanning and + renders it as a human-readable markdown document included in the release artifacts. + It runs in the same CI pipeline that produces the TRX test results, so a successful + pipeline run is evidence that SarifMark executed without error. + tags: [ots] + tests: + - SarifMark_SarifReading + - SarifMark_MarkdownReportGeneration diff --git a/docs/reqstream/ots-sonarmark.yaml b/docs/reqstream/ots-sonarmark.yaml new file mode 100644 index 0000000..791d57e --- /dev/null +++ b/docs/reqstream/ots-sonarmark.yaml @@ -0,0 +1,23 @@ +--- +# SonarMark OTS Requirements +# +# PURPOSE: +# - Define requirements for the SonarMark off-the-shelf SonarCloud reporting tool +# - SonarMark generates a SonarCloud quality report as part of release artifacts + +sections: + - title: SonarMark OTS Requirements + requirements: + - id: ReviewMark-OTS-SonarMark + title: SonarMark shall generate a SonarCloud quality report. + justification: | + DemaConsulting.SonarMark retrieves quality-gate and metrics data from SonarCloud and + renders it as a markdown document included in the release artifacts. It runs in the + same CI pipeline that produces the TRX test results, so a successful pipeline run is + evidence that SonarMark executed without error. + tags: [ots] + tests: + - SonarMark_QualityGateRetrieval + - SonarMark_IssuesRetrieval + - SonarMark_HotSpotsRetrieval + - SonarMark_MarkdownReportGeneration diff --git a/docs/reqstream/ots-versionmark.yaml b/docs/reqstream/ots-versionmark.yaml new file mode 100644 index 0000000..58f0928 --- /dev/null +++ b/docs/reqstream/ots-versionmark.yaml @@ -0,0 +1,21 @@ +--- +# VersionMark OTS Requirements +# +# PURPOSE: +# - Define requirements for the VersionMark off-the-shelf tool-version documentation tool +# - VersionMark publishes captured tool-version information as part of release artifacts + +sections: + - title: VersionMark OTS Requirements + requirements: + - id: ReviewMark-OTS-VersionMark + title: VersionMark shall publish captured tool-version information. + justification: | + DemaConsulting.VersionMark reads version metadata for each dotnet tool used in the + pipeline and writes a versions markdown document included in the release artifacts. + It runs in the same CI pipeline that produces the TRX test results, so a successful + pipeline run is evidence that VersionMark executed without error. + tags: [ots] + tests: + - VersionMark_CapturesVersions + - VersionMark_GeneratesMarkdownReport diff --git a/docs/reqstream/reviewmark-system-requirements.yaml b/docs/reqstream/reviewmark-system.yaml similarity index 100% rename from docs/reqstream/reviewmark-system-requirements.yaml rename to docs/reqstream/reviewmark-system.yaml diff --git a/docs/reqstream/cli-requirements.yaml b/docs/reqstream/subsystem-cli.yaml similarity index 100% rename from docs/reqstream/cli-requirements.yaml rename to docs/reqstream/subsystem-cli.yaml diff --git a/docs/reqstream/configuration-requirements.yaml b/docs/reqstream/subsystem-configuration.yaml similarity index 100% rename from docs/reqstream/configuration-requirements.yaml rename to docs/reqstream/subsystem-configuration.yaml diff --git a/docs/reqstream/index-requirements.yaml b/docs/reqstream/subsystem-index.yaml similarity index 100% rename from docs/reqstream/index-requirements.yaml rename to docs/reqstream/subsystem-index.yaml diff --git a/docs/reqstream/glob-matcher-requirements.yaml b/docs/reqstream/unit-glob-matcher.yaml similarity index 100% rename from docs/reqstream/glob-matcher-requirements.yaml rename to docs/reqstream/unit-glob-matcher.yaml diff --git a/docs/reqstream/path-helpers-requirements.yaml b/docs/reqstream/unit-path-helpers.yaml similarity index 100% rename from docs/reqstream/path-helpers-requirements.yaml rename to docs/reqstream/unit-path-helpers.yaml diff --git a/docs/reqstream/validation-requirements.yaml b/docs/reqstream/unit-validation.yaml similarity index 100% rename from docs/reqstream/validation-requirements.yaml rename to docs/reqstream/unit-validation.yaml diff --git a/requirements.yaml b/requirements.yaml index 395bbf7..fd41a52 100644 --- a/requirements.yaml +++ b/requirements.yaml @@ -24,12 +24,17 @@ # --- includes: - - docs/reqstream/reviewmark-system-requirements.yaml - - docs/reqstream/cli-requirements.yaml - - docs/reqstream/configuration-requirements.yaml - - docs/reqstream/glob-matcher-requirements.yaml - - docs/reqstream/index-requirements.yaml - - docs/reqstream/path-helpers-requirements.yaml + - docs/reqstream/reviewmark-system.yaml + - docs/reqstream/subsystem-cli.yaml + - docs/reqstream/subsystem-configuration.yaml + - docs/reqstream/subsystem-index.yaml + - docs/reqstream/unit-glob-matcher.yaml + - docs/reqstream/unit-path-helpers.yaml + - docs/reqstream/unit-validation.yaml - docs/reqstream/platform-requirements.yaml - - docs/reqstream/validation-requirements.yaml - - docs/reqstream/ots-requirements.yaml + - docs/reqstream/ots-mstest.yaml + - docs/reqstream/ots-reqstream.yaml + - docs/reqstream/ots-buildmark.yaml + - docs/reqstream/ots-versionmark.yaml + - docs/reqstream/ots-sarifmark.yaml + - docs/reqstream/ots-sonarmark.yaml From f21af7f1f61790ca703bc534ff7f8ce8e8e328a0 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Sat, 28 Mar 2026 21:32:55 +0000 Subject: [PATCH 05/18] Add unit-context/program requirements, rename Index.cs to ReviewIndex.cs, convert diagrams to text Agent-Logs-Url: https://github.com/demaconsulting/ReviewMark/sessions/2cd94fd3-6efa-4e14-87c6-aaa5861db6d6 Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com> --- .reviewmark.yaml | 21 ++++----- docs/design/introduction.md | 43 +++++------------- docs/design/program.md | 21 +++------ docs/design/{index.md => review-index.md} | 17 +++---- docs/design/system.md | 35 ++++++--------- docs/reqstream/unit-context.yaml | 41 +++++++++++++++++ docs/reqstream/unit-program.yaml | 44 +++++++++++++++++++ ...stem-index.yaml => unit-review-index.yaml} | 2 +- requirements.yaml | 4 +- .../{Index.cs => ReviewIndex.cs} | 0 10 files changed, 139 insertions(+), 89 deletions(-) rename docs/design/{index.md => review-index.md} (87%) create mode 100644 docs/reqstream/unit-context.yaml create mode 100644 docs/reqstream/unit-program.yaml rename docs/reqstream/{subsystem-index.yaml => unit-review-index.yaml} (99%) rename src/DemaConsulting.ReviewMark/{Index.cs => ReviewIndex.cs} (100%) diff --git a/.reviewmark.yaml b/.reviewmark.yaml index b086516..a2c7b54 100644 --- a/.reviewmark.yaml +++ b/.reviewmark.yaml @@ -32,7 +32,7 @@ reviews: - id: ReviewMark-Context title: Review of Context software unit (command-line argument handling) paths: - - "docs/reqstream/subsystem-cli.yaml" # requirements + - "docs/reqstream/unit-context.yaml" # requirements - "docs/design/context.md" # design - "src/**/Context.cs" # implementation - "test/**/ContextTests.cs" # tests @@ -45,12 +45,12 @@ reviews: - "src/**/GlobMatcher.cs" # implementation - "test/**/GlobMatcherTests.cs" # tests - - id: ReviewMark-Index - title: Review of Index software unit (review evidence indexing) + - id: ReviewMark-ReviewIndex + title: Review of ReviewIndex software unit (review evidence indexing) paths: - - "docs/reqstream/subsystem-index.yaml" # requirements - - "docs/design/index.md" # design - - "src/**/Index.cs" # implementation + - "docs/reqstream/unit-review-index.yaml" # requirements + - "docs/design/review-index.md" # design + - "src/**/ReviewIndex.cs" # implementation - "test/**/IndexTests.cs" # tests - id: ReviewMark-PathHelpers @@ -64,8 +64,7 @@ reviews: - id: ReviewMark-Program title: Review of Program software unit (main entry point and tool orchestration) paths: - - "docs/reqstream/subsystem-cli.yaml" # requirements - - "docs/reqstream/platform-requirements.yaml" # platform requirements + - "docs/reqstream/unit-program.yaml" # requirements - "docs/design/program.md" # design - "docs/guide/guide.md" # user guide - "src/**/Program.cs" # implementation @@ -108,7 +107,7 @@ reviews: - "docs/design/system.md" # system design - "docs/design/context.md" # Context design - "docs/design/glob-matcher.md" # GlobMatcher design - - "docs/design/index.md" # Index design + - "docs/design/review-index.md" # ReviewIndex design - "docs/design/path-helpers.md" # PathHelpers design - "docs/design/program.md" # Program design - "docs/design/review-mark-configuration.md" # ReviewMarkConfiguration design @@ -121,7 +120,9 @@ reviews: - "docs/reqstream/reviewmark-system.yaml" # system-level requirements - "docs/reqstream/subsystem-cli.yaml" # CLI subsystem requirements - "docs/reqstream/subsystem-configuration.yaml" # Configuration subsystem requirements - - "docs/reqstream/subsystem-index.yaml" # Index subsystem requirements + - "docs/reqstream/unit-context.yaml" # Context unit requirements + - "docs/reqstream/unit-program.yaml" # Program unit requirements + - "docs/reqstream/unit-review-index.yaml" # ReviewIndex unit requirements - "docs/reqstream/unit-glob-matcher.yaml" # GlobMatcher unit requirements - "docs/reqstream/unit-path-helpers.yaml" # PathHelpers unit requirements - "docs/reqstream/unit-validation.yaml" # Validation unit requirements diff --git a/docs/design/introduction.md b/docs/design/introduction.md index f3e1377..648be94 100644 --- a/docs/design/introduction.md +++ b/docs/design/introduction.md @@ -33,37 +33,18 @@ This document does not cover: The following diagram shows the decomposition of the ReviewMark software system into subsystems and software units. -```mermaid -graph LR - System["ReviewMark (Software System)"] - - CLI["CLI Subsystem"] - Program["Program (Software Unit)"] - Context["Context (Software Unit)"] - - Config["Configuration Subsystem"] - RMConfig["ReviewMarkConfiguration (Software Unit)"] - Glob["GlobMatcher (Software Unit)"] - - IndexSub["Index Subsystem"] - Index["Index (Software Unit)"] - PathHelpers["PathHelpers (Software Unit)"] - - Validation["Validation (Software Unit)"] - - System --> CLI - System --> Config - System --> IndexSub - System --> Validation - - CLI --> Program - CLI --> Context - - Config --> RMConfig - Config --> Glob - - IndexSub --> Index - IndexSub --> PathHelpers +```text +ReviewMark (Software System) +├── CLI Subsystem +│ ├── Program (Software Unit) +│ └── Context (Software Unit) +├── Configuration Subsystem +│ ├── ReviewMarkConfiguration (Software Unit) +│ └── GlobMatcher (Software Unit) +├── Index Subsystem +│ ├── ReviewIndex (Software Unit) +│ └── PathHelpers (Software Unit) +└── Validation (Software Unit) ``` ## Audience diff --git a/docs/design/program.md b/docs/design/program.md index d018122..cf70763 100644 --- a/docs/design/program.md +++ b/docs/design/program.md @@ -28,21 +28,12 @@ via `Context.WriteError()`, and causes a non-zero exit code to be returned. `Program.Run(Context)` evaluates the parsed flags in the following priority order, executing the first matching action and returning: -```mermaid -graph TD - A[Start] --> B{--version flag?} - B -->|yes| C[Print version and return] - B -->|no| D{--help flag?} - D -->|yes| E[Print banner and return] - D -->|no| F{--validate flag?} - F -->|yes| G[Run self-validation and return] - F -->|no| H{--lint flag?} - H -->|yes| I[Run configuration lint and return] - H -->|no| J{--index paths provided?} - J -->|yes| K[Scan and write evidence index, then return] - J -->|no| L[Generate Review Plan and/or Review Report] - L --> M[Return] -``` +1. If `--version` — print version and return +2. If `--help` — print banner and return +3. If `--validate` — run self-validation and return +4. If `--lint` — run configuration lint and return +5. If `--index` paths provided — scan and write evidence index, then return +6. Otherwise — generate Review Plan and/or Review Report and return Only one top-level action is performed per invocation. Actions later in the priority order are not reached if an earlier flag is set. diff --git a/docs/design/index.md b/docs/design/review-index.md similarity index 87% rename from docs/design/index.md rename to docs/design/review-index.md index d83277d..0b126b8 100644 --- a/docs/design/index.md +++ b/docs/design/review-index.md @@ -1,8 +1,8 @@ -# Index +# ReviewIndex ## Purpose -The `Index` software unit manages the loading, querying, and creation of the review +The `ReviewIndex` software unit manages the loading, querying, and creation of the review evidence index. It abstracts the evidence store behind a uniform interface so that the rest of the tool does not need to know whether evidence is stored on a fileshare, served over HTTP, or absent entirely. @@ -48,14 +48,11 @@ source type is `none`, resulting in all review-sets being reported as Missing. `ReviewIndex.GetStatus(id, fingerprint)` determines the review status of a review-set by looking up the `id` in the loaded index: -```mermaid -graph TD - A[Look up id in index] --> B{id found?} - B -->|no| C[Return Missing] - B -->|yes| D{fingerprint matches AND result is pass?} - D -->|yes| E[Return Current] - D -->|no| F[Return Stale] -``` +1. Look up `id` in the index + - If not found — return `Missing` +2. Check if `fingerprint` matches AND result is `pass` + - If yes — return `Current` + - If no — return `Stale` | Status | Meaning | | ------ | ------- | diff --git a/docs/design/system.md b/docs/design/system.md index f7d59c9..8674b8e 100644 --- a/docs/design/system.md +++ b/docs/design/system.md @@ -12,27 +12,20 @@ documents: a Review Plan and a Review Report. ## Main Workflow -The following diagram illustrates the end-to-end processing flow. - -```mermaid -graph TD - A[Parse CLI arguments] --> B[Load .reviewmark.yaml] - B --> C[Resolve file lists via glob patterns] - C --> D[Compute SHA-256 fingerprints] - D --> E[Load evidence index] - E --> F{Evidence source type?} - F -->|none| G[Use empty index] - F -->|fileshare| H[Load index.json from file path] - F -->|url| I[Load index.json from HTTP URL] - G --> J[Generate Review Plan / Review Report] - H --> J - I --> J - J --> K{Enforce flag set?} - K -->|yes| L{All sets Current?} - L -->|no| M[Return non-zero exit code] - L -->|yes| N[Return success] - K -->|no| N -``` +The following steps describe the end-to-end processing flow. + +1. Parse CLI arguments +2. Load `.reviewmark.yaml` +3. Resolve file lists via glob patterns +4. Compute SHA-256 fingerprints +5. Load evidence index + - `none` — use an empty index (no evidence store configured) + - `fileshare` — load `index.json` from a local or network file path + - `url` — download `index.json` from an HTTP or HTTPS URL +6. Generate Review Plan and/or Review Report +7. If `--enforce` flag is set: + - If all review-sets are Current — return success + - Otherwise — return a non-zero exit code ## Evidence Source Types diff --git a/docs/reqstream/unit-context.yaml b/docs/reqstream/unit-context.yaml new file mode 100644 index 0000000..2c880de --- /dev/null +++ b/docs/reqstream/unit-context.yaml @@ -0,0 +1,41 @@ +--- +# Context Software Unit Requirements +# +# PURPOSE: +# - Define requirements for the Context software unit +# - This unit parses command-line arguments into an in-memory context object +# - It also provides unified output and logging across the tool + +sections: + - title: Context Unit Requirements + requirements: + - id: ReviewMark-Context-Parsing + title: The Context unit shall parse command-line arguments into a strongly-typed Context object. + justification: | + All downstream processing reads options from the Context object rather than + directly from the raw argument array. The Context.Create factory method processes + arguments sequentially, recognizing flag and value arguments, and returns a fully + initialized Context. Unknown arguments must raise an ArgumentException so the + caller can report a clear error message. + tests: + - Context_Create_NoArguments_ReturnsDefaultContext + - Context_Create_VersionFlag_SetsVersionTrue + - Context_Create_HelpFlag_SetsHelpTrue + - Context_Create_SilentFlag_SetsSilentTrue + - Context_Create_ValidateFlag_SetsValidateTrue + - Context_Create_ResultsFlag_SetsResultsFile + - Context_Create_LogFlag_OpensLogFile + - Context_Create_UnknownArgument_ThrowsArgumentException + - Context_Create_ShortVersionFlag_SetsVersionTrue + + - id: ReviewMark-Context-Output + title: The Context unit shall provide WriteLine and WriteError methods for unified output and logging. + justification: | + All output goes through Context so that the --silent flag is honoured and + optionally duplicated to a log file opened by the --log flag. WriteError must + additionally set the error exit code so that the process exits with a non-zero + status when any error is reported. + tests: + - Context_WriteError_NotSilent_WritesToConsole + - Context_WriteError_SetsErrorExitCode + - Context_WriteLine_Silent_DoesNotWriteToConsole diff --git a/docs/reqstream/unit-program.yaml b/docs/reqstream/unit-program.yaml new file mode 100644 index 0000000..9a9c391 --- /dev/null +++ b/docs/reqstream/unit-program.yaml @@ -0,0 +1,44 @@ +--- +# Program Software Unit Requirements +# +# PURPOSE: +# - Define requirements for the Program software unit +# - This unit is the main entry point and top-level orchestrator of the tool +# - It dispatches to processing logic based on parsed CLI flags + +sections: + - title: Program Unit Requirements + requirements: + - id: ReviewMark-Program-EntryPoint + title: >- + The Program unit shall construct a Context, dispatch to the appropriate operation, + and return the Context exit code as the process exit code. + justification: | + Program.Main is the process entry point. It must create the execution context, + call Program.Run to perform the requested operation, and return the exit code + from the context so that callers can detect success or failure programmatically. + Any unhandled exception must be caught, written to error output, and cause a + non-zero exit code rather than an unhandled process crash. + tests: + - Program_Run_WithVersionFlag_DisplaysVersionOnly + - Program_Version_ReturnsNonEmptyString + - Program_Run_WithHelpFlag_DisplaysUsageInformation + + - id: ReviewMark-Program-Dispatch + title: >- + The Program unit shall dispatch to exactly one operation per invocation based on + the priority order of CLI flags. + justification: | + --version, --help, --validate, --lint, --index, and plan/report operations must + be evaluated in a fixed priority order so that the behavior is predictable and + documented. Only the first matching flag action is executed; later flags are + not reached. + tests: + - Program_Run_WithVersionFlag_DisplaysVersionOnly + - Program_Run_WithHelpFlag_DisplaysUsageInformation + - Program_Run_WithValidateFlag_RunsValidation + - Program_Run_WithLintFlag_ValidConfig_ReportsSuccess + - Program_Run_WithHelpFlag_IncludesElaborateOption + - Program_Run_WithHelpFlag_IncludesLintOption + - Program_Run_WithElaborateFlag_OutputsElaboration + - Program_Run_WithElaborateFlag_UnknownId_ReportsError diff --git a/docs/reqstream/subsystem-index.yaml b/docs/reqstream/unit-review-index.yaml similarity index 99% rename from docs/reqstream/subsystem-index.yaml rename to docs/reqstream/unit-review-index.yaml index 0abf7b8..ad05a88 100644 --- a/docs/reqstream/subsystem-index.yaml +++ b/docs/reqstream/unit-review-index.yaml @@ -1,5 +1,5 @@ --- -# Index Software Unit Requirements +# ReviewIndex Software Unit Requirements # # PURPOSE: # - Define requirements for the ReviewIndex software unit diff --git a/requirements.yaml b/requirements.yaml index fd41a52..341b6db 100644 --- a/requirements.yaml +++ b/requirements.yaml @@ -27,7 +27,9 @@ includes: - docs/reqstream/reviewmark-system.yaml - docs/reqstream/subsystem-cli.yaml - docs/reqstream/subsystem-configuration.yaml - - docs/reqstream/subsystem-index.yaml + - docs/reqstream/unit-context.yaml + - docs/reqstream/unit-program.yaml + - docs/reqstream/unit-review-index.yaml - docs/reqstream/unit-glob-matcher.yaml - docs/reqstream/unit-path-helpers.yaml - docs/reqstream/unit-validation.yaml diff --git a/src/DemaConsulting.ReviewMark/Index.cs b/src/DemaConsulting.ReviewMark/ReviewIndex.cs similarity index 100% rename from src/DemaConsulting.ReviewMark/Index.cs rename to src/DemaConsulting.ReviewMark/ReviewIndex.cs From 819798fa7d77650fbb138c153ead94bbd5df3a12 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Sat, 28 Mar 2026 21:39:30 +0000 Subject: [PATCH 06/18] Add ReviewEvidence record documentation to review-index.md Agent-Logs-Url: https://github.com/demaconsulting/ReviewMark/sessions/5bc38306-41ce-46fc-ab70-c8424822b0b7 Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com> --- docs/design/review-index.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/docs/design/review-index.md b/docs/design/review-index.md index 0b126b8..92c6890 100644 --- a/docs/design/review-index.md +++ b/docs/design/review-index.md @@ -7,6 +7,23 @@ evidence index. It abstracts the evidence store behind a uniform interface so th the rest of the tool does not need to know whether evidence is stored on a fileshare, served over HTTP, or absent entirely. +## ReviewEvidence Record + +`ReviewEvidence` is an immutable record that holds the in-memory representation of a +single review record once the index has been loaded or scanned. + +| Property | Type | Description | +| -------- | ---- | ----------- | +| `Id` | string | The review-set identifier | +| `Fingerprint` | string | The SHA-256 fingerprint of the reviewed files | +| `Date` | string | The date of the review (e.g. `2026-02-14`) | +| `Result` | string | The review outcome (`pass` or `fail`) | +| `File` | string | The file name of the review evidence PDF | + +The `ReviewIndex` holds these records in a two-level +`Dictionary>` keyed first by `Id` and +then by `Fingerprint`, which enables O(1) lookup by both fields simultaneously. + ## Evidence Index Format The evidence index is a JSON file (`index.json`) containing an array of review records. From 20cc0a1521747bd38e697e9c4a333339ef7ddfc9 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Sat, 28 Mar 2026 22:06:30 +0000 Subject: [PATCH 07/18] Fix requirements test linkages to match actual test method names Agent-Logs-Url: https://github.com/demaconsulting/ReviewMark/sessions/69ec20b8-a181-4b63-b73e-400ec7abd055 Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com> --- docs/reqstream/reviewmark-system.yaml | 4 ---- docs/reqstream/unit-glob-matcher.yaml | 8 +++----- docs/reqstream/unit-path-helpers.yaml | 10 ++++------ 3 files changed, 7 insertions(+), 15 deletions(-) diff --git a/docs/reqstream/reviewmark-system.yaml b/docs/reqstream/reviewmark-system.yaml index 807b362..5e51e6e 100644 --- a/docs/reqstream/reviewmark-system.yaml +++ b/docs/reqstream/reviewmark-system.yaml @@ -19,7 +19,6 @@ sections: evidence automatically on each CI/CD run, replacing manual tracking spreadsheets. tests: - ReviewMark_ReviewPlanGeneration - - IntegrationTest_PlanFlag_GeneratesReviewPlan - id: ReviewMark-System-ReviewReport title: The tool shall generate a Review Report document listing every review-set and its current review status. @@ -30,7 +29,6 @@ sections: status for each review-set. tests: - ReviewMark_ReviewReportGeneration - - IntegrationTest_ReportFlag_GeneratesReviewReport - id: ReviewMark-System-Enforce title: The tool shall return a non-zero exit code when enforcement is enabled and any review-set is not current. @@ -40,7 +38,6 @@ sections: Stale or Missing status, making incomplete review coverage a build-breaking condition. tests: - ReviewMark_Enforce - - IntegrationTest_EnforceFlag_FailsWhenReviewStale - id: ReviewMark-System-IndexScan title: The tool shall scan PDF evidence files and write an index.json when the --index flag is provided. @@ -51,7 +48,6 @@ sections: without manual maintenance of the index file. tests: - ReviewMark_IndexScan - - IntegrationTest_IndexFlag_ScansEvidence - id: ReviewMark-System-Validate title: The tool shall execute self-validation tests when the --validate flag is provided. diff --git a/docs/reqstream/unit-glob-matcher.yaml b/docs/reqstream/unit-glob-matcher.yaml index 74bd142..2529257 100644 --- a/docs/reqstream/unit-glob-matcher.yaml +++ b/docs/reqstream/unit-glob-matcher.yaml @@ -21,11 +21,9 @@ sections: fingerprinting regardless of filesystem iteration order. tests: - GlobMatcher_GetMatchingFiles_SingleIncludePattern_ReturnsMatchingFiles - - GlobMatcher_GetMatchingFiles_ExcludePattern_RemovesMatchingFiles - - GlobMatcher_GetMatchingFiles_MultiplePatterns_AppliesInOrder + - GlobMatcher_GetMatchingFiles_ExcludePattern_ExcludesMatchingFiles + - GlobMatcher_GetMatchingFiles_ReIncludeAfterExclude_ReturnsReIncludedFiles + - GlobMatcher_GetMatchingFiles_IncludeAndExclude_ReturnsFilteredFiles - GlobMatcher_GetMatchingFiles_NullBaseDirectory_ThrowsArgumentNullException - - GlobMatcher_GetMatchingFiles_EmptyBaseDirectory_ThrowsArgumentException - GlobMatcher_GetMatchingFiles_NullPatterns_ThrowsArgumentNullException - GlobMatcher_GetMatchingFiles_NoMatchingFiles_ReturnsEmptyList - - GlobMatcher_GetMatchingFiles_ResultsAreSorted - - GlobMatcher_GetMatchingFiles_ResultsUseForwardSlashes diff --git a/docs/reqstream/unit-path-helpers.yaml b/docs/reqstream/unit-path-helpers.yaml index da3cafb..b1cd13f 100644 --- a/docs/reqstream/unit-path-helpers.yaml +++ b/docs/reqstream/unit-path-helpers.yaml @@ -18,9 +18,7 @@ sections: contain '..' sequences or absolute path components, and performs a defense-in-depth check that the resolved combined path remains under the base directory. tests: - - PathHelpers_SafePathCombine_ValidPaths_ReturnsCombinedPath - - PathHelpers_SafePathCombine_NullBasePath_ThrowsArgumentNullException - - PathHelpers_SafePathCombine_NullRelativePath_ThrowsArgumentNullException - - PathHelpers_SafePathCombine_PathTraversal_ThrowsArgumentException - - PathHelpers_SafePathCombine_AbsoluteRelativePath_ThrowsArgumentException - - PathHelpers_SafePathCombine_NestedPath_ReturnsCombinedPath + - PathHelpers_SafePathCombine_ValidPaths_CombinesCorrectly + - PathHelpers_SafePathCombine_PathTraversalWithDoubleDots_ThrowsArgumentException + - PathHelpers_SafePathCombine_AbsolutePath_ThrowsArgumentException + - PathHelpers_SafePathCombine_NestedPaths_CombinesCorrectly From 68f4dbc81f7de54daa3e893d47871795c4a13258 Mon Sep 17 00:00:00 2001 From: Malcolm Nixon Date: Sat, 28 Mar 2026 18:08:48 -0400 Subject: [PATCH 08/18] Update docs/design/review-mark-configuration.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/design/review-mark-configuration.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/design/review-mark-configuration.md b/docs/design/review-mark-configuration.md index 449d678..60bb63b 100644 --- a/docs/design/review-mark-configuration.md +++ b/docs/design/review-mark-configuration.md @@ -62,9 +62,9 @@ For each review-set the report includes: - The review-set `id` and `title` - The current fingerprint of the file-set -- The review status: `Current`, `Stale`, or `Missing` +- The review status: `Current`, `Stale`, `Missing`, or `Failed` -Status is determined by querying the loaded `ReviewIndex` via `GetStatus()`. +Status is computed by `ReviewMarkConfiguration.PublishReviewReport(...)`, which uses the loaded evidence index to determine whether the current fingerprint has a passing, failing, stale, or missing review result. - The `--report-depth` argument controls the heading level used for sections - The `--elaborate` flag expands the list of files covered by each review-set From 6d322cf704b062b447e4b74f0482a413ba984a5c Mon Sep 17 00:00:00 2001 From: Malcolm Nixon Date: Sat, 28 Mar 2026 18:09:24 -0400 Subject: [PATCH 09/18] Update docs/design/program.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/design/program.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/design/program.md b/docs/design/program.md index cf70763..6d260cc 100644 --- a/docs/design/program.md +++ b/docs/design/program.md @@ -20,8 +20,10 @@ time from the assembly metadata and follows semantic versioning conventions. 2. Calls `Program.Run(Context)` to perform the requested operation 3. Returns `Context.ExitCode` as the process exit code -Any unhandled exception that escapes `Run()` is caught, written to the error output -via `Context.WriteError()`, and causes a non-zero exit code to be returned. +Any unexpected exception that escapes `Run()` is logged to the standard error stream +via `Console.Error` and then rethrown. As a result, the process terminates due to the +unhandled exception and the final exit code is determined by the .NET runtime rather +than by `Program.Main` explicitly returning a non-zero value. ## Run() Dispatch Logic From acfd7b6b35e93518b1793b521fe1656839d511fe Mon Sep 17 00:00:00 2001 From: Malcolm Nixon Date: Sat, 28 Mar 2026 18:09:48 -0400 Subject: [PATCH 10/18] Update docs/reqstream/unit-program.yaml Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/reqstream/unit-program.yaml | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/reqstream/unit-program.yaml b/docs/reqstream/unit-program.yaml index 9a9c391..356f383 100644 --- a/docs/reqstream/unit-program.yaml +++ b/docs/reqstream/unit-program.yaml @@ -17,8 +17,9 @@ sections: Program.Main is the process entry point. It must create the execution context, call Program.Run to perform the requested operation, and return the exit code from the context so that callers can detect success or failure programmatically. - Any unhandled exception must be caught, written to error output, and cause a - non-zero exit code rather than an unhandled process crash. + Unexpected exceptions are written to error output and then rethrown, so callers + may observe either a normal exit code or a process termination due to an + unhandled exception. tests: - Program_Run_WithVersionFlag_DisplaysVersionOnly - Program_Version_ReturnsNonEmptyString From 4395bf097e0b648665857355aff5d0bc958285eb Mon Sep 17 00:00:00 2001 From: Malcolm Nixon Date: Sat, 28 Mar 2026 18:10:17 -0400 Subject: [PATCH 11/18] Update docs/design/review-index.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/design/review-index.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/design/review-index.md b/docs/design/review-index.md index 92c6890..a43b87f 100644 --- a/docs/design/review-index.md +++ b/docs/design/review-index.md @@ -52,8 +52,10 @@ source type: `ReviewIndex.Scan(directory, patterns)` scans a directory for PDF files matching the given glob patterns. For each PDF file found, it reads embedded metadata to -extract the review record fields and writes the resulting records to `index.json` -in the scanned directory. This supports the `--index` workflow. +extract the review record fields and returns a populated in-memory `ReviewIndex`. +The caller (e.g., `Program`) is responsible for choosing an output path and calling +`Save(...)` on the returned index to produce `index.json` as part of the `--index` +workflow. ## ReviewIndex.Empty() From 1fce71cbe70bacabed162faa0c4b8a3c7192d7f8 Mon Sep 17 00:00:00 2001 From: Malcolm Nixon Date: Sat, 28 Mar 2026 18:11:04 -0400 Subject: [PATCH 12/18] Update docs/design/review-index.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/design/review-index.md | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/docs/design/review-index.md b/docs/design/review-index.md index a43b87f..dd85d93 100644 --- a/docs/design/review-index.md +++ b/docs/design/review-index.md @@ -69,12 +69,15 @@ review-set by looking up the `id` in the loaded index: 1. Look up `id` in the index - If not found — return `Missing` -2. Check if `fingerprint` matches AND result is `pass` - - If yes — return `Current` - - If no — return `Stale` +2. Check if there is a record whose `Fingerprint` matches the supplied `fingerprint` + - If no matching fingerprint exists — return `Stale` + - If a matching fingerprint exists: + - If the `Result` is `pass` — return `Current` + - If the `Result` is not `pass` — return `Failed` | Status | Meaning | | ------ | ------- | | `Current` | The review record matches the current fingerprint and has a passing result | -| `Stale` | A review record exists for the id but the fingerprint does not match (files have changed since review) | +| `Failed` | The review record matches the current fingerprint but the result is not passing | +| `Stale` | A review record exists for the id but the fingerprint does not match (files have changed since review or a different revision was reviewed) | | `Missing` | No review record exists for the id | From ae75b57e116eceae699ba5d52bdf13371beec533 Mon Sep 17 00:00:00 2001 From: Malcolm Nixon Date: Sat, 28 Mar 2026 18:11:27 -0400 Subject: [PATCH 13/18] Update docs/design/context.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/design/context.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/design/context.md b/docs/design/context.md index cd718f8..9a3d5ad 100644 --- a/docs/design/context.md +++ b/docs/design/context.md @@ -34,8 +34,10 @@ arguments: `Context.Create(string[] args)` is a factory method that processes the argument array sequentially, recognizing both flag arguments (e.g., `--validate`) and -value arguments (e.g., `--plan `). Unrecognized arguments are silently -ignored. The resulting `Context` instance holds the fully parsed state. +value arguments (e.g., `--plan `). Unrecognized or unsupported arguments +cause `Context.ParseArgument` to throw an `ArgumentException`, which callers of +`Context.Create` are expected to handle and surface as a CLI error. The resulting +`Context` instance holds the fully parsed state when argument parsing succeeds. ## Output Methods From 4453f0227483062c52c27934b5d6d0622473523d Mon Sep 17 00:00:00 2001 From: Malcolm Nixon Date: Sat, 28 Mar 2026 18:11:48 -0400 Subject: [PATCH 14/18] Update docs/design/system.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/design/system.md | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/docs/design/system.md b/docs/design/system.md index 8674b8e..37f0369 100644 --- a/docs/design/system.md +++ b/docs/design/system.md @@ -48,14 +48,21 @@ and written to a configurable output path. ### Review Report The Review Report lists every review-set defined in the configuration, the current -fingerprint of its file-set, and the review status (Current, Stale, or Missing). +fingerprint of its file-set, and the review status (Current, Stale, Missing, or Failed). It is generated by the `--report` flag and written to a configurable output path. +The statuses have the following meanings: +- **Current** — Evidence exists for the current fingerprint and the recorded result is `pass`. +- **Stale** — Evidence exists, but it corresponds to an older fingerprint than the current one. +- **Missing** — No evidence exists for this review-set. +- **Failed** — Evidence exists for the current fingerprint, but the recorded result is not `pass`. + ## Enforcement When the `--enforce` flag is set, ReviewMark returns a non-zero exit code if any -review-set does not have Current status. This allows CI/CD pipelines to fail builds -when review coverage is incomplete or out of date. +review-set does not have Current status (i.e., is Stale, Missing, or Failed). This allows +CI/CD pipelines to fail builds when review coverage is incomplete, out of date, or has +failed results for the current fingerprint. ## Index Management From ab8253700dcc9def7822a30dc4d1c1fbf0335ffa Mon Sep 17 00:00:00 2001 From: Malcolm Nixon Date: Sat, 28 Mar 2026 18:12:37 -0400 Subject: [PATCH 15/18] Update docs/design/definition.yaml Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/design/definition.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/design/definition.yaml b/docs/design/definition.yaml index c0daf41..1b115fd 100644 --- a/docs/design/definition.yaml +++ b/docs/design/definition.yaml @@ -8,7 +8,7 @@ input-files: - docs/design/system.md - docs/design/context.md - docs/design/glob-matcher.md - - docs/design/index.md + - docs/design/review-index.md - docs/design/path-helpers.md - docs/design/program.md - docs/design/review-mark-configuration.md From beef75d18d4bad69f8f4caea1f6b10b283f7ee22 Mon Sep 17 00:00:00 2001 From: Malcolm Nixon Date: Sat, 28 Mar 2026 18:13:08 -0400 Subject: [PATCH 16/18] Update docs/reqstream/unit-path-helpers.yaml Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/reqstream/unit-path-helpers.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/reqstream/unit-path-helpers.yaml b/docs/reqstream/unit-path-helpers.yaml index b1cd13f..f9295bc 100644 --- a/docs/reqstream/unit-path-helpers.yaml +++ b/docs/reqstream/unit-path-helpers.yaml @@ -4,7 +4,7 @@ # PURPOSE: # - Define requirements for the PathHelpers software unit # - This unit provides safe path operations that prevent path traversal attacks -# - It is used by Index.cs and Validation.cs when constructing file paths +# - It is used by ReviewIndex.cs and Validation.cs when constructing file paths sections: - title: PathHelpers Unit Requirements From 5456ec7f098d49a34d5d235af8644bfe42276ea2 Mon Sep 17 00:00:00 2001 From: Malcolm Nixon Date: Sat, 28 Mar 2026 18:13:34 -0400 Subject: [PATCH 17/18] Update docs/design/review-mark-configuration.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/design/review-mark-configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/design/review-mark-configuration.md b/docs/design/review-mark-configuration.md index 60bb63b..59f09a6 100644 --- a/docs/design/review-mark-configuration.md +++ b/docs/design/review-mark-configuration.md @@ -14,7 +14,7 @@ The `.reviewmark.yaml` file is deserialized into the following model: | Class | Description | | ----- | ----------- | | `ReviewMarkYaml` | Root configuration object containing the evidence source and review list | -| `EvidenceSourceYaml` | Describes how to locate the evidence index (`type`, `path` or `url`) | +| `EvidenceSourceYaml` | Describes how to locate the evidence index (`type`, `location`, optional `credentials`) | | `ReviewYaml` | Describes a single review-set (`id`, `title`, file patterns) | ## ReviewMarkConfiguration.Load() From 5bbfed4557213f303ed91cebae0529ea934c05f9 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Sat, 28 Mar 2026 22:23:42 +0000 Subject: [PATCH 18/18] Revise design docs to focus on WHAT/WHY rather than implementation HOW Agent-Logs-Url: https://github.com/demaconsulting/ReviewMark/sessions/1632c6d8-1a27-4cfa-a722-72ae2f98ecd2 Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com> --- docs/design/context.md | 6 ------ docs/design/glob-matcher.md | 24 +++++++--------------- docs/design/path-helpers.md | 22 ++++++++------------ docs/design/review-index.md | 2 +- docs/design/review-mark-configuration.md | 21 ++++++++----------- docs/design/system.md | 1 + docs/design/validation.md | 26 ++++++++---------------- 7 files changed, 35 insertions(+), 67 deletions(-) diff --git a/docs/design/context.md b/docs/design/context.md index 9a3d5ad..eed5e1e 100644 --- a/docs/design/context.md +++ b/docs/design/context.md @@ -57,9 +57,3 @@ a non-zero value when an error is detected. The value of `ExitCode` is returned When a log file path is provided via the relevant CLI argument, `Context` opens and holds the log file handle for the duration of the tool run. All output written through `WriteLine` and `WriteError` is duplicated to the log file. - -## Resource Cleanup - -`Context` implements `IDisposable`. Disposal closes the log file handle if one is -open. `Program.Main()` constructs `Context` inside a `using` statement to guarantee -timely disposal regardless of how the tool exits. diff --git a/docs/design/glob-matcher.md b/docs/design/glob-matcher.md index b36b462..71c9a1a 100644 --- a/docs/design/glob-matcher.md +++ b/docs/design/glob-matcher.md @@ -10,23 +10,13 @@ file enumeration primitive used by the Configuration subsystem to expand the ## Algorithm `GlobMatcher.GetMatchingFiles(baseDirectory, patterns)` processes patterns in the -order they are declared. Each pattern is evaluated as follows: - -```mermaid -graph TD - A[For each pattern in order] --> B{Starts with '!'?} - B -->|yes| C[Remove '!' prefix → exclusion pattern] - B -->|no| D[Inclusion pattern] - C --> E[Remove matching paths from result set] - D --> F[Add matching paths to result set] - F --> A - E --> A - A --> G[Sort result set] - G --> H[Return as relative paths with forward slashes] -``` - -Patterns are evaluated against the base directory using standard glob semantics. -The `**` wildcard matches any number of path segments (recursive matching). +order they are declared. Patterns prefixed with `!` are exclusion patterns; all +others are inclusion patterns. Each inclusion pattern adds matching paths to the +result set; each exclusion pattern removes matching paths from the result set. +Because patterns are applied in declaration order, a later pattern can re-include +files excluded by an earlier one, or exclude files included by an earlier one. The +`**` wildcard matches any number of path segments, enabling recursive matching. +After all patterns are processed, the result set is sorted and returned. ## Return Value diff --git a/docs/design/path-helpers.md b/docs/design/path-helpers.md index b75bc68..942cae2 100644 --- a/docs/design/path-helpers.md +++ b/docs/design/path-helpers.md @@ -12,19 +12,15 @@ file system paths to evidence PDF files referenced in the evidence index. with an untrusted relative path from the evidence index, validating that the result does not escape the base directory. -The algorithm is: - -```mermaid -graph TD - A[Receive basePath and relativePath] --> B{relativePath contains '..' segments?} - B -->|yes| C[Throw exception: path traversal rejected] - B -->|no| D{relativePath is rooted?} - D -->|yes| E[Throw exception: absolute path rejected] - D -->|no| F[Combine basePath and relativePath] - F --> G{Combined path starts with basePath?} - G -->|no| H[Throw exception: traversal detected after combination] - G -->|yes| I[Return combined path] -``` +The validation steps are: + +1. Reject any relative path that contains `..` segments (explicit traversal attempt). +2. Reject any relative path that is rooted (absolute path supplied where a relative one is required). +3. Combine the base path and relative path. +4. Verify that the combined path still begins with the base path (catches edge cases + such as platform-specific path normalization that might otherwise bypass the + earlier checks). +5. Return the combined path. The double-check strategy (pre-validation of segments plus post-combination verification) defends against edge cases such as URL-encoded separators or diff --git a/docs/design/review-index.md b/docs/design/review-index.md index dd85d93..1a3ef95 100644 --- a/docs/design/review-index.md +++ b/docs/design/review-index.md @@ -79,5 +79,5 @@ review-set by looking up the `id` in the loaded index: | ------ | ------- | | `Current` | The review record matches the current fingerprint and has a passing result | | `Failed` | The review record matches the current fingerprint but the result is not passing | -| `Stale` | A review record exists for the id but the fingerprint does not match (files have changed since review or a different revision was reviewed) | +| `Stale` | A record exists for the id but the fingerprint does not match the current one | | `Missing` | No review record exists for the id | diff --git a/docs/design/review-mark-configuration.md b/docs/design/review-mark-configuration.md index 59f09a6..9692eb1 100644 --- a/docs/design/review-mark-configuration.md +++ b/docs/design/review-mark-configuration.md @@ -29,16 +29,10 @@ and returns a fully initialized configuration object ready for plan/report gener The fingerprint for a review-set uniquely identifies the exact content of its file-set. The algorithm is: -```mermaid -graph TD - A[For each file in review-set sorted list] --> B[Read file contents] - B --> C[Compute SHA-256 hash of file contents] - C --> D[Collect all per-file hashes] - D --> E[Sort hashes lexicographically] - E --> F[Concatenate sorted hashes] - F --> G[Compute SHA-256 of concatenated hashes] - G --> H[Return as hex string fingerprint] -``` +1. For each file in the review-set, read its contents and compute a SHA-256 hash. +2. Collect all per-file hashes and sort them lexicographically. +3. Concatenate the sorted hashes and compute a SHA-256 hash of the result. +4. Return the final hash as a hex string — this is the review-set fingerprint. Sorting the per-file hashes before combining them ensures that the fingerprint is sensitive to content changes but not to the order in which files happen to be @@ -46,7 +40,7 @@ enumerated by the operating system. ## Review Plan Generation -The Review Plan is generated by `ReviewMarkConfiguration.WritePlan()`. It produces +The Review Plan is generated by `ReviewMarkConfiguration.PublishReviewPlan()`. It produces a Markdown document that lists every file in the `needs-review` file-set and, for each file, identifies which review-sets provide coverage. @@ -55,7 +49,7 @@ each file, identifies which review-sets provide coverage. ## Review Report Generation -The Review Report is generated by `ReviewMarkConfiguration.WriteReport()`. It +The Review Report is generated by `ReviewMarkConfiguration.PublishReviewReport()`. It produces a Markdown document that lists every review-set with its current status. For each review-set the report includes: @@ -64,7 +58,8 @@ For each review-set the report includes: - The current fingerprint of the file-set - The review status: `Current`, `Stale`, `Missing`, or `Failed` -Status is computed by `ReviewMarkConfiguration.PublishReviewReport(...)`, which uses the loaded evidence index to determine whether the current fingerprint has a passing, failing, stale, or missing review result. +Status is determined by looking up the current fingerprint in the loaded evidence +index to establish whether a passing, failing, stale, or missing review result exists. - The `--report-depth` argument controls the heading level used for sections - The `--elaborate` flag expands the list of files covered by each review-set diff --git a/docs/design/system.md b/docs/design/system.md index 37f0369..0f37a4f 100644 --- a/docs/design/system.md +++ b/docs/design/system.md @@ -52,6 +52,7 @@ fingerprint of its file-set, and the review status (Current, Stale, Missing, or It is generated by the `--report` flag and written to a configurable output path. The statuses have the following meanings: + - **Current** — Evidence exists for the current fingerprint and the recorded result is `pass`. - **Stale** — Evidence exists, but it corresponds to an older fingerprint than the current one. - **Missing** — No evidence exists for this review-set. diff --git a/docs/design/validation.md b/docs/design/validation.md index d3646c8..04ff878 100644 --- a/docs/design/validation.md +++ b/docs/design/validation.md @@ -23,27 +23,19 @@ Results are written using the `DemaConsulting.TestResults` library, which suppor both TRX (Visual Studio Test Results) and JUnit XML output formats. The output format is inferred from the file extension of `ResultsFile`. -## Test Naming Convention - -All test names follow the pattern `ReviewMark_MethodOrScenario`, where -`MethodOrScenario` identifies the behavior under test. This convention ensures -test results are identifiable in test dashboards and traceability matrices. - ## Test Coverage The self-validation suite covers the following scenarios: -| Test Name | Scenario | -| --------- | -------- | -| `ReviewMark_Version` | Tool correctly reports its version | -| `ReviewMark_Help` | Tool correctly displays help text | -| `ReviewMark_Plan` | Review Plan is generated correctly for a known configuration | -| `ReviewMark_Report` | Review Report is generated correctly for a known configuration | -| `ReviewMark_IndexScan` | Evidence index is created correctly by scanning a directory | -| `ReviewMark_Enforce` | Tool returns non-zero exit code when enforce mode detects uncovered sets | -| `ReviewMark_WorkingDirectory` | Working directory override resolves paths correctly | -| `ReviewMark_Elaborate` | Elaborate mode expands file lists in generated documents | -| `ReviewMark_Lint` | Lint mode detects configuration errors correctly | +- **Version display**: Tool correctly reports its version +- **Help display**: Tool correctly displays help text +- **Plan generation**: Review Plan is generated correctly for a known configuration +- **Report generation**: Review Report is generated correctly for a known configuration +- **Index scanning**: Evidence index is created correctly by scanning a directory +- **Enforce mode**: Tool returns non-zero exit code when enforce mode detects uncovered review sets +- **Working directory override**: Relative paths are resolved correctly when the working directory is overridden +- **Elaborate mode**: File lists are expanded in generated documents when elaborate mode is active +- **Lint mode**: Configuration errors are detected correctly ## Console Output