diff --git a/.cspell.yaml b/.cspell.yaml index d93083d..2dd7f9d 100644 --- a/.cspell.yaml +++ b/.cspell.yaml @@ -15,6 +15,7 @@ language: en # Project-specific technical terms and tool names words: - Blockquotes + - Postconditions - buildmark - buildtransitive - contentfiles diff --git a/.github/agents/developer.agent.md b/.github/agents/developer.agent.md index a95c562..5f2b988 100644 --- a/.github/agents/developer.agent.md +++ b/.github/agents/developer.agent.md @@ -14,8 +14,12 @@ Perform software development tasks by determining and applying appropriate stand 2. **Read relevant standards** using the selection matrix in AGENTS.md 3. **Pre-flight verification** before making any changes: - List files that will be created, modified, or deleted + - For each file to be **created**, check whether a counterpart exists in the + template (URL in the `# Reference Template` section of `AGENTS.md`). + If one exists, fetch it as the starting point; adjust placeholder names and heading + depth to match the target path before writing the file - For each modified file, identify which companion artifacts need updating - (requirements, design docs, tests, review-sets) + (requirements, design docs, verification docs, tests, review-sets, README.md, user guides) - Include companion artifact updates in the work plan 4. **Execute work** following standards requirements and quality checks 5. **Formatting**: Run `pwsh ./fix.ps1` to silently apply all @@ -31,8 +35,7 @@ Perform software development tasks by determining and applying appropriate stand # Developer Agent Report **Result**: (SUCCEEDED|FAILED) - -## Work Summary +**Report**: `.agent-logs/developer-{subject}-{unique-id}.md` - **Files Modified**: {List of files created/modified/deleted} - **Languages Detected**: {Languages identified} diff --git a/.github/agents/formal-review.agent.md b/.github/agents/formal-review.agent.md index 7dd8e84..cb78ddf 100644 --- a/.github/agents/formal-review.agent.md +++ b/.github/agents/formal-review.agent.md @@ -31,12 +31,14 @@ standards from the selection matrix in AGENTS.md. 1. Download the review checklist from . If the download fails, report the failure rather than proceeding without the template. -2. Use `dotnet reviewmark --elaborate {review-set}` to get the files to review -3. Review all files holistically, checking for cross-file consistency and - compliance with the review checklist -4. Save the populated review checklist to `.agent-logs/reviews/review-report-{review-set}.md`. +2. Run `dotnet reviewmark --elaborate {review-set}`. Read all files listed under + `## Context` first — these are reference material, not under review — then review + all files listed under `## Files` holistically, using the context to understand + the intended role and scope within the broader system, and checking for cross-file + consistency and compliance with the review checklist. +3. Save the populated review checklist to `.agent-logs/reviews/review-report-{review-set}.md`. This directory holds formal review artifacts, not agent logs. -5. Generate a completion report per the AGENTS.md reporting requirements. +4. Generate a completion report per the AGENTS.md reporting requirements. # Report Template @@ -44,6 +46,7 @@ standards from the selection matrix in AGENTS.md. # Formal Review Report **Result**: (SUCCEEDED|FAILED) +**Report**: `.agent-logs/formal-review-{subject}-{unique-id}.md` ## Review Summary diff --git a/.github/agents/implementation.agent.md b/.github/agents/implementation.agent.md index 7cc0352..d94e880 100644 --- a/.github/agents/implementation.agent.md +++ b/.github/agents/implementation.agent.md @@ -28,36 +28,18 @@ The state-transitions include retrying a limited number of times: ## PLANNING State (start) -Call the **explore** agent as a sub-agent (built-in agent type) with: +Call the **planning** agent as a sub-agent (custom agent from `.github/agents/`) with: - **context**: the user's request + any previous quality findings + retry context -- **goal**: produce a verified implementation plan through these steps: - - 1. Investigate the codebase and develop a concrete implementation plan that - addresses the request - 2. **Identify companion artifact deliverables**: for every code change in the - plan, list the requirements files, design documents, and review-set entries - that must be created or updated - traceability must flow requirements → - design → code, so these are mandatory deliverables, not optional extras - 3. Review the plan for assumptions, weaknesses, and gaps - identify up to 5 - key assumptions and rate each as: - - **VERIFIED**: confirmed by codebase evidence - - **LIKELY**: consistent with codebase patterns but not directly confirmed - - **UNVERIFIED**: not confirmed by any evidence - 4. For any assumption rated UNVERIFIED or LIKELY, attempt to resolve it - through additional investigation and revise the plan to address identified - weaknesses - repeat the critique-and-strengthen cycle up to 2 additional - times if unresolved issues remain, but stop as soon as the plan is stable - 5. List up to 5 risks to the implementation - 6. Assess feasibility: can this be implemented in a single development pass? - 7. State a **recommendation**: GO or INCOMPLETE - GO if the plan is sound, or - INCOMPLETE if critical unknowns remain that only the user can resolve - -Once the explore sub-agent finishes: - -- IF recommendation is INCOMPLETE: Transition to REPORT with Result: INCOMPLETE, +- **goal**: produce a verified implementation plan, or a targeted plan to address + the identified quality issues if this is a retry + +Once the planning sub-agent finishes: + +- IF Result is FAILED: Transition to REPORT with Result: FAILED +- IF Result is INCOMPLETE: Transition to REPORT with Result: INCOMPLETE, listing the unknowns and what CAN be implemented once they are resolved -- OTHERWISE (GO): Transition to DEVELOPMENT +- OTHERWISE (SUCCEEDED): Transition to DEVELOPMENT ## DEVELOPMENT State @@ -76,7 +58,8 @@ Once the developer sub-agent finishes: Call the **quality** agent as a sub-agent (custom agent from `.github/agents/`) with: -- **context**: the user's request + development summary + files changed + previous issues (if any) +- **context**: the user's request + development summary + files changed + planning companion artifact table + + previous issues (if any) - **goal**: check the quality of the work performed for any issues Once the quality sub-agent finishes: @@ -92,6 +75,9 @@ Once the quality sub-agent finishes: this agent may report INCOMPLETE when the request cannot be implemented without information only the user can provide. +For full planning details (assumptions, risks, feasibility), read the planning +report file referenced in the planning agent's response. + Generate the completion report using the template below, then save it to `.agent-logs/{agent-name}-{subject}-{unique-id}.md` per the AGENTS.md reporting requirements, and return the summary to the caller. @@ -102,19 +88,20 @@ requirements, and return the summary to the caller. # Implementation Orchestration Report **Result**: (SUCCEEDED|FAILED|INCOMPLETE) -**Final State**: (PLANNING|DEVELOPMENT|QUALITY|REPORT) +**Report**: `.agent-logs/implementation-{subject}-{unique-id}.md` +**Last Active State**: (PLANNING|DEVELOPMENT|QUALITY) **Retry Count**: ## State Machine Execution -- **Planning Results**: {Implementation plan, assumption ratings, risks, and recommendation} +- **Planning Results**: {Planning report path; plan summary and SUCCEEDED/INCOMPLETE/FAILED result} - **Development Results**: {Summary of developer agent results} - **Quality Results**: {Summary of quality agent results} - **State Transitions**: {Log of state changes and decisions} ## Sub-Agent Coordination -- **Explore Agent (Planning)**: {Plan, assumption verdicts, top risks, GO/INCOMPLETE recommendation} +- **Planning Agent**: {Report file path, SUCCEEDED/INCOMPLETE/FAILED result, plan summary} - **Developer Agent**: {Development status and files modified} - **Quality Agent**: {Validation results and compliance status} @@ -123,4 +110,9 @@ requirements, and return the summary to the caller. - **Implementation Success**: {Overall completion status} - **Quality Compliance**: {Final quality validation status} - **Issues Resolved**: {Problems encountered and resolution attempts} + +## Unknowns (only when Result is INCOMPLETE) + +- **Unresolved Questions**: {List each question the user must answer} +- **What Can Proceed**: {Work that can be done without the missing information} ``` diff --git a/.github/agents/lint-fix.agent.md b/.github/agents/lint-fix.agent.md index 549e751..36d3ca1 100644 --- a/.github/agents/lint-fix.agent.md +++ b/.github/agents/lint-fix.agent.md @@ -68,8 +68,7 @@ submission, not during normal development. # Lint Fix Report **Result**: (SUCCEEDED|FAILED) - -## Summary +**Report**: `.agent-logs/lint-fix-{subject}-{unique-id}.md` - **Iterations**: {Number of fix-loop iterations performed} - **Files Modified**: {List of all files changed} diff --git a/.github/agents/planning.agent.md b/.github/agents/planning.agent.md new file mode 100644 index 0000000..20e75ee --- /dev/null +++ b/.github/agents/planning.agent.md @@ -0,0 +1,134 @@ +--- +name: planning +description: Planning agent that investigates the codebase, develops a verified implementation plan, and identifies all companion artifact deliverables. +user-invocable: true +--- + +# Planning Agent + +Investigate the codebase and produce a verified implementation plan with all +companion artifact deliverables. + +## Step 1 — Load Standards + +Read the relevant standards from `.github/standards/` using the selection matrix +in `AGENTS.md` based on the artifact types in scope for the request (requirements, +design, verification, documentation, code). + +## Step 2 — Investigate and Plan + +Read `docs/design/introduction.md` first (if present), then investigate the +codebase to develop a concrete implementation plan: + +- Identify all files to create, modify, or delete +- Describe the change required for each file + +## Step 3 — Identify Companion Artifact Deliverables + +For each planned change, assess the mandatory companion artifacts below (create/update/N/A +with justification): + +- **Requirements** — functional changes require a requirement entry +- **Design Documentation** — new or changed components require design docs +- **Verification Documentation** — new or changed components require verification docs +- **Tests** — functional changes require test coverage +- **Review Sets** — changes to the software item hierarchy (units or subsystems + added, removed, or reorganized) require review-set updates +- **README.md** — user-facing changes require README updates +- **User Guide** — user-facing features require user guide updates + +## Step 4 — Critique and Strengthen + +Identify up to 5 key assumptions and rate each: + +- **VERIFIED**: confirmed by codebase evidence +- **LIKELY**: consistent with codebase patterns but not directly confirmed +- **UNVERIFIED**: not confirmed by any evidence + +For UNVERIFIED or LIKELY assumptions, investigate further and revise the plan. +Repeat up to 2 more times, stopping when the plan is stable. + +## Step 5 — Risk Assessment + +List up to 5 risks with a brief mitigation for each. + +## Step 6 — Feasibility Assessment + +State whether this can be implemented in a single development pass and any +preconditions that affect feasibility. + +## Step 7 — Recommendation + +- **SUCCEEDED** — the plan is sound and the developer agent can proceed +- **INCOMPLETE** — critical unknowns remain that only the user can resolve; + list each unknown explicitly +- **FAILED** — investigation could not produce a viable plan + +# REPORT Phase + +Save the full analysis to `.agent-logs/planning-{subject}-{unique-id}.md` per +the AGENTS.md reporting requirements. + +Then respond to the caller with ONLY the lean structured summary below. + +# Report Template + +```markdown +# Planning Report + +**Result**: (SUCCEEDED|INCOMPLETE|FAILED) +**Request Summary**: {Brief restatement of the task as understood} +**Report**: `.agent-logs/planning-{subject}-{unique-id}.md` + +## Implementation Plan + +| File | Action | Description | +|------|--------|-------------| +| {path} | create/modify/delete | {what changes and why} | + +## Companion Artifact Deliverables + +| Category | File | Action | +|----------|------|--------| +| Requirements | {path} | create/update/N/A — {justification} | +| Design Documentation | {path} | create/update/N/A — {justification} | +| Verification Documentation | {path} | create/update/N/A — {justification} | +| Tests | {path} | create/update/N/A — {justification} | +| Review Sets | {path} | create/update/N/A — {justification} | +| README.md | {path} | create/update/N/A — {justification} | +| User Guide | {path} | create/update/N/A — {justification} | + +## Assumption Analysis + +| # | Assumption | Rating | Resolution | +|---|-----------|--------|------------| +| 1 | {assumption} | VERIFIED/LIKELY/UNVERIFIED | {resolution or N/A} | + +## Risk Assessment + +1. **[severity]** {risk} — {mitigation} + +## Feasibility Assessment + +{Single-pass or not, and why. Any preconditions.} + +## Unknowns + +{Only present when Result is INCOMPLETE. List each question the user must +resolve before implementation can proceed.} +``` + +# Lean Structured Response (returned to caller) + +```markdown +**Result**: (SUCCEEDED|INCOMPLETE|FAILED) +**Report**: `.agent-logs/planning-{subject}-{unique-id}.md` + +**Plan**: +{Repeat the Implementation Plan table} + +**Companion Artifacts**: +{Repeat the Companion Artifact Deliverables table} + +**Unknowns**: {Only if INCOMPLETE — list questions for the user} +``` diff --git a/.github/agents/quality.agent.md b/.github/agents/quality.agent.md index da467d4..26fd251 100644 --- a/.github/agents/quality.agent.md +++ b/.github/agents/quality.agent.md @@ -13,14 +13,23 @@ Grade and validate software development work by ensuring compliance with project 1. **Analyze the task request AND completed work** to determine scope: identify which artifact categories were changed, and which *should have been changed* given the task - new user-visible features always require requirements, - design, and review-set coverage regardless of whether those files were touched; - test-only additions (corner-case tests, defensive boundary tests, regression - tests) do not require a corresponding requirement + design, verification docs, and README/user guide updates regardless of + whether those files were touched; Review Sets are always in scope when + the software item hierarchy changes (units or subsystems added, removed, or + reorganized); test-only additions (corner-case tests, defensive boundary + tests, regression tests) do not require a corresponding requirement; if a + planning companion artifact table is provided in context, cross-reference it + — any artifact listed as create/update must be covered in the evaluation and + FAIL if the artifact was not produced 2. **Read relevant standards** using the selection matrix in AGENTS.md 3. **Evaluate all in-scope categories** - N/A only when the task genuinely cannot affect a category; if the task introduces new user-visible features or - structural changes then Requirements, Design Documentation, and Review - Management are always in scope and FAIL if the artifacts were not updated + structural changes then Requirements, Design Documentation, and Verification + Documentation are always in scope and FAIL if the artifacts were not updated; + Documentation (README/user guide) is always in scope for user-facing changes + and FAIL if not updated; Review Sets are always in scope when the + software item hierarchy changes (units or subsystems added, removed, or + reorganized) and FAIL if review-sets were not updated 4. **Validate tool compliance** using ReqStream, ReviewMark, and build tools 5. **Generate focused quality report** per the AGENTS.md reporting requirements - save to `.agent-logs/{agent-name}-{subject}-{unique-id}.md` and return the summary to the caller @@ -36,6 +45,7 @@ For each checklist item in the template below, record as `(PASS|FAIL|N/A) - {one # Quality Assessment Report **Result**: (SUCCEEDED|FAILED) +**Report**: `.agent-logs/quality-{subject}-{unique-id}.md` **Overall Grade**: (PASS|FAIL) ## Required Fixes (only when Result is FAILED) @@ -50,25 +60,17 @@ Priority-ordered list of issues that MUST be resolved for the next retry: - **Evaluated**: {List sections assessed and why} - **Skipped**: {One-line per skipped section with reason, e.g., "Design - Documentation: N/A - no design files modified"} + Documentation: N/A - no component behavior, structure, or interface changed"} ## Requirements Compliance: (PASS|FAIL|N/A) -- Were requirements updated to reflect functional changes? -- Were new requirements created for new features? -- Do requirement IDs follow semantic naming standards? -- Do requirement files follow kebab-case naming convention? -- Are requirement files organized under `docs/reqstream/` with proper folder structure? -- Are OTS requirements properly placed in `docs/reqstream/ots/` subfolder? -- Were source filters applied appropriately for platform-specific requirements? -- Is requirements traceability maintained to tests? +- Were requirements created/updated for all functional changes? +- Were source filters applied for platform-specific requirements? +- Is forward traceability from requirements to verification artifacts preserved? ## Design Documentation Compliance: (PASS|FAIL|N/A) -- Were design documents updated for architectural changes? -- Were new design artifacts created for new components? -- Do design folder names use kebab-case convention matching source structure? -- Are design files properly named ({subsystem-name}.md, {unit-name}.md patterns)? +- Were design artifacts created/updated for all new or changed components? - Is `docs/design/introduction.md` present with required Software Structure section? - Are design decisions documented with rationale? - Is system/subsystem/unit categorization maintained? @@ -76,55 +78,57 @@ Priority-ordered list of issues that MUST be resolved for the next retry: ## Code Quality Compliance: (PASS|FAIL|N/A) -- Are language-specific standards followed (from applicable standards files)? -- Are quality checks from standards files satisfied? -- Is code properly categorized (system/subsystem/unit/OTS)? -- Is appropriate separation of concerns maintained? -- Was language-specific build tooling executed and passing? +- Do language-specific quality checks from loaded standards pass? +- Is code properly categorized (system/subsystem/unit/OTS/Shared Package)? +- Does the build pass? ## Testing Compliance: (PASS|FAIL|N/A) - Were tests created/updated for all functional changes? - Is test coverage maintained for all requirements? -- Are testing standards followed (AAA pattern, etc.)? -- Do tests respect software item hierarchy boundaries (System/Subsystem/Unit scope)? +- Do tests respect software item hierarchy boundaries? - Are cross-hierarchy test dependencies documented in design docs? -- Does test categorization align with code structure? -- Do all tests pass without failures? +- Do all tests pass? -## Review Management Compliance: (PASS|FAIL|N/A) +## Verification Documentation Compliance: (PASS|FAIL|N/A) -- Were review-sets updated for structural changes (new/deleted systems, subsystems, or units)? -- Do file patterns follow include-then-exclude approach? +- Were verification documents created/updated for all new or changed components? +- Do verification documents include all mandatory sections (Verification Approach, Test Environment, + Acceptance Criteria, Test Scenarios)? +- Is requirements-to-test coverage tracked via the ReqStream trace matrix (not embedded in verification docs)? + +## Review Sets Compliance: (PASS|FAIL|N/A) + +- Were review-sets updated for structural changes? - Is review scope appropriate for change magnitude? -- Was ReviewMark tooling executed and passing? -- Were review artifacts generated correctly? +- Does ReviewMark pass? ## Documentation Compliance: (PASS|FAIL|N/A) -- Was README.md updated for user-facing changes? -- Were user guides updated for feature changes? +- Were README.md and user guides updated for user-facing changes? - Does API documentation reflect code changes? - Was compliance documentation generated? -- Does documentation follow standards formatting? -- Is documentation organized under `docs/` following standard folder structure? -- Do Pandoc collections include proper `introduction.md` with Purpose and Scope sections? - Are auto-generated markdown files left unmodified? -- Do README.md files use absolute URLs and include concrete examples? -- Is documentation integrated into ReviewMark review-sets for formal review? +- Is documentation integrated into ReviewMark review-sets? ## Software Item Completeness: (PASS|FAIL|N/A) +- Load `software-items.md` before evaluating this section. + - Does every identified software unit have its own requirements file? - Does every identified software unit have its own design document? - Does every identified subsystem have its own requirements file? - Does every identified subsystem have its own design document? +## Repository Structure Compliance: (PASS|FAIL|N/A) + +- Load `repository-map.md` from the template URL in the `# Reference Template` + section of `AGENTS.md` before evaluating this section. + +- Are parallel artifact trees in sync (reqstream/design/verification/src/test)? +- Does the repository conform to the template `repository-map.md`? + ## Process Compliance: (PASS|FAIL|N/A) -- Was Continuous Compliance workflow followed? -- Did all quality gates execute successfully? -- Were appropriate tools used for validation? -- Were standards consistently applied across work? -- Was compliance evidence generated and preserved? +- Was compliance evidence (test results, review artifacts, generated docs) generated and preserved? ``` diff --git a/.github/agents/software-architect.agent.md b/.github/agents/software-architect.agent.md index 494568d..de5efa2 100644 --- a/.github/agents/software-architect.agent.md +++ b/.github/agents/software-architect.agent.md @@ -13,7 +13,7 @@ Interview the user and produce evolving architecture documentation with prioriti # Standards Read `.github/standards/software-items.md` before starting. Use its definitions -(Software Package, System, Subsystem, Unit, OTS) as vocabulary throughout. +(Software Package, System, Subsystem, Unit, OTS, Shared Package) as vocabulary throughout. # Approach diff --git a/.github/agents/template-sync.agent.md b/.github/agents/template-sync.agent.md new file mode 100644 index 0000000..df4d488 --- /dev/null +++ b/.github/agents/template-sync.agent.md @@ -0,0 +1,140 @@ +--- +name: template-sync +description: Audits or synchronizes repository files against the canonical template. + Supports four modes - Audit, Sync, Scaffold, and Recreate. +user-invocable: true +--- + +# Template Sync Agent + +This agent is an orchestrator supporting four modes: + +- **Audit** - report structural deviations; no changes +- **Sync** - patch missing sections into existing files +- **Scaffold** - create files that do not yet exist; skip existing files +- **Recreate** - rebuild existing files from the template, migrating old content + +Read the template URL and `repository-map.md` from the `# Reference Template` +section in `AGENTS.md`, then map the requested scope onto the work groups below. +Delegate each group to a sub-agent. + +# Work Groups + +- **Root config files** - all non-collection files at the repository root +- **One group per flat `docs/` folder** - e.g. `docs/build_notes/`, `docs/user_guide/` +- **One group for root files in each of `docs/design/`, `docs/verification/`, + `docs/reqstream/`** - e.g. `docs/design/introduction.md` — separate from the + system subtrees beneath them +- **One group per system subtree** in `docs/design/`, `docs/verification/`, `docs/reqstream/` - + each subtree and all its descendants is one group + +# Orchestration + +For Audit mode, call an **explore** sub-agent (built-in) per group. +For Sync, Scaffold, and Recreate modes, call a **general-purpose** sub-agent (built-in) per group. + +For each group intersecting the requested scope, call the appropriate sub-agent with: + +- **context**: + - Group scope and template URL from the `# Reference Template` section in `AGENTS.md` + - Applicable standards from the `# Standards Application` matrix in `AGENTS.md` + for the file types in the group scope + - Project-specific names substitute for placeholders at matching path depth + (e.g. `MySystem` → `{SystemName}`, `my-system` → `{system-name}`) + - For files within `{system-name}/` subtrees in `docs/design/`, `docs/verification/`, + and `docs/reqstream/`: consult `docs/design/introduction.md` to determine whether + each item is a subsystem or unit, then select the appropriate template + (`subsystem-name.*` or `unit-name.*`) regardless of the item's folder depth — + do not infer item type from path depth alone + - If a file has no template counterpart, skip it and report it as + "No template found" — this is not a failure + - If a file appears in `repository-map.md` but its template cannot be fetched, + report Result: FAILED and list the affected files +- **goal**: + - Based on the given mode: + - **Audit** - fetch each template counterpart; compare headings; report missing + sections and depth mismatches; no changes + - **Sync** - as Audit, then insert each missing section; run `pwsh ./fix.ps1` + - **Scaffold** - fetch `repository-map.md` from the template URL in `AGENTS.md` + to identify files that should exist but don't; for each, fetch the template, + populate all sections, write the file; run `pwsh ./fix.ps1` + - **Recreate** - fetch the template and use it as the blueprint for a + freshly authored document: + - Work through the template section by section; for each section, find + any `TEMPLATE-DIRECTIVE` blocks (both `` + in markdown and `# ` in YAML) — execute + each directive (read specified standards, apply structural guidance, + substitute content), then **remove the directive block entirely** from + the output; gather the relevant technical details from all available + sources — the old file, README, related docs, sibling files, and any + other repo context — to populate that section correctly; the old file's + structure and headings are irrelevant; only its factual content is mined + as a source + - **Gap-check**: after all template sections are filled, scan the old + file once more for any technical information not yet captured; if + found, preserve it by appending new relevant sections at the end + - **Before writing**: do a mandatory self-check — for every section that + has a `TEMPLATE-DIRECTIVE` block in the template, explicitly state what + format the directive requires, then verify the drafted content matches + that format exactly (e.g. if the directive says "no sub-headings", + confirm there are no `###` headings inside that section; if it says + "bold-name paragraph blocks", confirm each entry is `**Name**: prose` + with no sub-heading); fix any mismatches before writing the file + - Write the rebuilt file; run `pwsh ./fix.ps1` + - When writing any section: `TEMPLATE-DIRECTIVE` blocks are directives — + execute them (read specified standards, apply structural guidance, substitute + content) and **remove the block entirely** from the written file; inline + `TODO:` placeholders in YAML string values (e.g. `title:`, `justification:`) + are content placeholders — always resolve them to real content; infer from + README, related files, sibling docs, and path; if confident write directly; + if ambiguous, **do not ask interactively** — return the unresolved questions + in the result so the orchestrator can ask the user and re-invoke; never leave + a TODO or TEMPLATE-DIRECTIVE in the output unless the user explicitly requests it + - Return results in this format for each file in the group: + + ```markdown + ### {file-path} + + - **Template**: {template path or "not found"} + - **Missing sections**: {list or "none"} + - **Heading depth issues**: {list or "none"} + - **Content format issues**: {list or "none"} *(Recreate only)* + - **Action**: (Reported | Sections added | Created | Rebuilt | No template found) + - **Unresolved Questions**: {list or "none"} + ``` + +If any sub-agent returns unresolved questions, collect them, ask the user, then +re-invoke the affected sub-agent(s) with the answers before assembling the final report. +If questions remain unresolved after asking the user, report Result: INCOMPLETE. + +Collect sub-agent results and assemble the final report. + +# Report Template + +```markdown +# Template Sync Report + +**Result**: (SUCCEEDED|FAILED|INCOMPLETE) +**Report**: `.agent-logs/template-sync-{subject}-{unique-id}.md` +**Mode**: (Audit|Sync|Scaffold|Recreate) + +## Files + +### {file-path} + +- **Template**: {template path} +- **Missing sections**: {list or "none"} +- **Heading depth issues**: {list or "none"} +- **Content format issues**: {list of sections where intra-section content did not + match the template comment's prescribed format, or "none"} *(Recreate only)* +- **Action**: (Reported | Sections added | Created | Rebuilt | No template found) +- **Unresolved Questions**: {list or "none"} + +## Summary + +- **Conformant**: {count} | **Deviations**: {count} | **Updated**: {count} + +## Unknowns (only when Result is INCOMPLETE) + +- **Unresolved Questions**: {List each placeholder or ambiguity the user must resolve} +``` diff --git a/.github/standards/coding-principles.md b/.github/standards/coding-principles.md index 9e67fbb..6797c61 100644 --- a/.github/standards/coding-principles.md +++ b/.github/standards/coding-principles.md @@ -3,11 +3,6 @@ name: Coding Principles description: Follow these standards when developing any software code. --- -# Coding Principles Standards - -This document defines universal coding principles and quality standards for software development within -Continuous Compliance environments. - # Core Principles ## Literate Coding @@ -20,10 +15,9 @@ All code MUST follow literate programming principles: matches design intent without reading the full codebase - **Logical Separation**: Complex functions use block comments to separate and describe logical steps within the implementation -- **Full Symbol Documentation**: ALL symbols have comprehensive documentation - because reviewers and auditors must verify every implementation detail, not - just the public interface - access-level specifics (public, protected, - private, internal, etc.) vary by language; see the language-specific standard +- **Full Symbol Documentation**: ALL symbols have comprehensive documentation — + not just the public interface, because reviewers and auditors must verify every + implementation detail. Access-level specifics vary by language; see the language-specific standard. - **Clarity Over Cleverness**: Code should be immediately understandable by team members ## API Documentation @@ -79,13 +73,13 @@ interface correctly without reading the implementation: ## Universal Anti-Patterns -- **Skip Literate Coding**: Don't skip literate programming comments - they are required for maintainability -- **Ignore Compiler Warnings**: Don't ignore compiler warnings - they exist for quality enforcement +- **Skip Literate Coding**: Don't skip literate programming comments +- **Ignore Compiler Warnings**: Don't ignore compiler warnings - **Hidden Dependencies**: Don't create untestable code with hidden dependencies - **Hidden Functionality**: Don't implement functionality without requirement traceability because untraced functionality cannot be validated during audits - **Monolithic Functions**: Don't write monolithic functions with multiple responsibilities -- **Overcomplicated Solutions**: Don't make solutions more complex than necessary - favor simplicity and clarity +- **Overcomplicated Solutions**: Don't make solutions more complex than necessary - **Premature Optimization**: Don't optimize for performance before establishing correctness - **Copy-Paste Programming**: Don't duplicate logic - extract common functionality into reusable components - **Magic Numbers**: Don't use unexplained constants - either name them or add clear comments diff --git a/.github/standards/cpp-language.md b/.github/standards/cpp-language.md new file mode 100644 index 0000000..c424bf5 --- /dev/null +++ b/.github/standards/cpp-language.md @@ -0,0 +1,100 @@ +--- +name: C++ Language +description: Follow these standards when developing C++ source code. +globs: ["**/*.cpp", "**/*.hpp", "**/*.h"] +--- + +# Required Standards + +Read these standards first before applying this standard: + +- **`coding-principles.md`** - Universal coding principles and quality gates + +# File Organization + +C++ projects use two parallel top-level folders — `include/` (public API) and +`src/` (implementation) — both mirroring the same system/subsystem/unit hierarchy +(test layout is covered in `cpp-testing.md`): + +```text +include/ +└── {system_name}/ + └── {subsystem_name}/ + └── {unit_name}.hpp # public API - installed with the package + +src/ +└── {system_name}/ + └── {subsystem_name}/ + ├── {unit_name}.cpp # implementation + └── {unit_name}_impl.hpp # internal header - not part of the public API +``` + +Subsystems may nest to any depth: `{system_name}[/{subsystem_name}...]/{unit_name}.hpp/cpp`. + +Protect every header with `#pragma once`. + +# Naming and Style Conventions + +- **Symbols**: `snake_case` for all identifiers - variables, functions, types, and + namespaces - to align with STL naming +- **Bracing**: 4-space Allman style - opening brace on its own line +- **Data objects**: use `struct` for passive data; may include simple constructors + or helper methods but must not encapsulate invariants (use `class` for those) + +# API Documentation and Literate Coding Example + +Use `///` C++ Doxygen line comments. + +```cpp +/// @brief Converts a raw sensor reading into a validated measurement. +/// +/// Clamping is preferred over throwing on out-of-range values because +/// sensor drift at range boundaries is expected; clamping produces a usable +/// result where rejection would discard valid near-boundary readings. +/// Stateless and thread-safe; the calibration profile is read but never modified. +/// +/// @param reading Raw sensor value. Must be finite (NaN and infinities are rejected). +/// @param calibration Calibration profile providing offset and range. +/// @returns Corrected value clamped to [calibration.minimum, calibration.maximum]. +/// @throws std::invalid_argument When reading is NaN or infinite. +double process_reading(double reading, const calibration_profile& calibration) +{ + // Reject invalid inputs before any calculation - non-finite readings cannot be + // corrected, and the calibration profile provides no offset or range to apply + if (!std::isfinite(reading)) + throw std::invalid_argument("reading must be a finite number"); + + // Apply the calibration offset to convert raw counts to physical units + double corrected = reading + calibration.offset; + + // Clamp to the operational range so consumers can rely on the documented contract + return std::clamp(corrected, calibration.minimum, calibration.maximum); +} +``` + +Key qualities demonstrated above: + +- **`@brief`** is a concise one-liner explaining *what* the function does +- **Extended description** carries the extended intent - *why* it exists, design decisions, + thread-safety, and side-effect disclosures +- **`@param` tags** state constraints so callers know what is valid without reading the body +- **`@returns`** documents the boundary guarantee so consumers can rely on the contract +- **`@throws`** names every thrown exception and the condition that triggers it +- **Inline block comments** follow the Literate Coding principles from + `coding-principles.md`, separating logical steps so reviewers can verify each + step against design intent + +# Code Formatting + +Apply clang-format using the repository `.clang-format` configuration: + +- **Format file**: `clang-format -i my_file.cpp` +- Run `pwsh ./fix.ps1` to apply across the project + +# Quality Checks + +- [ ] Zero compiler warnings (`-Wall -Wextra -Werror`) +- [ ] Doxygen documentation complete on all symbols +- [ ] clang-format applied (run `pwsh ./fix.ps1`) +- [ ] All headers protected with `#pragma once` +- [ ] No raw owning pointers - use `std::unique_ptr` or `std::shared_ptr` diff --git a/.github/standards/cpp-testing.md b/.github/standards/cpp-testing.md new file mode 100644 index 0000000..87431d2 --- /dev/null +++ b/.github/standards/cpp-testing.md @@ -0,0 +1,72 @@ +--- +name: C++ Testing +description: Follow these standards when developing C++ tests. +globs: ["**/test/**/*.cpp", "**/tests/**/*.cpp", "**/*_test.cpp", "**/*_tests.cpp"] +--- + +# Required Standards + +Read these standards first before applying this standard: + +- **`testing-principles.md`** - Universal testing principles and dependency boundaries +- **`cpp-language.md`** - C++ language development standards + +# File Organization + +Test files mirror the `src/` tree under `test/`, with `_tests` appended to the +system folder name and each test file name: + +```text +test/ +└── {system_name}_tests/ + └── {subsystem_name}/ + └── {unit_name}_tests.cpp # unit tests for src/{system_name}[/{subsystem_name}...]/{unit_name}.cpp +``` + +# Package Reference + +Use `GTest` and `GMock` from the CMake `GTest` package. Link test targets with +`GTest::gtest_main` and `GTest::gmock`. + +# Test Style + +Test names appear in requirements traceability matrices - use the hierarchical +naming pattern with snake_case, split across the gtest suite and test name: + +- **System tests**: `TEST({system_name}_test, {functionality}_{scenario}_{expected_behavior})` +- **Subsystem tests**: `TEST({subsystem_name}_test, {functionality}_{scenario}_{expected_behavior})` +- **Unit tests**: `TEST({class_name}_test, {method_name}_{scenario}_{expected_behavior})` +- Use `TEST_F` with a fixture class when shared setup is needed + +```cpp +/// @brief Validates that an invalid email format throws std::invalid_argument. +TEST(user_validator_test, validate_email_invalid_format_throws) +{ + // Arrange: create a validator with default configuration + user_validator validator; + + // Act / Assert: email with no domain throws + EXPECT_THROW(validator.validate_email("not-an-email"), std::invalid_argument); +} +``` + +# gtest/gmock Specifics + +These are non-obvious behaviors that differ from common assumptions: + +- **`EXPECT_*` vs `ASSERT_*`**: `ASSERT_*` aborts the test immediately; prefer + `EXPECT_*` for independent checks to surface all failures in one run +- **`EXPECT_CALL` placement**: all mock expectations must be set up in Arrange, + before the Act step - expectations placed after the call under test are never triggered +- **`NiceMock` vs `StrictMock`**: bare mocks warn on unexpected calls; `NiceMock` + silences them; `StrictMock` makes them failures - choose deliberately + +# Quality Checks + +- [ ] All tests follow AAA pattern with descriptive section comments +- [ ] Test suite and test names follow hierarchical naming pattern above +- [ ] Each test verifies single, specific behavior (no shared state between tests) +- [ ] Both success and failure scenarios covered including edge cases +- [ ] External dependencies mocked with GMock +- [ ] Tests linked to requirements with source filters where needed +- [ ] Test results generated in JUnit XML format for ReqStream compatibility (`--gtest_output=xml`) diff --git a/.github/standards/csharp-language.md b/.github/standards/csharp-language.md index 6df39cd..ec05a25 100644 --- a/.github/standards/csharp-language.md +++ b/.github/standards/csharp-language.md @@ -12,9 +12,6 @@ Read these standards first before applying this standard: # API Documentation and Literate Coding Example -The example below demonstrates good XmlDoc API documentation combined with -literate coding comments. - ```csharp /// /// Converts a raw sensor reading into a validated measurement ready for downstream consumers. diff --git a/.github/standards/csharp-testing.md b/.github/standards/csharp-testing.md index 181de02..1f93b72 100644 --- a/.github/standards/csharp-testing.md +++ b/.github/standards/csharp-testing.md @@ -66,8 +66,6 @@ These are non-obvious v3 behaviors that differ from v2 or common assumptions: # Quality Checks -Before submitting C# tests, verify: - - [ ] All tests follow AAA pattern with clear section comments - [ ] Test names follow hierarchical naming pattern above - [ ] Each test verifies single, specific behavior (no shared state between tests) diff --git a/.github/standards/design-documentation.md b/.github/standards/design-documentation.md index 3b448f3..e5b7bf9 100644 --- a/.github/standards/design-documentation.md +++ b/.github/standards/design-documentation.md @@ -4,196 +4,112 @@ description: Follow these standards when creating design documentation. globs: ["docs/design/**/*.md"] --- -# Design Documentation Standards - -This document defines standards for design documentation within Continuous -Compliance environments, extending the general technical documentation -standards with specific requirements for software design artifacts. - -## Required Standards - -Read these standards first before applying this standard: +# Required Standards - **`technical-documentation.md`** - General technical documentation standards -- **`software-items.md`** - Software categorization (System/Subsystem/Unit/OTS) - -# Core Principles - -Design documentation serves as the bridge between requirements and -implementation, providing detailed technical specifications that enable: - -- **Formal Code Review**: Reviewers can verify implementation matches design -- **Compliance Evidence**: Auditors can trace requirements through design to code -- **Maintenance Support**: Developers can understand system structure and interactions -- **Quality Assurance**: Testing teams can validate against detailed specifications - -# Required Structure and Documents +- **`software-items.md`** - Software categorization (System/Subsystem/Unit/OTS/Shared Package) -Design documentation must be organized under `docs/design/` with folder structure -mirroring source code organization because reviewers need clear navigation from -design to implementation: +# Folder Structure ```text docs/design/ -├── introduction.md # Design overview with software structure -└── {system-name}/ # System-level design folder (one per system) - ├── {system-name}.md # System-level design documentation - ├── {subsystem-name}/ # Subsystem (kebab-case); may nest recursively - │ ├── {subsystem-name}.md # Subsystem overview and design - │ ├── {child-subsystem}/ # Child subsystem (same structure as parent) - │ └── {unit-name}.md # Unit-level design documents - └── {unit-name}.md # Top-level unit design documents (if not in subsystem) +├── introduction.md # heading depth # +├── {system-name}.md # heading depth # +├── {system-name}/ +│ ├── {subsystem-name}.md # heading depth ## +│ ├── {subsystem-name}/ +│ │ └── {unit-name}.md # heading depth ### +│ └── {unit-name}.md # heading depth ## +├── ots.md # heading depth # (if OTS items exist) +├── ots/ +│ └── {ots-name}.md # heading depth ## +├── shared.md # heading depth # (if Shared Packages exist) +└── shared/ + └── {package-name}.md # heading depth ## ``` -## introduction.md (MANDATORY) +All sections in every file are mandatory; write "N/A - {justification}" rather than removing any. +Determine subsystem vs. unit classification from `docs/design/introduction.md` — folder depth does not determine classification. +Do not record version numbers anywhere in design documentation — version information is managed in SBOMs. -The `introduction.md` file serves as the design entry point and MUST include -these sections because auditors need clear scope boundaries and architectural -overview: +# introduction.md (MANDATORY) -### Purpose Section +Must include: -Clear statement of the design document's purpose, audience, and regulatory -or compliance drivers. +- **Purpose**: audience and compliance drivers +- **Scope**: items covered and explicitly excluded (no test projects) +- **Software Structure**: text tree showing all Systems/Subsystems/Units/OTS/Shared items +- **Folder Layout**: text tree showing source folder structure +- **Companion Artifact Structure**: parallel paths for requirements, design, verification, source, tests +- **References** _(if applicable)_: external standards or specifications - only in `introduction.md` -### Scope Section +# System Design (MANDATORY) -Define what software items are covered and what is explicitly excluded. -Design documentation must NOT include test projects, test classes, or test -infrastructure because design documentation documents the architecture of -shipping product code, not ancillary content used to validate it. +Create `{system-name}.md` (`#` heading) and `{system-name}/` folder: -### Software Structure Section (MANDATORY) +- **Architecture**: software items, relationships, and collaboration +- **External Interfaces**: name, direction, format, constraints +- **Dependencies**: OTS and Shared Packages used; cross-reference their design docs +- **Risk Control Measures**: segregation required for risk control (IEC 62304 §5.3.3) +- **Data Flow**: inputs to outputs +- **Design Constraints**: platform, performance, security, regulatory -Include a text-based tree diagram showing the software organization across -System, Subsystem, and Unit levels. Agents MUST read `software-items.md` -to understand these classifications before creating this section. +# Subsystem Design (MANDATORY) -Example format: +Place `{subsystem-name}.md` in the **parent** folder; create `{subsystem-name}/` for children: -```text -Project1Name (System) -├── ComponentA (Subsystem) -│ ├── SubComponentP (Subsystem) -│ │ └── ClassW (Unit) -│ ├── ClassX (Unit) -│ └── ClassY (Unit) -├── ComponentB (Subsystem) -│ └── ClassZ (Unit) -└── UtilityClass (Unit) - -Project2Name (System) -└── HelperClass (Unit) -``` +- **Overview**: responsibility, boundaries, contained units +- **Interfaces**: what it exposes and consumes +- **Design**: how internal units collaborate -### Folder Layout Section (MANDATORY) +# Unit Design (MANDATORY) -Include a text-based tree diagram showing how the source code folders -mirror the software structure, with file paths and brief descriptions. +Place `{unit-name}.md` in the **parent** folder: -Example format: +- **Purpose**: single responsibility +- **Data Model**: fields, properties, types, invariants (IEC 62304 §5.4.2) +- **Key Methods**: name, purpose, algorithm, preconditions, postconditions, parameter types +- **Error Handling**: detection and handling; what is propagated vs. handled locally +- **Dependencies**: other units, subsystems, OTS items, and shared packages used +- **Callers**: units or subsystems that call or consume this unit -```text -src/Project1Name/ -├── ComponentA/ -│ ├── SubComponentP/ -│ │ └── ClassW.cs - Specialized processing engine -│ ├── ClassX.cs - Core business logic handler -│ └── ClassY.cs - Data validation service -├── ComponentB/ -│ └── ClassZ.cs - Integration interface -└── UtilityClass.cs - Common utility functions - -src/Project2Name/ -└── HelperClass.cs - Helper functions -``` +# OTS Integration Design (when OTS items exist) -### References Section (RECOMMENDED) +Create `docs/design/ots.md` (`#` heading) covering the overall OTS integration strategy. -If the design references external documents (standards, specifications), include -a `## References` section in `introduction.md`. This is the **only** place in the -design document collection where a References section should appear - do not add -one to any other design file. +For each OTS item, create `docs/design/ots/{ots-name}.md` (`##` heading) with sections: -### Companion Artifact Structure (RECOMMENDED) +- **Purpose**: why chosen and what it provides to the local system +- **Features Used**: which specific features, APIs, or capabilities are consumed +- **Integration Pattern**: how it is consumed; initialization, configuration, disposal requirements -Include a brief note explaining that each software item has parallel artifacts -across the repository, so agents and reviewers can navigate from any one -artifact to all related files: +# Shared Package Integration Design (when Shared Packages exist) -Example format: +Create `docs/design/shared.md` (`#` heading) covering the overall consumption strategy. -```text -Each software item in the structure above has corresponding artifacts in -parallel directory trees: - -- Requirements: `docs/reqstream/{system}/.../{item}.yaml` (kebab-case) -- Design docs: `docs/design/{system}/.../{item}.md` (kebab-case) -- Verification design: `docs/verification/{system}/.../{item}.md` (kebab-case) -- Source code: `src/{System}/.../{Item}.{ext}` (cased per language - see `software-items.md`) -- Tests: `test/{System}.Tests/.../{Item}Tests.{ext}` (cased per language - see `software-items.md`) -- Review-sets: defined in `.reviewmark.yaml` -``` +For each Shared Package, create `docs/design/shared/{package-name}.md` (`##` heading) with sections: -## System Design Documentation (MANDATORY) - -For each system identified in the repository: - -- Create a kebab-case folder matching the system name -- Include `{system-name}.md` with system-level design documentation such as: - - System architecture and major components - - External interfaces and dependencies - - Data flow and control flow - - System-wide design constraints and decisions - - Integration patterns and communication protocols - -## Subsystem and Unit Design Documents - -For each subsystem identified in the software structure: - -- Create a kebab-case folder matching the subsystem name (enables automated tooling) -- Include `{subsystem-name}.md` with subsystem overview and design -- Include unit design documents for ALL units within the subsystem - -For every unit identified in the software structure: - -- Document data models, algorithms, and key methods -- Describe interactions with other units -- Include sufficient detail for formal code review -- Place in appropriate subsystem folder or at design root level - -# Software Items Integration (CRITICAL) - -Read `software-items.md` before creating design documentation - correct -System/Subsystem/Unit categorization is required for software structure -diagrams and folder layout. +- **Advertised Features Consumed**: which features the local system relies on +- **Integration Pattern**: how the package is referenced, initialized, and consumed +- **Assumptions**: any assumptions the local system makes about the package's behavior # Writing Guidelines -Design documentation must be technical and specific because it serves as the -implementation specification for formal code review: - -- **Implementation Detail**: Provide sufficient detail for code review and implementation -- **Architectural Clarity**: Clearly define component boundaries and interfaces -- **Traceability**: Link to requirements where applicable using ReqStream patterns -- **Verbal Cross-References**: Reference other parts of the design by name (e.g., - "See *Parser Design* for more details") - do not use markdown hyperlinks, which - break in compiled PDFs - -# Mermaid Diagram Integration - -Use Mermaid diagrams to supplement text descriptions (diagrams must not replace text content). +- Use Mermaid diagrams to supplement (not replace) text +- Use verbal cross-references ("see _Parser Design_") - not markdown hyperlinks (break in PDF) +- Provide sufficient detail for formal code review # Quality Checks -Before submitting design documentation, verify: - -- [ ] `introduction.md` includes both Software Structure and Folder Layout sections -- [ ] Software structure correctly categorizes items as System/Subsystem/Unit per `software-items.md` -- [ ] Folder layout mirrors software structure organization -- [ ] Design documents provide sufficient detail for code review -- [ ] System documentation provides comprehensive system-level design -- [ ] Subsystem documentation folders use kebab-case names while mirroring source subsystem names and structure -- [ ] All documents follow technical documentation formatting standards -- [ ] Content is current with implementation and requirements -- [ ] Documents are integrated into ReviewMark review-sets for formal review +- [ ] `introduction.md` includes Software Structure, Folder Layout, and Companion Artifact Structure +- [ ] Software structure correctly categorizes items per `software-items.md` +- [ ] Each file's heading depth matches its folder depth +- [ ] All folders use kebab-case mirroring source structure +- [ ] System design includes all mandatory sections (Architecture, External Interfaces, Dependencies, + Risk Control Measures, Data Flow, Design Constraints) +- [ ] Subsystem design includes all mandatory sections (Overview, Interfaces, Design) +- [ ] Unit design includes all mandatory sections (Purpose, Data Model, Key Methods, Error Handling, Dependencies, Callers) +- [ ] Non-applicable mandatory sections contain "N/A - {justification}" +- [ ] `docs/design/ots.md` and `docs/design/ots/{ots-name}.md` exist when OTS items are present +- [ ] `docs/design/shared.md` and `docs/design/shared/{package-name}.md` exist when Shared Packages are present +- [ ] Documents are integrated into ReviewMark review-sets diff --git a/.github/standards/reqstream-usage.md b/.github/standards/reqstream-usage.md index 58b08b4..2371164 100644 --- a/.github/standards/reqstream-usage.md +++ b/.github/standards/reqstream-usage.md @@ -9,7 +9,7 @@ globs: ["requirements.yaml", "docs/reqstream/**/*.yaml"] Read these standards first before applying this standard: - **`requirements-principles.md`** - Requirements principles and unidirectionality -- **`software-items.md`** - Software categorization (System/Subsystem/Unit/OTS) +- **`software-items.md`** - Software categorization (System/Subsystem/Unit/OTS/Shared Package) # Requirements Organization @@ -18,54 +18,83 @@ because ReqStream discovers files via the includes chain in `requirements.yaml` and organizes report output by this hierarchy: ```text -requirements.yaml # Root file (includes only) +requirements.yaml # Root file (includes only) docs/reqstream/ -├── {system-name}/ # System-level requirements folder (one per system) -│ ├── {system-name}.yaml # System-level requirements +├── {system-name}.yaml # System-level requirements +├── {system-name}/ # System folder (one per system) │ ├── platform-requirements.yaml # Platform support requirements -│ ├── {subsystem-name}/ # Subsystem (kebab-case); may nest recursively -│ │ ├── {subsystem-name}.yaml # Requirements for this subsystem -│ │ ├── {child-subsystem}/ # Child subsystem (same structure as parent) -│ │ └── {unit-name}.yaml # Requirements for units within this subsystem -│ └── {unit-name}.yaml # Requirements for top-level units (outside subsystems) -└── ots/ # OTS items appear as a distinct section in reports - └── {ots-name}.yaml # Requirements for OTS components +│ ├── {subsystem-name}.yaml # Subsystem requirements +│ ├── {subsystem-name}/ # Subsystem folder (kebab-case); may nest recursively +│ │ ├── {subsystem-name}.yaml # Child subsystem requirements +│ │ ├── {subsystem-name}/ # Child subsystem folder +│ │ └── {unit-name}.yaml # Unit requirements +│ └── {unit-name}.yaml # System-level unit requirements +├── ots/ # OTS items appear as a distinct section in reports +│ └── {ots-name}.yaml # Requirements for OTS components +└── shared/ # Shared Packages appear as a distinct section in reports + └── {package-name}.yaml # Requirements for Shared Package dependencies ``` +Local items have matching relative paths across `docs/reqstream/`, `docs/design/`, and `docs/verification/`: + +- Requirements: `{system-name}[/{subsystem-name}...]/{item-name}.yaml` +- Design: `{system-name}[/{subsystem-name}...]/{item-name}.md` +- Verification: `{system-name}[/{subsystem-name}...]/{item-name}.md` + # Requirements File Format -```yaml -sections: - - title: Functional Requirements - requirements: - - id: System-Component-Feature # Used as-is in all reports - make it readable - title: The system shall perform the required function. - justification: | - Business rationale and any regulatory references. - # ReqStream extracts this field into the justifications report (--justifications) - children: # ReqStream validates this decomposition chain - - ChildSystem-Feature-Behavior # Downward links only (see requirements-principles.md) - tests: # ReqStream matches these by method name in test results - - TestMethodName - - windows@PlatformSpecificTest # Only test runs on Windows count as evidence -``` +Each file adds requirements at exactly one level of the hierarchy. The file spells out +its full ancestry as nested `{ItemName} Requirements` sections down to that level, then +places requirements there. ReqStream merges identical section title paths across included +files automatically. Always determine item classification from `docs/design/introduction.md` - +folder depth does not determine whether an item is a subsystem or unit. + +Valid section nestings (names in `{braces}` are placeholders): -# OTS Software Requirements +```text +{SystemName} Requirements # system-level requirements +├── {SubsystemName} Requirements # root subsystem requirements +│ ├── {SubsystemName} Requirements # nested subsystem (may recurse) +│ │ └── {UnitName} Requirements # unit under a nested subsystem +│ └── {UnitName} Requirements # unit under a root subsystem +└── {UnitName} Requirements # unit directly under the system +OTS Software Requirements # OTS root section (fixed title) +└── {OtsName} Requirements # requirements for one OTS item +Shared Package Requirements # shared package root section (fixed title) +└── {PackageName} Requirements # requirements for one shared package +``` -Use nested sections in `docs/reqstream/ots/` because ReqStream renders the `ots/` -subtree as a distinct section in generated reports, separate from in-house -system requirements: +Each file implements one path through this tree: ```yaml sections: - - title: OTS Software Requirements + - title: '{SystemName} Requirements' sections: - - title: System.Text.Json + - title: '{SubsystemName} Requirements' requirements: - - id: TemplateTool-SystemTextJson-ReadJson - title: System.Text.Json shall be able to read JSON files. - tests: - - JsonReaderTests.TestReadValidJson + - id: System-Subsystem-Feature # Used as-is in all reports - make it readable + title: The subsystem shall perform the required function. + justification: | # ReqStream extracts this into the justifications report (--justifications) + Business rationale and any regulatory references. + tags: # Optional: categorize for filtering with --filter + - security + children: # Optional: ReqStream validates this decomposition chain + - System-Subsystem-Unit-Feat # Downward links only (see requirements-principles.md) + tests: # ReqStream matches these by method name in test results + - TestMethodName + - windows@PlatformSpecificTest # Only test runs on Windows count as evidence +``` + +# Tags (OPTIONAL) + +Tags are free-form - no mandatory vocabulary. Common tags: `security`, `safety`, `performance`, +`compliance`, `reliability`, `critical`. Use `--filter` to selectively export or enforce subsets +(OR logic across comma-separated tags): + +```bash +dotnet reqstream --requirements requirements.yaml \ + --filter security,critical \ + --report docs/requirements_doc/generated/security_requirements.md ``` # Semantic IDs (MANDATORY) @@ -120,12 +149,9 @@ dotnet reqstream --requirements requirements.yaml \ Before submitting requirements, verify: -- [ ] All requirements have semantic IDs (`System-Section-Feature` pattern) -- [ ] Every requirement links to at least one passing test +- [ ] All requirements have semantic IDs (`System-Component-Feature` pattern) +- [ ] Every requirement has a justification explaining business/regulatory need +- [ ] Every requirement links to at least one test - [ ] Platform-specific requirements use source filters (`platform@TestName`) -- [ ] Comprehensive justification explains business/regulatory need -- [ ] Files organized under `docs/reqstream/` following folder structure patterns -- [ ] Subsystem folders use kebab-case naming matching source code -- [ ] OTS requirements placed in `ots/` subfolder -- [ ] Valid YAML syntax passes yamllint validation -- [ ] Test result formats compatible (TRX, JUnit XML) +- [ ] All files and folders use kebab-case names matching source code structure +- [ ] All files are organized under `docs/reqstream/` following the folder structure above diff --git a/.github/standards/reviewmark-usage.md b/.github/standards/reviewmark-usage.md index 2f778dc..e1c3771 100644 --- a/.github/standards/reviewmark-usage.md +++ b/.github/standards/reviewmark-usage.md @@ -9,7 +9,7 @@ description: Follow these standards when configuring file reviews with ReviewMar Read these standards first before applying this standard: -- **`software-items.md`** - Software categorization (System/Subsystem/Unit/OTS) +- **`software-items.md`** - Software categorization (System/Subsystem/Unit/OTS/Shared Package) ## Purpose @@ -21,169 +21,226 @@ review, organizes them into review-sets, and generates review plans and reports. - **Lint Configuration**: `dotnet reviewmark --lint` - **Elaborate Review-Set**: `dotnet reviewmark --elaborate {review-set}` - **Generate Plan**: `dotnet reviewmark --plan docs/code_review_plan/generated/plan.md --enforce` - -> **Note**: `--enforce` causes the plan to fail with a non-zero exit code if any repository -> files are not covered by a review-set. Uncovered files indicate a gap in review-set -> configuration that should be addressed. + (exits non-zero if any files are uncovered) ## Repository Structure -Required repository items for ReviewMark operation: - - `.reviewmark.yaml` - Configuration for review-sets, file-patterns, and review evidence-source. -- `docs/code_review_plan/generated/` - Generated review plan (build output, do not edit) -- `docs/code_review_report/generated/` - Generated review report (build output, do not edit) # Review Definition Structure Configure reviews in `.reviewmark.yaml` at repository root: ```yaml -# Patterns identifying all files that require review needs-review: - # Include source code (adjust file extensions for your repo) - - "**/*.cs" # C# source files - - "**/*.cpp" # C++ source files - - "**/*.hpp" # C++ header files - - "!**/bin/**" # Generated source in build outputs - - "!**/obj/**" # Generated source in build intermediates - - # Include requirement files - - "requirements.yaml" # Root requirements file - - "docs/reqstream/**/*.yaml" # Requirements files - - # Include critical documentation files - - "README.md" # Root level README - - "docs/user_guide/**/*.md" # User guide - - "docs/design/**/*.md" # Design documentation - - "docs/verification/**/*.md" # Verification design documentation - -# Source of review evidence + - "**/*.cs" + - "**/*.cpp" + - "**/*.hpp" + - "!**/bin/**" + - "!**/obj/**" + - "requirements.yaml" + - "docs/reqstream/**/*.yaml" + - "README.md" + - "docs/user_guide/**/*.md" + - "docs/design/**/*.md" + - "docs/verification/**/*.md" + evidence-source: type: none + +context: + - docs/design/introduction.md + +reviews: + - id: Purpose + title: Review that README and User Guide are Coherent and Complete + paths: + - "README.md" + - "docs/user_guide/**/*.md" + - id: Decomposition + title: Review that {SystemName} Decomposition Addresses the Stated Purpose + context: + - "README.md" + - "docs/user_guide/**/*.md" + paths: + - "requirements.yaml" + - "docs/design/introduction.md" ``` -# Review-Set Design Principles +For a complete annotated example with template directives, see `.reviewmark.yaml` in the +reference template (`{template-url}/.reviewmark.yaml` per `AGENTS.md`). -When constructing review-sets, follow these principles to maintain manageable scope and effective compliance evidence: +# Review-Set Design Principles - **Hierarchical Scope**: Higher-level reviews exclude lower-level implementation details, relying instead on design documents to describe what components they use. System reviews exclude subsystem/unit details, subsystem reviews exclude unit source code, only unit reviews include actual implementation. - **Single Focus**: Each review-set proves one specific compliance question (user promises, system architecture, design consistency, etc.) +- **Parent Context**: Unit and subsystem reviews include parent design and requirements as + context so reviewers understand the intended role and scope; see the Context Files section. - **Context Management**: Keep file counts manageable to prevent context overflow while maintaining complete coverage through the hierarchy -# Review-Set Organization +# Context Files -Organize review-sets using these standard patterns to ensure comprehensive coverage -while keeping each review manageable in scope: +Context files are shown to reviewers for orientation but not fingerprinted. Add a top-level +`context:` key for global context (every reviewer) and a per-review-set `context:` between +`title:` and `paths:` for review-specific context. Always include `docs/design/introduction.md` +as global context. -**Naming conventions**: See `software-items.md` - kebab-case placeholders -(e.g., `{system-name}`) are always kebab-case; cased placeholders -(e.g., `{SystemName}`) follow your language's convention. +| Review Type | Context to add | +| :---------- | :------------- | +| `Decomposition` | `README.md`, `docs/user_guide/**/*.md` | +| `{SystemName}-Architecture` | `README.md`, `docs/user_guide/**/*.md` | +| `{SystemName}-Design` | `docs/reqstream/{system-name}.yaml` | +| `{SystemName}-Verification` | `docs/reqstream/{system-name}.yaml` | +| `{SystemName}-AllRequirements` | Parent system design doc + `docs/reqstream/{system-name}.yaml` | +| `{SystemName}-{UnitName}` (direct unit) | Parent system design doc + parent system requirements | +| `{SystemName}-{SubsystemName}` (subsystem) | Parent system design doc + parent system requirements | +| `{SystemName}-{SubsystemName}-{UnitName}` (unit under subsystem) | System + subsystem design docs + requirements | -## `Purpose` Review (only one per repository) +# Review-Set Organization -Reviews user-facing capabilities and system promises: +**Naming conventions**: Placeholders in documentation, requirements, design, and +verification file paths are kebab-case (e.g., `{system-name}`). Placeholders in +source and test file paths may use the casing conventional for the project's +source language or repository layout (e.g., `{SystemName}`). Review-set name +placeholders are always PascalCase (e.g., `{SystemName}`). + +## `Purpose` Review (only one per repository) -- **Purpose**: Proves that the systems provide the capabilities the user is being told about -- **Title**: "Review that Advertised Features Match System Design" -- **Scope**: Excludes subsystem and unit files, relying on system-level design documents - to describe what subsystems and units they use +- **Purpose**: Proves that the user-facing docs are coherent and complete — the north-star for the hierarchy +- **Title**: "Review that README and User Guide are Coherent and Complete" +- **ID**: `Purpose` (no system prefix — one per repository) +- **Scope**: README and user_guide only; no requirements or design files - **File Path Patterns**: - README: `README.md` - User guide: `docs/user_guide/**/*.md` - - System requirements: `docs/reqstream/{system-name}/{system-name}.yaml` - - Design introduction: `docs/design/introduction.md` - - System design: `docs/design/{system-name}/{system-name}.md` -## `{System}-Architecture` Review (one per system) +## `Decomposition` Review (only one per repository) + +- **Purpose**: Proves that the software items tree breakdown logically addresses + the user-facing promise; the structural mirror of the decomposition decision +- **Title**: "Review that {SystemName} Decomposition Addresses the Stated Purpose" +- **ID**: `Decomposition` (no system prefix — one per repository) +- **Scope**: introduction.md (the decomposition narrative) and requirements.yaml + (the structural tree); no system-level detail +- **File Path Patterns**: + - Root requirements: `requirements.yaml` + - Design introduction: `docs/design/introduction.md` +- **Context Files**: `README.md`, `docs/user_guide/**/*.md` -Reviews system architecture and operational validation: +## `{SystemName}-Architecture` Review (one per system) - **Purpose**: Proves that the system is designed and tested to satisfy its requirements -- **Title**: "Review that {System} Architecture Satisfies Requirements" +- **Title**: "Review that {SystemName} Architecture Satisfies Requirements" - **Scope**: Excludes subsystem and unit files, relying on system-level design to describe what subsystems and units it uses - **File Path Patterns**: - - System requirements: `docs/reqstream/{system-name}/{system-name}.yaml` + - System requirements: `docs/reqstream/{system-name}.yaml` - Design introduction: `docs/design/introduction.md` - - System design: `docs/design/{system-name}/{system-name}.md` + - System design: `docs/design/{system-name}.md` - Verification introduction: `docs/verification/introduction.md` - - System verification design: `docs/verification/{system-name}/{system-name}.md` + - System verification design: `docs/verification/{system-name}.md` - System integration tests: `test/{SystemName}.Tests/{SystemName}Tests.{ext}` +- **Context Files**: `README.md`, `docs/user_guide/**/*.md` -## `{System}-Design` Review (one per system) - -Reviews architectural and design consistency: +## `{SystemName}-Design` Review (one per system) - **Purpose**: Proves the system design is consistent and complete -- **Title**: "Review that {System} Design is Consistent and Complete" +- **Title**: "Review that {SystemName} Design is Consistent and Complete" - **Scope**: Only brings in top-level requirements and relies on brevity of design documentation +- **Context Files**: `docs/reqstream/{system-name}.yaml` - **File Path Patterns**: - - System requirements: `docs/reqstream/{system-name}/{system-name}.yaml` - Platform requirements: `docs/reqstream/{system-name}/platform-requirements.yaml` - Design introduction: `docs/design/introduction.md` + - System design: `docs/design/{system-name}.md` - System design files: `docs/design/{system-name}/**/*.md` + - OTS overview: `docs/design/ots.md` _(only if OTS items exist)_ + - Shared Package overview: `docs/design/shared.md` _(only if Shared Package items exist)_ -## `{System}-AllRequirements` Review (one per system) +## `{SystemName}-Verification` Review (one per system) -Reviews requirements quality and traceability: +- **Purpose**: Proves the system verification design is consistent and covers all requirements +- **Title**: "Review that {SystemName} Verification is Consistent and Complete" +- **Scope**: Only brings in top-level requirements and all verification docs for the system +- **Context Files**: `docs/reqstream/{system-name}.yaml` +- **File Path Patterns**: + - Verification introduction: `docs/verification/introduction.md` + - System verification: `docs/verification/{system-name}.md` + - System verification files: `docs/verification/{system-name}/**/*.md` + - OTS overview: `docs/verification/ots.md` _(only if OTS items exist)_ + - Shared Package overview: `docs/verification/shared.md` _(only if Shared Package items exist)_ + +## `{SystemName}-AllRequirements` Review (one per system) - **Purpose**: Proves the requirements are consistent and complete -- **Title**: "Review that All {System} Requirements are Complete" +- **Title**: "Review that All {SystemName} Requirements are Complete" - **Scope**: Only brings in requirements files to keep review manageable - **File Path Patterns**: - - Root requirements: `requirements.yaml` - - System requirements: `docs/reqstream/{system-name}/**/*.yaml` - - OTS requirements: `docs/reqstream/ots/**/*.yaml` (if applicable) + - Subsystem/unit requirements: `docs/reqstream/{system-name}/**/*.yaml` +- **Context Files**: `docs/design/{system-name}.md`, `docs/reqstream/{system-name}.yaml` -## `{System}-{Subsystem[-Child...]}` Review (one per subsystem at any depth) - -Reviews subsystem architecture and interfaces: +## `{SystemName}-{SubsystemName}[-{SubsystemName}...]` Review (one per subsystem at any depth) - **Purpose**: Proves that the subsystem is designed and tested to satisfy its requirements -- **Title**: "Review that {System} {Subsystem} Satisfies Subsystem Requirements" +- **Title**: "Review that {SystemName} {SubsystemName} Satisfies Subsystem Requirements" - **Scope**: Excludes units under the subsystem, relying on subsystem design to describe what units it uses - **File Path Patterns**: - - Requirements: `docs/reqstream/{system-name}/.../{subsystem-name}/{subsystem-name}.yaml` - - Design: `docs/design/{system-name}/.../{subsystem-name}/{subsystem-name}.md` - - Verification design: `docs/verification/{system-name}/.../{subsystem-name}/{subsystem-name}.md` - - Tests: `test/{SystemName}.Tests/.../{SubsystemName}/{SubsystemName}Tests.{ext}` - -## `{System}-{Subsystem[-Child...]}-{Unit}` Review (one per unit) + - Requirements: `docs/reqstream/{system-name}[/{subsystem-name}...]/{subsystem-name}.yaml` + - Design: `docs/design/{system-name}[/{subsystem-name}...]/{subsystem-name}.md` + - Verification design: `docs/verification/{system-name}[/{subsystem-name}...]/{subsystem-name}.md` + - Tests: `test/{SystemName}.Tests[/{SubsystemName}...]/{SubsystemName}Tests.{ext}` +- **Context Files**: `docs/design/{system-name}.md`, `docs/reqstream/{system-name}.yaml` -Reviews individual software unit implementation: +## `{SystemName}-{SubsystemName}[-{SubsystemName}...]-{UnitName}` Review (one per unit) - **Purpose**: Proves the unit is designed, implemented, and tested to satisfy its requirements -- **Title**: "Review that {System} {Subsystem} {Unit} Implementation is Correct" +- **Title**: "Review that {SystemName} {SubsystemName} {UnitName} Implementation is Correct" - **Scope**: Complete unit review including all artifacts - **File Path Patterns**: - - Requirements: `docs/reqstream/{system-name}/.../{unit-name}.yaml` - - Design: `docs/design/{system-name}/.../{unit-name}.md` - - Verification design: `docs/verification/{system-name}/.../{unit-name}.md` - - Source: `src/{SystemName}/.../{UnitName}.{ext}` - - Tests: `test/{SystemName}.Tests/.../{UnitName}Tests.{ext}` + - Requirements: `docs/reqstream/{system-name}[/{subsystem-name}...]/{unit-name}.yaml` + - Design: `docs/design/{system-name}[/{subsystem-name}...]/{unit-name}.md` + - Verification design: `docs/verification/{system-name}[/{subsystem-name}...]/{unit-name}.md` + - Source (C# example): `src/{SystemName}[/{SubsystemName}...]/{UnitName}.cs` + - Tests (C# example): `test/{SystemName}.Tests[/{SubsystemName}...]/{UnitName}Tests.cs` + - Source (snake_case C++ example): `src/{system_name}[/{subsystem_name}...]/{unit_name}.cpp` + - Tests (snake_case C++ example): `test/{system_name}_tests[/{subsystem_name}...]/{unit_name}_tests.cpp` +- **Context Files**: Parent system design + requirements; add subsystem design + + requirements for each subsystem level above the unit. + +## `OTS-{OtsName}` Review (one per OTS item) + +- **Purpose**: Proves that the OTS item provides the required functionality and is correctly integrated +- **Title**: "Review that {OtsName} Provides Required Functionality" +- **Scope**: No local source code; review covers integration design, requirements, and verification evidence +- **File Path Patterns**: + - OTS requirements: `docs/reqstream/ots/{ots-name}.yaml` + - OTS integration design: `docs/design/ots/{ots-name}.md` + - OTS verification: `docs/verification/ots/{ots-name}.md` + - Tests (if applicable): `test/OtsSoftwareTests/...` (C#) or `test/ots_software_tests/...` + (Python/other) — fixed repo-level name, no system prefix + +## `Shared-{PackageName}` Review (one per Shared Package) + +- **Purpose**: Proves that the Shared Package provides the required advertised features and is correctly integrated +- **Title**: "Review that {PackageName} Provides Required Features" +- **Scope**: No local source code; review covers integration design, requirements, and verification evidence +- **File Path Patterns**: + - Shared Package requirements: `docs/reqstream/shared/{package-name}.yaml` + - Shared Package integration design: `docs/design/shared/{package-name}.md` + - Shared Package verification: `docs/verification/shared/{package-name}.md` **Note**: File path patterns use `{ext}` as a placeholder for language-specific extensions (`.cs`, `.cpp`/`.hpp`, `.py`, etc.). Adapt to your repository's languages. # Quality Checks -Before submitting ReviewMark configuration, verify: - - [ ] `.reviewmark.yaml` exists at repository root with proper structure - [ ] Review-set organization follows the standard hierarchy patterns -- [ ] Purpose review-set includes README.md, user guide, system requirements, design introduction, and system design files -- [ ] System-level reviews follow hierarchical scope principle (exclude subsystem/unit details) -- [ ] Subsystem reviews follow hierarchical scope principle (exclude unit source code) -- [ ] Only unit reviews include actual source code files -- [ ] Architecture review-sets include system verification design alongside system design -- [ ] Subsystem review-sets include subsystem verification design -- [ ] Unit review-sets include unit verification design - [ ] Each review-set focuses on a single compliance question (single focus principle) - [ ] File patterns use correct glob syntax and match intended files - [ ] Review-set file counts remain manageable (context management principle) +- [ ] Context configured per the Context Files section diff --git a/.github/standards/software-items.md b/.github/standards/software-items.md index 4e5c90e..6c29525 100644 --- a/.github/standards/software-items.md +++ b/.github/standards/software-items.md @@ -3,20 +3,16 @@ name: Software Items description: Follow these standards when categorizing software components. --- -# Software Items Definition Standards - -This document defines standards for categorizing software items within -Continuous Compliance environments because proper categorization determines -requirements management approach, testing strategy, and review scope. - # Software Item Categories -Categorize all software into five primary groups: +Categorize all software into six primary groups: - **Software Package**: Distributable unit delivered to end users or dependent systems, containing one software system with all its components. All software - systems are delivered as a software package. When consumed by another system, - our software package is treated as an OTS Software Item by that system. + systems are delivered as a software package. When consumed by a system outside + the producing program, our software package is treated as an OTS Software Item + by that system. When consumed by another repository within the same program, + it is treated as a Shared Package. - **Software System**: Complete deliverable product including all components and external interfaces, contained within a software package - **Software Subsystem**: Major architectural component with well-defined @@ -24,7 +20,11 @@ Categorize all software into five primary groups: - **Software Unit**: Individual class, function, or tightly coupled set of functions that can be tested in isolation - **OTS Software Item**: Third-party component (library, framework, tool, or - published software package) providing functionality not developed in-house + published software package) providing functionality not developed within the program +- **Shared Package**: A software package produced by a different repository within + the same program, consumed as a dependency. Referenced by its advertised features + rather than internal design; traceability to program-level requirements runs + through the top-level project. **Naming**: When names collide in hierarchy, add descriptive suffix to higher-level entity: @@ -34,17 +34,28 @@ Categorize all software into five primary groups: # Naming Conventions in File Path Patterns -Two placeholder styles appear in path patterns across these standards: +Three placeholder forms appear in path patterns across these standards: -- **Kebab-case** (`{system-name}`, `{unit-name}`): always kebab-case - - used in documentation and requirements paths -- **Cased** (`{SystemName}`, `{UnitName}`): follow your language's convention - - `PascalCase` for C#/Java, `snake_case` for C++/Python - - used in source and test file paths +- **Kebab-case** (`{system-name}`, `{unit-name}`): always kebab-case — + documentation and requirements file paths +- **PascalCase IDs** (`{SystemName}`, `{UnitName}`): always PascalCase — + requirements IDs, ReviewMark IDs, and other documentation identifiers +- **Language-cased** (`{SystemName}` or `{system_name}`): follow your language's + convention — `PascalCase` for C#/Java, `snake_case` for C++/Python — + source and test file/folder names -# Categorization Guidelines +## Nesting Depth Notation -Choose the appropriate category based on scope and testability: +Subsystems nest to any depth. Patterns use bracket-ellipsis to express this without +enumerating levels — `[/{subsystem-name}...]` in paths, `[-{SubsystemName}...]` in +dash-separated IDs. Examples covering all three forms: + +- `{SystemName}[-{SubsystemName}...]-{UnitName}-Feature` (PascalCase ID) +- `docs/design/{system-name}[/{subsystem-name}...]/{unit-name}.md` (kebab-case doc path) +- `src/{SystemName}[/{SubsystemName}...]/{UnitName}.cs` (C# source path) +- `src/{system_name}[/{subsystem_name}...]/{unit_name}.cpp` (C++/Python source path) + +# Categorization Guidelines ## Software Package @@ -75,12 +86,37 @@ Choose the appropriate category based on scope and testability: ## OTS Software Item -- External dependency not developed in-house - typically a third-party published +- External dependency from outside the program - typically a third-party published software package (NuGet, npm, etc.), hosted service, or tool -- Our own published software package becomes an OTS item to any system that - consumes it +- A package produced by an unrelated program (inside or outside the organization) + is treated as OTS by any consuming system - Tested through integration tests proving required functionality works - Examples: System.Text.Json, Entity Framework, third-party APIs +- **Artifact locations** (OTS items have no internal design documentation): + - Requirements: `docs/reqstream/ots/{ots-name}.yaml` + - Design: `docs/design/ots/{ots-name}.md` (integration/usage design) + - Verification: `docs/verification/ots/{ots-name}.md` + - These folders sit parallel to system folders (not inside any system folder) +- System design documentation records which OTS items each system depends on +- **OTS test project**: If no other verification evidence is available (e.g., vendor test results, + published compliance reports), a dedicated test project holds OTS integration tests - one test + file per OTS item requiring tests. OTS items are repo-level (not per-system), so the project + uses a fixed repo-level name: `test/OtsSoftwareTests/` (C#) or `test/ots_software_tests/` + (Python/other) — never prefixed with a system or project name. + +## Shared Package + +- A software package produced by a different repository within the same program +- The consuming repository references advertised features, not internal design or source +- Traceability to program-level requirements runs through the top-level project, + not directly between repositories +- Verified through any appropriate approach in the consuming repository - most commonly + downstream integration tests that transitively prove the advertised features are functional +- **Artifact locations** (no internal design documentation in the consuming repository): + - Requirements: `docs/reqstream/shared/{package-name}.yaml` + - Design: `docs/design/shared/{package-name}.md` (integration/usage design) + - Verification: `docs/verification/shared/{package-name}.md` + - These folders sit parallel to system and OTS folders # Software Item Artifact Model @@ -89,7 +125,8 @@ unit - because reviewing any one artifact in isolation cannot determine whether the item is correct, well-designed, and proven to work: - **Requirements** - WHAT the item must do (drives all other artifacts; applies to all item types) -- **Design** - HOW the item satisfies its requirements (in-house items only: system, subsystem, unit) +- **Design** - HOW the item satisfies its requirements (full design for local items: system, + subsystem, unit; integration/usage design for OTS and Shared Package) - **Verification Design** - HOW the requirements will be tested (applies to all item types) -- **Source code** - The implementation of the design (in-house units only) +- **Source code** - The implementation of the design (local units only; not applicable to OTS or Shared Package) - **Tests** - PROOF the item does WHAT it is required to do (applies to all item types) diff --git a/.github/standards/technical-documentation.md b/.github/standards/technical-documentation.md index 7ff5b5a..23893bd 100644 --- a/.github/standards/technical-documentation.md +++ b/.github/standards/technical-documentation.md @@ -6,9 +6,6 @@ globs: ["docs/**/*.md", "README.md", "!docs/**/generated/**"] # Technical Documentation Standards -This document defines standards for technical documentation within Continuous -Compliance environments. - # Core Principles Technical documentation serves as compliance evidence and must be structured @@ -40,8 +37,11 @@ docs/{collection}/ Without `title.txt` and `definition.yaml` the pipeline cannot generate the document. When creating a new document collection, create these three files together and use -the existing collections under `docs/` as templates - they share a consistent -structure across all collections. +the existing collections under `docs/` as templates. + +The `generated/` folder is **never committed** to the repository - it is created +locally and in CI by the build pipeline. Do not flag its absence as a conformance +issue. **`title.txt`** - YAML front matter with document metadata. Use the existing files under `docs/` as a pattern and keep fields consistent with the rest of @@ -81,23 +81,34 @@ elsewhere causes duplicate sections in the compiled PDF. ## Document Ordering -List documents in logical reading order in Pandoc configuration because -readers need coherent information flow from general to specific topics. +List documents in logical reading order in `definition.yaml`. + +## Heading Depth Rule (MANDATORY) + +A file's top-level heading depth must equal its folder depth under the document +collection root - this ensures Pandoc can concatenate all files in `definition.yaml` +order and produce a coherent outline with no heading-shift configuration: + +| Folder depth | Top heading | +| --- | --- | +| 0 - collection root | `#` | +| 1 - one subfolder deep | `##` | +| 2 - two subfolders deep | `###` | +| N - N subfolders deep | `#` × (N+1) | + +Internal sections use the next heading level down (e.g. a `##` file uses `###` +for *Overview*, *Interfaces*, etc.). Deeply nested files have fewer heading levels +available - keep internal structure flat to avoid excessive nesting. # Writing Guidelines Write technical documentation for clarity and compliance verification: - **Clear and Concise**: Use direct language and avoid unnecessary complexity. - Regulatory reviewers must understand content quickly. -- **Structured Sections**: Use consistent heading hierarchy and section - organization. Enables automated processing and review. -- **Specific Examples**: Include concrete examples with actual values rather - than placeholders. Supports implementation verification. +- **Structured Sections**: Use consistent heading hierarchy and section organization. +- **Specific Examples**: Include concrete examples with actual values rather than placeholders. - **Current Information**: Keep documentation synchronized with code changes. - Outdated documentation invalidates compliance evidence. -- **Traceable Content**: Link documentation to requirements and implementation - where applicable for audit trails. +- **Traceable Content**: Link documentation to requirements and implementation where applicable. ## References Sections @@ -117,26 +128,16 @@ Instead use **verbal references** - plain prose that identifies the target by na > > Refer to the *System Requirements* document for the full specification. -Verbal references are readable by both AI agents and humans in any rendering environment. - # Markdown Format Requirements -Markdown documentation in this repository must follow the formatting standards -defined in `.markdownlint-cli2.yaml` (subject to any exclusions configured there) -for consistency and professional presentation: - -- **120 Character Line Limit**: Keep lines 120 characters or fewer for readability. - Break long lines naturally at punctuation or logical breaks. -- **No Trailing Whitespace**: Remove all trailing spaces and tabs from line - endings to prevent formatting inconsistencies. -- **Blank Lines Around Headings**: Include a blank line both before and after - each heading to improve document structure and readability. -- **Blank Lines Around Lists**: Include a blank line both before and after - numbered and bullet lists to ensure proper rendering and visual separation. -- **ATX-Style Headers**: Use `#` syntax for headers instead of underline style - for consistency across all documentation. -- **Consistent List Indentation**: Use 2-space indentation for nested list - items to maintain uniform formatting. +Follow `.markdownlint-cli2.yaml` formatting standards: + +- **120 Character Line Limit**: Keep lines 120 characters or fewer; break at punctuation or logical breaks. +- **No Trailing Whitespace**: Remove all trailing spaces and tabs. +- **Blank Lines Around Headings**: Include a blank line before and after each heading. +- **Blank Lines Around Lists**: Include a blank line before and after numbered and bullet lists. +- **ATX-Style Headers**: Use `#` syntax, not underline style. +- **Consistent List Indentation**: Use 2-space indentation for nested list items. # Auto-Generated Content (CRITICAL) @@ -147,8 +148,6 @@ build outputs that are overwritten on every CI run: respective `docs/` sections, or in `docs/generated/` for final release artifacts - **Source Modification**: Update source files (requirements YAML, `.reviewmark.yaml`, tool configuration) instead of generated output -- **Tool Integration**: Generated content integrates with CI/CD pipelines and - manual changes disrupt automation # README.md Best Practices @@ -171,20 +170,12 @@ Structure README.md for both human readers and AI agent processing: - **Code Block Languages**: Specify language for syntax highlighting and tool processing - **Clear Prerequisites**: List exact version requirements and dependencies -## Quality Guidelines - -- **Scannable Structure**: Use bullet points, headings, and short paragraphs -- **Current Examples**: Verify all code examples work with current version -- **Link Validation**: Ensure all external links are accessible and current -- **Consistent Tone**: Professional, helpful tone appropriate for technical audience - # Quality Checks Before submitting technical documentation, verify: - [ ] Documentation organized under `docs/` following standard folder structure - [ ] Pandoc collections include `introduction.md` with Purpose and Scope sections -- [ ] Content follows clear and concise writing guidelines with specific examples - [ ] No modifications made to auto-generated markdown files in compliance folders - [ ] README.md includes all required sections with absolute URLs and concrete examples - [ ] Documentation integrated into ReviewMark review-sets for formal review diff --git a/.github/standards/testing-principles.md b/.github/standards/testing-principles.md index 73974ff..917463e 100644 --- a/.github/standards/testing-principles.md +++ b/.github/standards/testing-principles.md @@ -3,11 +3,6 @@ name: Testing Principles description: Follow these standards when developing any software tests. --- -# Testing Principles Standards - -This document defines universal testing principles and quality standards for test development within -Continuous Compliance environments. - # Test Dependency Boundaries (MANDATORY) Respect software item hierarchy boundaries to ensure review-sets can validate proper architectural scope. diff --git a/.github/standards/verification-documentation.md b/.github/standards/verification-documentation.md index f6f407f..494e40f 100644 --- a/.github/standards/verification-documentation.md +++ b/.github/standards/verification-documentation.md @@ -6,123 +6,96 @@ globs: ["docs/verification/**/*.md"] # Required Standards -Read these standards first before applying this standard: - - **`technical-documentation.md`** - General technical documentation standards -- **`software-items.md`** - Software categorization (System/Subsystem/Unit/OTS) - -# Core Principles - -Verification design is the bridge between requirements and tests - it documents HOW -requirements will be verified, enabling reviewers to confirm test completeness without -reading implementation code. +- **`software-items.md`** - Software categorization (System/Subsystem/Unit/OTS/Shared Package) -# Required Structure and Documents - -Organize under `docs/verification/` mirroring the software item hierarchy: +# Folder Structure ```text docs/verification/ -├── introduction.md # Verification overview -├── {system-name}/ # System-level verification folder (one per system) -│ ├── {system-name}.md # System-level verification design -│ ├── {subsystem-name}/ # Subsystem (kebab-case); may nest recursively -│ │ ├── {subsystem-name}.md # Subsystem verification design -│ │ ├── {child-subsystem}/ # Child subsystem (same structure as parent) -│ │ └── {unit-name}.md # Unit-level verification design documents -│ └── {unit-name}.md # Top-level unit verification documents (if not in subsystem) -└── ots/ # OTS items (one verification file per OTS item) - └── {ots-name}.md # Verification evidence for each OTS item +├── introduction.md # heading depth # +├── {system-name}.md # heading depth # +├── {system-name}/ +│ ├── {subsystem-name}.md # heading depth ## +│ ├── {subsystem-name}/ +│ │ └── {unit-name}.md # heading depth ### +│ └── {unit-name}.md # heading depth ## +├── ots.md # heading depth # (if OTS items exist) +├── ots/ +│ └── {ots-name}.md # heading depth ## +├── shared.md # heading depth # (if Shared Packages exist) +└── shared/ + └── {package-name}.md # heading depth ## ``` -## introduction.md (MANDATORY) +All sections in every file are mandatory; write "N/A - {justification}" rather than removing any. +Determine subsystem vs. unit classification from `docs/design/introduction.md` — folder depth does not determine classification. -Follow the standard `introduction.md` format from `technical-documentation.md`. Scope -covers all software items including OTS items (via self-validation if appropriate). +# introduction.md (MANDATORY) -Include a Companion Artifact Structure note so agents and reviewers can navigate from any -artifact to all related files: +Must include: -```text -In-house items have parallel artifacts in: -- Requirements: `docs/reqstream/{system}/.../{item}.yaml` (kebab-case) -- Design: `docs/design/{system}/.../{item}.md` (kebab-case) -- Verification: `docs/verification/{system}/.../{item}.md` (kebab-case) -- Source: `src/{System}/.../{Item}.{ext}` (cased per language) -- Tests: `test/{System}.Tests/.../{Item}Tests.{ext}` (cased per language) - -OTS items have parallel artifacts in: -- Requirements: `docs/reqstream/ots/{ots-name}.yaml` (kebab-case) -- Verification: `docs/verification/ots/{ots-name}.md` (kebab-case) -- Tests: `test/{OtsName}.Tests/...` (cased per language, if required) - -Review-sets: defined in `.reviewmark.yaml` -``` +- **Purpose**: audience and compliance drivers +- **Scope**: items covered and explicitly excluded (no test projects) +- **Companion Artifact Structure**: parallel paths for requirements, design, verification, source, tests +- **References** _(if applicable)_: external standards or specifications - only in `introduction.md` -If the verification design references external documents (standards, specifications), include -a `## References` section in `introduction.md` only - do not add one to any other verification file. +# System Verification Design (MANDATORY) -## System Verification Design (MANDATORY) +Create `{system-name}.md` (`#` heading) and `{system-name}/` folder: -For each system, create a kebab-case folder and `{system-name}.md` covering: +- **Verification Approach**: test types (unit, integration, end-to-end), framework, project structure +- **Test Environment**: OS, runtime, external services, files, or configuration required +- **Acceptance Criteria**: what constitutes a passing system test (IEC 62304 §5.7.2) +- **Test Scenarios**: named scenarios for each system requirement -- System verification strategy and overall test approach -- Test environments and configuration required -- External interface simulation and test-harness design -- End-to-end and integration test scenarios covering system requirements -- Acceptance criteria and pass/fail conditions at the system boundary -- Coverage mapping of system requirements to system-level test scenarios +# Subsystem Verification Design (MANDATORY) -## Subsystem Verification Design (MANDATORY) +Place `{subsystem-name}.md` in the **parent** folder; create `{subsystem-name}/` for children: -For each subsystem, create a kebab-case folder and `{subsystem-name}.md` covering: +- **Verification Approach**: integration test approach and mocking at subsystem boundary +- **Test Environment**: any environment setup beyond the standard test runner +- **Acceptance Criteria**: what constitutes a passing subsystem test (IEC 62304 §5.5.2) +- **Test Scenarios**: named scenarios including boundary conditions, error paths, and normal operation -- Subsystem verification strategy and integration test approach -- Dependencies that must be mocked or stubbed at the subsystem boundary -- Integration test scenarios covering subsystem requirements -- Coverage mapping of subsystem requirements to subsystem-level test scenarios +# Unit Verification Design (MANDATORY) -## Unit Verification Design (MANDATORY) +Place `{unit-name}.md` in the **parent** folder: -For each unit, create `{unit-name}.md` covering: +- **Verification Approach**: what is mocked/stubbed and why; injected vs. real dependencies +- **Test Environment**: any environment setup beyond the standard test runner +- **Acceptance Criteria**: what constitutes passing unit tests (IEC 62304 §5.5.2) +- **Test Scenarios**: named scenarios including boundary values, error paths, and normal operation -- Verification approach for each unit requirement -- Named test scenarios including boundary conditions, error paths, and normal-operation cases -- Which dependencies are mocked and how they are configured -- Coverage mapping of every unit requirement to at least one named test scenario +# OTS Verification Evidence (when OTS items exist) -## OTS Verification Evidence (when OTS items are used) +Create `docs/verification/ots.md` (`#` heading) covering the overall OTS verification strategy. -For each OTS item, create `docs/verification/ots/{ots-name}.md` covering: +For each OTS item, create `docs/verification/ots/{ots-name}.md` (`##` heading) covering: +verification approach (self-validation, integration tests, vendor evidence). -- The OTS item's required functionality (reference `docs/reqstream/ots/{ots-name}.yaml`) -- Verification of each requirement (using self-validation evidence if appropriate) -- Coverage mapping of OTS requirements to test scenarios +# Shared Package Verification Evidence (when Shared Packages exist) -# Writing Guidelines +Create `docs/verification/shared.md` (`#` heading) covering the overall Shared Package verification strategy. -- **Test Coverage**: Map every requirement to at least one named test scenario so - reviewers can verify completeness without reading test code -- **Scenario Clarity**: Name each scenario clearly - "Valid input returns parsed result" not "Test 1" -- **Boundary Conditions**: Call out boundary values, error inputs, and edge cases explicitly -- **Isolation Strategy**: Describe what is mocked or stubbed and why at each level -- **Traceability**: Link to requirements where applicable using ReqStream patterns -- **Verbal Cross-References**: Reference other documents by name - do not use markdown - hyperlinks, which break in compiled PDFs +For each Shared Package, create `docs/verification/shared/{package-name}.md` (`##` heading) covering: +verification approach. + +# Writing Guidelines -Mermaid diagrams may supplement text descriptions where test flow benefits from visual -representation, but must not replace text content. +- Name scenarios clearly ("Valid input returns parsed result", not "Test 1") +- Use verbal cross-references - not markdown hyperlinks (break in PDF) +- Use Mermaid diagrams to supplement (not replace) text # Quality Checks -Before submitting verification documentation, verify: - -- [ ] Every requirement at each level is mapped to at least one named test scenario -- [ ] System verification documents cover end-to-end and integration scenarios -- [ ] Subsystem verification documents identify mocked boundaries and integration scenarios -- [ ] Unit verification documents identify individual scenarios including boundary and error paths -- [ ] Subsystem documentation folders use kebab-case names mirroring the source subsystem structure -- [ ] All documents follow technical documentation formatting standards -- [ ] Content is current with requirements and test implementation -- [ ] Every OTS item has `docs/verification/ots/{ots-name}.md` with requirement coverage -- [ ] Documents are integrated into ReviewMark review-sets for formal review +- [ ] `introduction.md` includes Companion Artifact Structure +- [ ] Each file's heading depth matches its folder depth +- [ ] All folders use kebab-case mirroring source structure +- [ ] Each system/subsystem/unit file includes all mandatory sections (Verification Approach, + Test Environment, Acceptance Criteria, Test Scenarios) +- [ ] Non-applicable mandatory sections contain "N/A - {justification}" +- [ ] Requirements-to-test coverage is tracked via the ReqStream trace matrix, not in these documents +- [ ] `docs/verification/ots.md` and `docs/verification/ots/{ots-name}.md` exist when OTS items are present +- [ ] `docs/verification/shared.md` and `docs/verification/shared/{package-name}.md` exist when Shared Packages are present +- [ ] Documents are integrated into ReviewMark review-sets diff --git a/.github/workflows/build.yaml b/.github/workflows/build.yaml index 61d6257..c618ce8 100644 --- a/.github/workflows/build.yaml +++ b/.github/workflows/build.yaml @@ -446,8 +446,25 @@ jobs: dotnet-version: 10.x - name: Install npm dependencies + env: + PUPPETEER_SKIP_DOWNLOAD: "true" run: npm install + - name: Set browser path for Mermaid (Windows) + shell: pwsh + run: | + $chromePaths = @( + "C:\Program Files\Google\Chrome\Application\chrome.exe", + "C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" + ) + foreach ($path in $chromePaths) { + if (Test-Path $path) { + "PUPPETEER_EXECUTABLE_PATH=$path" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append + Write-Host "Set PUPPETEER_EXECUTABLE_PATH to $path" + break + } + } + - name: Restore Tools run: dotnet tool restore diff --git a/.reviewmark.yaml b/.reviewmark.yaml index f156aaa..9ecf55e 100644 --- a/.reviewmark.yaml +++ b/.reviewmark.yaml @@ -26,54 +26,78 @@ evidence-source: type: url location: https://raw.githubusercontent.com/demaconsulting/TemplateDotNetTool/reviews/index.json +# Context included for all reviews. +context: + - docs/design/introduction.md + # Review sets following hierarchical scope principles. # Each review-set focuses on a single compliance question with manageable file counts. reviews: # Purpose - id: Purpose title: Review of user-facing capabilities and system promises + context: + - "!docs/design/introduction.md" paths: - "README.md" - "docs/user_guide/**/*.md" - - "docs/reqstream/template-dot-net-tool/template-dot-net-tool.yaml" - - "docs/design/introduction.md" - - "docs/design/template-dot-net-tool/template-dot-net-tool.md" + + # Decomposition Review (only one per repository) + - id: Decomposition + title: Review that TemplateDotNetTool Decomposition Addresses the Stated Purpose + context: + - README.md + - "docs/user_guide/**/*.md" + paths: + - requirements.yaml + - docs/design/introduction.md # TemplateTool - Specials - id: TemplateTool-Architecture title: Review of Template DotNet Tool system architecture and operational validation + context: + - README.md + - "docs/user_guide/**/*.md" paths: - - "docs/reqstream/template-dot-net-tool/template-dot-net-tool.yaml" + - "docs/reqstream/template-dot-net-tool.yaml" - "docs/design/introduction.md" - - "docs/design/template-dot-net-tool/template-dot-net-tool.md" - - "docs/verification/template-dot-net-tool/template-dot-net-tool.md" + - "docs/design/template-dot-net-tool.md" + - "docs/verification/introduction.md" + - "docs/verification/template-dot-net-tool.md" - "test/**/IntegrationTests.cs" - id: TemplateTool-Design title: Review of Template DotNet Tool architectural and design consistency + context: + - docs/reqstream/template-dot-net-tool.yaml paths: - - "docs/reqstream/template-dot-net-tool/template-dot-net-tool.yaml" - "docs/reqstream/template-dot-net-tool/platform-requirements.yaml" - "docs/design/introduction.md" - "docs/design/template-dot-net-tool/**/*.md" - - "docs/verification/introduction.md" - id: TemplateTool-AllRequirements title: Review of Template DotNet Tool requirements quality and traceability + context: + - docs/design/template-dot-net-tool.md + - docs/reqstream/template-dot-net-tool.yaml paths: - - "requirements.yaml" - "docs/reqstream/template-dot-net-tool/**/*.yaml" - - "docs/reqstream/ots/*.yaml" - id: OTS-Verification title: Review of OTS software verification evidence paths: - "docs/reqstream/ots/*.yaml" + - "docs/design/ots.md" + - "docs/design/ots/*.md" + - "docs/verification/ots.md" - "docs/verification/ots/*.md" # TemplateTool - Program - id: TemplateTool-Program title: Review of Template DotNet Tool Program unit implementation + context: + - docs/design/template-dot-net-tool.md + - docs/reqstream/template-dot-net-tool.yaml paths: - "docs/reqstream/template-dot-net-tool/program.yaml" - "docs/design/template-dot-net-tool/program.md" @@ -84,14 +108,22 @@ reviews: # TemplateTool - Cli - id: TemplateTool-Cli title: Review of Template DotNet Tool Cli subsystem architecture and interfaces + context: + - docs/design/template-dot-net-tool.md + - docs/reqstream/template-dot-net-tool.yaml paths: - - "docs/reqstream/template-dot-net-tool/cli/cli.yaml" - - "docs/design/template-dot-net-tool/cli/cli.md" - - "docs/verification/template-dot-net-tool/cli/cli.md" + - "docs/reqstream/template-dot-net-tool/cli.yaml" + - "docs/design/template-dot-net-tool/cli.md" + - "docs/verification/template-dot-net-tool/cli.md" - "test/**/Cli/CliSubsystemTests.cs" - id: TemplateTool-Cli-Context title: Review of Template DotNet Tool Context unit implementation + context: + - docs/design/template-dot-net-tool.md + - docs/reqstream/template-dot-net-tool.yaml + - docs/design/template-dot-net-tool/cli.md + - docs/reqstream/template-dot-net-tool/cli.yaml paths: - "docs/reqstream/template-dot-net-tool/cli/context.yaml" - "docs/design/template-dot-net-tool/cli/context.md" @@ -102,14 +134,22 @@ reviews: # TemplateTool - SelfTest - id: TemplateTool-SelfTest title: Review of Template DotNet Tool SelfTest subsystem architecture and interfaces + context: + - docs/design/template-dot-net-tool.md + - docs/reqstream/template-dot-net-tool.yaml paths: - - "docs/reqstream/template-dot-net-tool/self-test/self-test.yaml" - - "docs/design/template-dot-net-tool/self-test/self-test.md" - - "docs/verification/template-dot-net-tool/self-test/self-test.md" + - "docs/reqstream/template-dot-net-tool/self-test.yaml" + - "docs/design/template-dot-net-tool/self-test.md" + - "docs/verification/template-dot-net-tool/self-test.md" - "test/**/SelfTest/SelfTestSubsystemTests.cs" - id: TemplateTool-SelfTest-Validation title: Review of Template DotNet Tool Validation unit implementation + context: + - docs/design/template-dot-net-tool.md + - docs/reqstream/template-dot-net-tool.yaml + - docs/design/template-dot-net-tool/self-test.md + - docs/reqstream/template-dot-net-tool/self-test.yaml paths: - "docs/reqstream/template-dot-net-tool/self-test/validation.yaml" - "docs/design/template-dot-net-tool/self-test/validation.md" @@ -120,14 +160,22 @@ reviews: # TemplateTool - Utilities - id: TemplateTool-Utilities title: Review of Template DotNet Tool Utilities subsystem architecture and interfaces + context: + - docs/design/template-dot-net-tool.md + - docs/reqstream/template-dot-net-tool.yaml paths: - - "docs/reqstream/template-dot-net-tool/utilities/utilities.yaml" - - "docs/design/template-dot-net-tool/utilities/utilities.md" - - "docs/verification/template-dot-net-tool/utilities/utilities.md" + - "docs/reqstream/template-dot-net-tool/utilities.yaml" + - "docs/design/template-dot-net-tool/utilities.md" + - "docs/verification/template-dot-net-tool/utilities.md" - "test/**/Utilities/UtilitiesSubsystemTests.cs" - id: TemplateTool-Utilities-PathHelpers title: Review of Template DotNet Tool PathHelpers unit implementation + context: + - docs/design/template-dot-net-tool.md + - docs/reqstream/template-dot-net-tool.yaml + - docs/design/template-dot-net-tool/utilities.md + - docs/reqstream/template-dot-net-tool/utilities.yaml paths: - "docs/reqstream/template-dot-net-tool/utilities/path-helpers.yaml" - "docs/design/template-dot-net-tool/utilities/path-helpers.md" @@ -140,58 +188,68 @@ reviews: title: Review of BuildMark OTS verification evidence paths: - "docs/reqstream/ots/buildmark.yaml" + - "docs/design/ots/buildmark.md" - "docs/verification/ots/buildmark.md" - id: OTS-FileAssert title: Review of FileAssert OTS verification evidence paths: - "docs/reqstream/ots/fileassert.yaml" + - "docs/design/ots/fileassert.md" - "docs/verification/ots/fileassert.md" - id: OTS-Pandoc title: Review of Pandoc OTS verification evidence paths: - "docs/reqstream/ots/pandoc.yaml" + - "docs/design/ots/pandoc.md" - "docs/verification/ots/pandoc.md" - id: OTS-ReqStream title: Review of ReqStream OTS verification evidence paths: - "docs/reqstream/ots/reqstream.yaml" + - "docs/design/ots/reqstream.md" - "docs/verification/ots/reqstream.md" - id: OTS-ReviewMark title: Review of ReviewMark OTS verification evidence paths: - "docs/reqstream/ots/reviewmark.yaml" + - "docs/design/ots/reviewmark.md" - "docs/verification/ots/reviewmark.md" - id: OTS-SarifMark title: Review of SarifMark OTS verification evidence paths: - "docs/reqstream/ots/sarifmark.yaml" + - "docs/design/ots/sarifmark.md" - "docs/verification/ots/sarifmark.md" - id: OTS-SonarMark title: Review of SonarMark OTS verification evidence paths: - "docs/reqstream/ots/sonarmark.yaml" + - "docs/design/ots/sonarmark.md" - "docs/verification/ots/sonarmark.md" - id: OTS-VersionMark title: Review of VersionMark OTS verification evidence paths: - "docs/reqstream/ots/versionmark.yaml" + - "docs/design/ots/versionmark.md" - "docs/verification/ots/versionmark.md" - id: OTS-WeasyPrint title: Review of WeasyPrint OTS verification evidence paths: - "docs/reqstream/ots/weasyprint.yaml" + - "docs/design/ots/weasyprint.md" - "docs/verification/ots/weasyprint.md" - id: OTS-xUnit title: Review of xUnit OTS verification evidence paths: - "docs/reqstream/ots/xunit.yaml" + - "docs/design/ots/xunit.md" - "docs/verification/ots/xunit.md" diff --git a/AGENTS.md b/AGENTS.md index 7a629b5..08949f4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,12 +1,16 @@ # Project Overview > **Downstream customization required**: Replace the `TODO` values below with -> values specific to the target repository. +> values specific to the target repository. These values are the canonical source +> for project-wide template placeholders used throughout the repository. -- **name**: TODO -- **description**: TODO -- **languages**: TODO -- **technologies**: TODO +- **project-name**: TODO — repository/project name (e.g., `ReqStream`) +- **organization**: TODO — organization name for document authorship (e.g., `DEMA Consulting`) +- **project-tagline**: TODO — document subtitle; 4–8 words, noun phrase, no period + (e.g., `Requirements traceability for .NET projects`) +- **description**: TODO — full project description, may be multiple sentences +- **languages**: TODO — programming languages used (e.g., `C#`, `C++`) +- **technologies**: TODO — key technologies and frameworks (e.g., `.NET`, `CMake`) # Project Structure @@ -31,6 +35,19 @@ └── {test-project}/ ``` +# Language and Spelling (ALL Agents) + +Always use **US English** spelling in all output (code, comments, documentation, +commit messages, and reports). + +# Reference Template + +This repository follows a reference template for structure and file conventions. + +- **template-url**: `https://github.com/demaconsulting/Agents/raw/refs/heads/template` +- **Repository map**: `{template-url}/repository-map.md` +- **Template files**: `{template-url}/{file-path}` for files described in the map + # Codebase Navigation (ALL Agents) When working with source code, design, or requirements artifacts, read @@ -59,17 +76,15 @@ before searching the filesystem. Before performing any work, agents must read and apply the relevant standards from `.github/standards/`. Use this matrix to determine which to load: -| Work involves... | Load these standards | -|----------------------|------------------------------------------------------------------------------------| -| Any code | `coding-principles.md` | -| C# code | `coding-principles.md`, `csharp-language.md` | -| Any tests | `testing-principles.md` | -| C# tests | `testing-principles.md`, `csharp-testing.md` | -| Requirements | `requirements-principles.md`, `software-items.md`, `reqstream-usage.md` | -| Design docs | `software-items.md`, `design-documentation.md`, `technical-documentation.md` | -| Verification docs | `software-items.md`, `verification-documentation.md`, `technical-documentation.md` | -| Review configuration | `software-items.md`, `reviewmark-usage.md` | -| Any documentation | `technical-documentation.md` | +- **Any code**: `coding-principles.md` +- **C# code**: `coding-principles.md`, `csharp-language.md` +- **Any tests**: `testing-principles.md` +- **C# tests**: `testing-principles.md`, `csharp-testing.md` +- **Requirements**: `requirements-principles.md`, `software-items.md`, `reqstream-usage.md` +- **Design docs**: `software-items.md`, `design-documentation.md`, `technical-documentation.md` +- **Verification docs**: `software-items.md`, `verification-documentation.md`, `technical-documentation.md` +- **Review configuration**: `software-items.md`, `reviewmark-usage.md` +- **Any documentation**: `technical-documentation.md` Load only the standards relevant to your specific task scope. @@ -79,11 +94,15 @@ The default agent should handle simple, straightforward tasks directly. Delegate to specialized agents only for specific scenarios: - **Pre-PR lint cleanup** (fix all lint issues before pull request) → Call the lint-fix agent -- **Light development work** (small fixes, simple features) → Call the developer agent +- **Scoped fixes with no new user-visible behavior** (PR review comments, doc + corrections, known bug fixes with defined root cause) → Call the developer agent - **Light quality checking** (basic validation) → Call the quality agent -- **Formal feature implementation** (complex, multi-step) → Call the implementation agent -- **Formal bug resolution** (complex debugging, systematic fixes) → Call the implementation agent +- **Any change introducing new user-visible behavior** (features, enhancements, + new commands or options) → Call the implementation agent +- **Formal bug resolution** (complex debugging, unknown root cause) → Call the implementation agent - **Formal reviews** (compliance verification, detailed analysis) → Call the formal-review agent +- **Structural audit**: (repository layout vs. template) → Call the template-sync agent +- **Implementation planning only** (review a plan before committing to implementation) → Call the planning agent # Agent Reporting (Specialized Agents Must Follow) @@ -92,21 +111,21 @@ Specialized agents MUST generate a completion report: 1. Save to `.agent-logs/{agent-name}-{subject}-{unique-id}.md` where `{subject}` is a kebab-case task summary (max 5 words) and `{unique-id}` is a short unique suffix (e.g., 8-char hex or timestamp) -2. Start with `**Result**: (SUCCEEDED|FAILED)` as the first metadata field +2. Start with `**Result**: (SUCCEEDED|FAILED|INCOMPLETE)` as the first metadata field 3. Include the agent-specific report sections defined in each agent's prompt 4. Return the summary to the caller Result semantics for orchestrator decision-making: -- **SUCCEEDED**: Work completed and all applicable quality gates met +- **SUCCEEDED**: Work completed and all quality gates applicable to that agent's scope met - **FAILED**: Work could not be completed or quality gates not met - **INCOMPLETE**: Work cannot proceed without information only the user can - provide (implementation agent only) + provide (implementation, planning, and template-sync agents) # Formatting (After Making Changes) After making changes, run the auto-fix pass. This applies all available fixers -silently and **always exits 0** — agents do not need to respond to its output. +silently and **always exits 0** - agents do not need to respond to its output. ```pwsh pwsh ./fix.ps1 @@ -114,7 +133,7 @@ pwsh ./fix.ps1 This automatically handles: `dotnet format`, markdown formatting, and YAML formatting. Full lint compliance is a **pre-PR responsibility**, not an agent -responsibility — invoke the lint-fix agent once before submitting a pull request. +responsibility - invoke the lint-fix agent once before submitting a pull request. ## CI Quality Tools @@ -124,7 +143,7 @@ reqstream, versionmark, and reviewmark. # Scope Discipline (ALL Agents Must Follow) - **No generated file access**: Files inside any `generated/` folder are build - outputs — do not read, lint, or modify them + outputs - do not read, lint, or modify them - **Minimum necessary changes**: Only modify files directly required by the task - **No speculative refactoring**: Do not refactor code adjacent to the change unless the task explicitly requests it diff --git a/README.md b/README.md index 5f4deb4..b42addb 100644 --- a/README.md +++ b/README.md @@ -139,6 +139,11 @@ Generated documentation includes: - **Requirements Justifications**: Detailed requirement rationale - **Trace Matrix**: Requirements to test traceability +## Contributing + +See [CONTRIBUTING.md](https://github.com/demaconsulting/TemplateDotNetTool/blob/main/CONTRIBUTING.md) for +guidelines on reporting bugs, suggesting features, and submitting pull requests. + ## License Copyright (c) DEMA Consulting. Licensed under the MIT License. See [LICENSE][link-license] for details. diff --git a/docs/design/definition.yaml b/docs/design/definition.yaml index c18063a..506d1a9 100644 --- a/docs/design/definition.yaml +++ b/docs/design/definition.yaml @@ -5,19 +5,31 @@ resource-path: - docs/design/template-dot-net-tool/cli - docs/design/template-dot-net-tool/self-test - docs/design/template-dot-net-tool/utilities + - docs/design/ots - docs/template input-files: - docs/design/title.txt - docs/design/introduction.md - - docs/design/template-dot-net-tool/template-dot-net-tool.md + - docs/design/template-dot-net-tool.md - docs/design/template-dot-net-tool/program.md - - docs/design/template-dot-net-tool/cli/cli.md + - docs/design/template-dot-net-tool/cli.md - docs/design/template-dot-net-tool/cli/context.md - - docs/design/template-dot-net-tool/self-test/self-test.md + - docs/design/template-dot-net-tool/self-test.md - docs/design/template-dot-net-tool/self-test/validation.md - - docs/design/template-dot-net-tool/utilities/utilities.md + - docs/design/template-dot-net-tool/utilities.md - docs/design/template-dot-net-tool/utilities/path-helpers.md + - docs/design/ots.md + - docs/design/ots/buildmark.md + - docs/design/ots/fileassert.md + - docs/design/ots/pandoc.md + - docs/design/ots/reqstream.md + - docs/design/ots/reviewmark.md + - docs/design/ots/sarifmark.md + - docs/design/ots/sonarmark.md + - docs/design/ots/versionmark.md + - docs/design/ots/weasyprint.md + - docs/design/ots/xunit.md template: template.html table-of-contents: true number-sections: true diff --git a/docs/design/introduction.md b/docs/design/introduction.md index 3317e4c..7ba656b 100644 --- a/docs/design/introduction.md +++ b/docs/design/introduction.md @@ -13,22 +13,36 @@ how they are realized. ## Scope -This document covers the detailed design of the following subsystems and software units: +This document covers the detailed design of the following software items: -- **TemplateDotNetTool** — the system as a whole (`template-dot-net-tool.md`) -- **Program** — entry point and execution orchestrator (`Program.cs`) +**Local items:** + +- **TemplateDotNetTool** — the system as a whole +- **Program** — entry point and execution orchestrator - **Cli** subsystem - - **Context** — command-line argument parser and I/O owner (`Cli/Context.cs`) + - **Context** — command-line argument parser and I/O owner - **SelfTest** subsystem - - **Validation** — self-validation test runner (`SelfTest/Validation.cs`) + - **Validation** — self-validation test runner - **Utilities** subsystem - - **PathHelpers** — safe path combination utilities (`Utilities/PathHelpers.cs`) + - **PathHelpers** — safe path combination utilities + +**OTS items:** + +- **BuildMark** — integration and usage design +- **FileAssert** — integration and usage design +- **Pandoc** — integration and usage design +- **ReqStream** — integration and usage design +- **ReviewMark** — integration and usage design +- **SarifMark** — integration and usage design +- **SonarMark** — integration and usage design +- **VersionMark** — integration and usage design +- **WeasyPrint** — integration and usage design +- **xUnit** — integration and usage design The following topics are out of scope: -- External library internals -- Build pipeline configuration -- Deployment and packaging +- Design documents are not produced for the test projects or build pipeline CI configuration +- The internal design of OTS software items is excluded; only integration and usage design is documented ## Software Structure @@ -46,7 +60,20 @@ TemplateDotNetTool (System) └── PathHelpers (Unit) ``` -Each unit is described in detail in its own chapter within this document. +**OTS Dependencies:** + +- BuildMark (OTS) — build-notes documentation tool +- FileAssert (OTS) — document assertion tool +- Pandoc (OTS) — Markdown-to-HTML conversion tool +- ReqStream (OTS) — requirements traceability tool +- ReviewMark (OTS) — file review enforcement tool +- SarifMark (OTS) — SARIF report conversion tool +- SonarMark (OTS) — SonarCloud quality report tool +- VersionMark (OTS) — tool-version documentation tool +- WeasyPrint (OTS) — HTML-to-PDF conversion tool +- xUnit (OTS) — unit-testing framework + +Each local unit is described in detail in its own chapter within this document. ## Folder Layout @@ -54,18 +81,13 @@ The source code folder structure mirrors the top-level subsystem breakdown above reviewers an explicit navigation aid from design to code: ```text -src/DemaConsulting.TemplateDotNetTool/ -├── Program.cs — entry point and execution orchestrator -├── Cli/ -│ └── Context.cs — command-line argument parser and I/O owner -├── SelfTest/ -│ └── Validation.cs — self-validation test runner -└── Utilities/ - └── PathHelpers.cs — safe path combination utilities +src/ +└── DemaConsulting.TemplateDotNetTool/ — main application source + ├── Cli/ — command-line interface subsystem + ├── SelfTest/ — self-validation subsystem + └── Utilities/ — shared utilities subsystem ``` -The test project mirrors the same layout under `test/DemaConsulting.TemplateDotNetTool.Tests/`. - ## Document Conventions Throughout this document: @@ -78,16 +100,23 @@ Throughout this document: ## Companion Artifact Structure -Each software item in the structure above has corresponding artifacts in parallel directory trees: +Local software items have corresponding artifacts in parallel directory trees: - Requirements: `docs/reqstream/{system}/.../{item}.yaml` (kebab-case) - Design docs: `docs/design/{system}/.../{item}.md` (kebab-case) - Verification design: `docs/verification/{system}/.../{item}.md` (kebab-case) - Source code: `src/{System}/.../{Item}.cs` (PascalCase for C#) - Tests: `test/{System}.Tests/.../{Item}Tests.cs` (PascalCase for C#) -- Review-sets: defined in `.reviewmark.yaml` + +OTS items have integration/usage design documentation parallel to system folders: + +- Requirements: `docs/reqstream/ots/{ots-name}.yaml` +- Design: `docs/design/ots/{ots-name}.md` +- Verification: `docs/verification/ots/{ots-name}.md` + +Review-sets: defined in `.reviewmark.yaml` ## References -- [REF-1] Template DotNet Tool User Guide — `docs/user_guide/introduction.md` -- [REF-2] Template DotNet Tool Repository — `https://github.com/demaconsulting/TemplateDotNetTool` +- Template DotNet Tool User Guide +- Template DotNet Tool Repository () diff --git a/docs/design/ots.md b/docs/design/ots.md new file mode 100644 index 0000000..c3abed9 --- /dev/null +++ b/docs/design/ots.md @@ -0,0 +1,29 @@ +# OTS Software Integration Design + +This document provides the integration and usage design for all Off-The-Shelf (OTS) software +items used by the Template DotNet Tool. + +## Scope + +OTS items are third-party tools and libraries consumed by the project. This document covers the +integration pattern and usage design for each item. The internal design of OTS items is out of +scope; only how this project integrates and uses each item is documented. + +## OTS Items + +The following OTS items have integration design documentation: + +- BuildMark (_buildmark.md_) — build-notes documentation tool +- FileAssert (_fileassert.md_) — document assertion tool +- Pandoc (_pandoc.md_) — Markdown-to-HTML conversion tool +- ReqStream (_reqstream.md_) — requirements traceability tool +- ReviewMark (_reviewmark.md_) — file review enforcement tool +- SarifMark (_sarifmark.md_) — SARIF report conversion tool +- SonarMark (_sonarmark.md_) — SonarCloud quality report tool +- VersionMark (_versionmark.md_) — tool-version documentation tool +- WeasyPrint (_weasyprint.md_) — HTML-to-PDF conversion tool +- xUnit (_xunit.md_) — unit-testing framework + +## References + +N/A diff --git a/docs/design/ots/buildmark.md b/docs/design/ots/buildmark.md new file mode 100644 index 0000000..d55544b --- /dev/null +++ b/docs/design/ots/buildmark.md @@ -0,0 +1,25 @@ +## BuildMark + +This document describes the integration and usage design for the `BuildMark` OTS software item. + +### Purpose + +BuildMark (`DemaConsulting.BuildMark`) is chosen to capture GitHub Actions workflow run metadata — +Git history, issues, and pull requests — and render it as a markdown build-notes document. It +provides automated, repeatable build-notes generation for the release artifacts so that the +project does not maintain release notes by hand. + +### Features Used + +- Workflow run metadata capture (Git tags, commit history, issues, pull requests) +- Markdown build-notes document rendering with section routing rules +- Built-in self-validation suite (`--validate`) + +### Integration Pattern + +BuildMark is consumed as a dotnet tool restored from the tool manifest. In the CI documentation +build it is invoked with live GitHub Actions metadata via `dotnet buildmark ... --output` to +produce `docs/build_notes/generated/build_notes.md`, which Pandoc then converts to HTML. Tool +qualification evidence is produced by `dotnet buildmark --validate --results +artifacts/buildmark-self-validation.trx`, whose results are consumed by ReqStream. No +initialization or disposal beyond a single process invocation is required. diff --git a/docs/design/ots/fileassert.md b/docs/design/ots/fileassert.md new file mode 100644 index 0000000..db51402 --- /dev/null +++ b/docs/design/ots/fileassert.md @@ -0,0 +1,27 @@ +## FileAssert + +This document describes the integration and usage design for the `FileAssert` OTS software item. + +### Purpose + +FileAssert (`DemaConsulting.FileAssert`) is chosen to assert that generated documents exist, have +non-trivial size, and contain the expected content. It provides the pipeline output assertions that +turn document generation into verifiable evidence for OTS tools that do not have their own dotnet +self-validation. + +### Features Used + +- File existence and size assertions +- Text content assertions +- HTML structure assertions +- PDF content assertions +- Built-in self-validation suite (`--validate`) + +### Integration Pattern + +FileAssert is consumed as a dotnet tool restored from the tool manifest. After each Pandoc and +WeasyPrint document-generation step in the CI pipeline, `dotnet fileassert --results .trx` +asserts the generated HTML and PDF documents and records results for ReqStream. Tool qualification +evidence is produced by `dotnet fileassert --validate --results +artifacts/fileassert-self-validation.trx`. Each invocation is a single process call with no +persistent state. diff --git a/docs/design/ots/pandoc.md b/docs/design/ots/pandoc.md new file mode 100644 index 0000000..9afca76 --- /dev/null +++ b/docs/design/ots/pandoc.md @@ -0,0 +1,23 @@ +## Pandoc + +This document describes the integration and usage design for the `Pandoc` OTS software item. + +### Purpose + +Pandoc (`DemaConsulting.PandocTool`) is chosen to convert Markdown documents to HTML as part of the +documentation build pipeline. It provides reliable, repeatable Markdown-to-HTML conversion so that +each documentation collection can be compiled into a consistent rendered form. + +### Features Used + +- Markdown-to-HTML conversion +- Document definition driven input ordering and resource resolution + +### Integration Pattern + +Pandoc is consumed as a dotnet tool restored from the tool manifest. Each documentation section +(build notes, code quality, code review, design, verification, requirements, user guide) is +converted individually with `dotnet pandoc` before WeasyPrint renders it to PDF. Pandoc does not +provide dotnet self-validation; its output is validated by FileAssert assertions on the generated +HTML, so a successful FileAssert step is the integration evidence. Each invocation is a single +process call with no persistent state. diff --git a/docs/design/ots/reqstream.md b/docs/design/ots/reqstream.md new file mode 100644 index 0000000..ed2ee98 --- /dev/null +++ b/docs/design/ots/reqstream.md @@ -0,0 +1,26 @@ +## ReqStream + +This document describes the integration and usage design for the `ReqStream` OTS software item. + +### Purpose + +ReqStream (`DemaConsulting.ReqStream`) is chosen to process requirements YAML files and TRX test +result files to generate a requirements report, justifications document, and traceability matrix. +It provides the requirements-to-test traceability evidence required for compliance. + +### Features Used + +- Requirements YAML processing via the `requirements.yaml` includes chain +- TRX test result consumption for traceability +- Requirements, justifications, and trace-matrix report generation +- Enforcement mode (`--enforce`) that fails the build when a requirement lacks test evidence +- Built-in self-validation suite (`--validate`) + +### Integration Pattern + +ReqStream is consumed as a dotnet tool restored from the tool manifest. In the CI pipeline it runs +with `--enforce` against all previously generated TRX evidence to fail the build if any requirement +lacks passing test evidence, then generates the requirements and trace-matrix documents that Pandoc +compiles. Tool qualification evidence is produced by `dotnet reqstream --validate --results +artifacts/reqstream-self-validation.trx`. Each invocation is a single process call with no +persistent state. diff --git a/docs/design/ots/reviewmark.md b/docs/design/ots/reviewmark.md new file mode 100644 index 0000000..496c1c1 --- /dev/null +++ b/docs/design/ots/reviewmark.md @@ -0,0 +1,32 @@ +## ReviewMark + +This document describes the integration and usage design for the `ReviewMark` OTS software item. + +### Purpose + +ReviewMark (`DemaConsulting.ReviewMark`) is chosen to enforce file review currency based on the +`.reviewmark.yaml` configuration. It provides the formal review tracking and enforcement evidence +required for compliance, ensuring every file in every review-set has current review evidence. + +### Features Used + +- Review plan and report generation (`--plan` / `--report`) +- Configuration validation (`--lint`) +- Review-set elaboration for agent consumption (`--elaborate`) +- Review currency enforcement (`--enforce`) +- Built-in self-validation suite (`--validate`) + +### Integration Pattern + +ReviewMark is consumed as a dotnet tool restored from the tool manifest. It is used in several +contexts: + +- `--plan` / `--report` generate the review plan and report documents that Pandoc compiles +- `--lint` validates `.reviewmark.yaml` and is run in `lint.ps1` +- `--elaborate` expands review-set file lists for use by the formal-review agent +- `--enforce` enforces review currency and is run in the CI build, making stale evidence a + build-breaking condition + +Tool qualification evidence is produced by `dotnet reviewmark --validate --results +artifacts/reviewmark-self-validation.trx`. Each invocation is a single process call with no +persistent state. diff --git a/docs/design/ots/sarifmark.md b/docs/design/ots/sarifmark.md new file mode 100644 index 0000000..88898a1 --- /dev/null +++ b/docs/design/ots/sarifmark.md @@ -0,0 +1,24 @@ +## SarifMark + +This document describes the integration and usage design for the `SarifMark` OTS software item. + +### Purpose + +SarifMark (`DemaConsulting.SarifMark`) is chosen to read CodeQL SARIF output and render a +human-readable markdown code-quality report. It provides automated conversion of static-analysis +findings into a reviewable document and can fail the build when issues are detected. + +### Features Used + +- SARIF report reading +- Markdown code-quality report rendering +- Enforcement mode (`--enforce`) that fails the build when SARIF issues are detected +- Built-in self-validation suite (`--validate`) + +### Integration Pattern + +SarifMark is consumed as a dotnet tool restored from the tool manifest. In the CI pipeline it reads +the CodeQL SARIF output and renders the markdown code-quality report that Pandoc compiles, running +with `--enforce` so that detected issues break the build. Tool qualification evidence is produced +by `dotnet sarifmark --validate --results artifacts/sarifmark-self-validation.trx`. Each invocation +is a single process call with no persistent state. diff --git a/docs/design/ots/sonarmark.md b/docs/design/ots/sonarmark.md new file mode 100644 index 0000000..1f46cac --- /dev/null +++ b/docs/design/ots/sonarmark.md @@ -0,0 +1,24 @@ +## SonarMark + +This document describes the integration and usage design for the `SonarMark` OTS software item. + +### Purpose + +SonarMark (`DemaConsulting.SonarMark`) is chosen to retrieve quality-gate and metrics data from +SonarCloud and render a markdown code-quality report. It provides automated inclusion of SonarCloud +quality results in the project's documentation artifacts. + +### Features Used + +- SonarCloud quality-gate retrieval +- SonarCloud issues and hot-spots retrieval +- Markdown code-quality report rendering +- Built-in self-validation suite (`--validate`) + +### Integration Pattern + +SonarMark is consumed as a dotnet tool restored from the tool manifest. In the CI pipeline it +retrieves quality-gate and metrics data from SonarCloud and renders the markdown code-quality +report that Pandoc compiles. Tool qualification evidence is produced by `dotnet sonarmark +--validate --results artifacts/sonarmark-self-validation.trx`. Each invocation is a single process +call with no persistent state. diff --git a/docs/design/ots/versionmark.md b/docs/design/ots/versionmark.md new file mode 100644 index 0000000..47823c6 --- /dev/null +++ b/docs/design/ots/versionmark.md @@ -0,0 +1,25 @@ +## VersionMark + +This document describes the integration and usage design for the `VersionMark` OTS software item. + +### Purpose + +VersionMark (`DemaConsulting.VersionMark`) is chosen to read version metadata for each dotnet tool +used in the pipeline and write a versions markdown document. It provides the tool-version record +required for build reproducibility and compliance evidence. + +### Features Used + +- Per-tool version metadata capture (`--capture`) +- Versions markdown document generation (`--publish`) +- Configuration validation (`--lint`) +- Built-in self-validation suite (`--validate`) + +### Integration Pattern + +VersionMark is consumed as a dotnet tool restored from the tool manifest. Throughout the CI +pipeline it captures the versions of each tool used in a job, then publishes a consolidated +versions markdown document included in the build notes. `--lint` validates the configuration. Tool +qualification evidence is produced by `dotnet versionmark --validate --results +artifacts/versionmark-self-validation.trx`. Each invocation is a single process call with no +persistent state. diff --git a/docs/design/ots/weasyprint.md b/docs/design/ots/weasyprint.md new file mode 100644 index 0000000..644b550 --- /dev/null +++ b/docs/design/ots/weasyprint.md @@ -0,0 +1,22 @@ +## WeasyPrint + +This document describes the integration and usage design for the `WeasyPrint` OTS software item. + +### Purpose + +WeasyPrint (`DemaConsulting.WeasyPrintTool`) is chosen to convert HTML documents to PDF as part of +the documentation build pipeline. It provides reliable, repeatable HTML-to-PDF rendering so that +each documentation collection can be delivered as a final PDF artifact. + +### Features Used + +- HTML-to-PDF conversion +- PDF rendering of Pandoc-generated HTML documents + +### Integration Pattern + +WeasyPrint is consumed as a dotnet tool restored from the tool manifest. Each HTML document +produced by Pandoc is converted individually with `dotnet weasyprint` to produce the final PDF +artifact. WeasyPrint does not provide dotnet self-validation; its output is validated by FileAssert +assertions on the generated PDF, so a successful FileAssert step is the integration evidence. Each +invocation is a single process call with no persistent state. diff --git a/docs/design/ots/xunit.md b/docs/design/ots/xunit.md new file mode 100644 index 0000000..8b8daaa --- /dev/null +++ b/docs/design/ots/xunit.md @@ -0,0 +1,23 @@ +## xUnit + +This document describes the integration and usage design for the `xUnit` OTS software item. + +### Purpose + +xUnit (`xunit.v3` together with `xunit.runner.visualstudio`) is chosen as the unit-testing framework +for the project. It provides test discovery and execution and emits TRX result files that serve as +the traceability evidence consumed by ReqStream. + +### Features Used + +- `[Fact]` and `[Theory]` test discovery and execution +- Visual Studio test runner integration for `dotnet test` +- TRX result file output + +### Integration Pattern + +xUnit is consumed as a NuGet package reference in the test projects. Tests run via `dotnet test`, +which discovers and executes the `[Fact]` and `[Theory]` tests and writes TRX result files that +ReqStream consumes for requirements traceability. xUnit does not provide a self-validation mode; the +project's own passing test suite is the integration evidence. No additional initialization or +disposal is required beyond the standard test runner lifecycle. diff --git a/docs/design/template-dot-net-tool.md b/docs/design/template-dot-net-tool.md new file mode 100644 index 0000000..669ac6e --- /dev/null +++ b/docs/design/template-dot-net-tool.md @@ -0,0 +1,113 @@ +# TemplateDotNetTool + +## Architecture + +The Template DotNet Tool is a command-line application built on .NET. It is structured as one +system containing one top-level unit (`Program`) and three subsystems (`Cli`, `SelfTest`, +`Utilities`): + +```mermaid +flowchart TD + Program + subgraph Cli + Context + end + subgraph SelfTest + Validation + end + subgraph Utilities + PathHelpers + end + Program --> Context + Program --> Validation + Validation --> Program + Validation --> PathHelpers +``` + +`Program` is the entry point. It creates a `Context` from the `Cli` subsystem, dispatches to +`Validation` when `--validate` is passed, and returns the exit code from `Context`. `Validation` +calls `Program.Run` recursively to exercise the tool during self-testing, and uses `PathHelpers` +to construct safe temporary file paths. + +## External Interfaces + +**Command-Line Interface**: The primary input interface for tool invocation. + +- *Type*: CLI. +- *Role*: Consumer (the host environment invokes the system with command-line arguments). +- *Contract*: Accepts arguments `-v`/`--version`, `-?`/`-h`/`--help`, `--silent`, `--validate`, + `--results `, `--result ` (legacy alias for `--results`), `--depth `, and + `--log `. Returns exit code 0 for success and 1 for failures. +- *Constraints*: Unknown arguments cause exit code 1 and an error message on stderr. + +**Standard Output**: Normal program output written to `Console.Out`. + +- *Type*: Standard I/O. +- *Role*: Provider. +- *Contract*: Writes version, banner, help text, validation summary, or demo message depending + on the flags provided. Suppressed when `--silent` is active; the log file still receives all + output. +- *Constraints*: Human-readable text; no machine-parseable format contract. + +**Standard Error**: Error message output written to `Console.Error`. + +- *Type*: Standard I/O. +- *Role*: Provider. +- *Contract*: Writes error messages in red when a failure occurs. Suppressed when `--silent` + is active but the exit code is still set to 1. +- *Constraints*: Color output requires a terminal that supports `ConsoleColor`. + +**Log File**: Optional persistent output file. + +- *Type*: File. +- *Role*: Provider. +- *Contract*: When `--log ` is supplied, all `Context.WriteLine` and `Context.WriteError` + output is written to the file regardless of `--silent`. The file is truncated at open. +- *Constraints*: The path must be writable; failure to open the file raises an + `InvalidOperationException` and causes exit code 1. + +**Results File**: Optional self-validation results file. + +- *Type*: File. +- *Role*: Provider. +- *Contract*: When `--results ` is supplied alongside `--validate`, self-validation results + are serialized to the file. Extension `.trx` selects MSTest TRX format; `.xml` selects JUnit + XML format. +- *Constraints*: Any other extension causes an error message and exit code 1; no file is written. + +## Dependencies + +- **DemaConsulting.TestResults**: provides `TestResults`, `TestResult`, and `TestOutcome` for + accumulating self-validation results. +- **DemaConsulting.TestResults.IO**: provides `TrxSerializer` and `JUnitSerializer` for writing + results files. + +## Risk Control Measures + +N/A - not a safety-classified software item. + +## Data Flow + +1. The host environment starts the tool process and passes command-line arguments to + `Program.Main`. +2. `Program.Main` calls `Context.Create(args)`, which parses the arguments and opens the log + file if `--log` was specified. An `ArgumentException` or `InvalidOperationException` at + this point is caught, written to stderr, and causes exit code 1. +3. `Program.Run(context)` inspects the parsed flags and dispatches to one handler: + - `--version` flag → `context.WriteLine(Version)`, then return. + - Otherwise, `PrintBanner` is called first; then: + - `--help` flag → `PrintHelp(context)`, then return. + - `--validate` flag → `Validation.Run(context)`. + - No flags → `RunToolLogic(context)`. +4. `Program.Main` returns `context.ExitCode` (0 if no errors were reported, 1 otherwise). + +## Design Constraints + +- Platform: multi-targets net8.0, net9.0, and net10.0 framework compatibility specifications on Windows, Linux, and macOS. +- Threading: single-threaded console application; no shared mutable state between invocations. +- Immutability: `Context` properties are set once at construction via `init` accessors and are + read-only thereafter. +- Resource lifecycle: `Context` implements `IDisposable`; callers must dispose it to flush and + close any open log file handle. +- Path safety: all caller-supplied path components are validated by `PathHelpers.SafePathCombine` + before file-system use. diff --git a/docs/design/template-dot-net-tool/cli.md b/docs/design/template-dot-net-tool/cli.md new file mode 100644 index 0000000..0ab2cf9 --- /dev/null +++ b/docs/design/template-dot-net-tool/cli.md @@ -0,0 +1,69 @@ +## Cli + +### Overview + +The `Cli` subsystem is the boundary between the host environment's command-line invocation and +the tool's internal logic. It owns argument parsing, output channel management, and exit-code +tracking. All other subsystems receive a `Context` object from this subsystem to read parsed +flags and write output. The `Cli` subsystem contains one unit: `Context`. + +### Interfaces + +**Context.Create**: Factory method that constructs a `Context` from a command-line argument array. + +- *Type*: In-process .NET static method. +- *Role*: Provider. +- *Contract*: Parses `string[] args` into flag properties and opens the log file if `--log` is + present. Returns a fully initialized `Context`. Accepts `--result` as a legacy alias for + `--results`. +- *Constraints*: Throws `ArgumentException` for unknown or malformed arguments; throws + `InvalidOperationException` when the log file cannot be opened. + +**Context.WriteLine**: Writes a message to stdout and to the log file. + +- *Type*: In-process .NET instance method. +- *Role*: Provider. +- *Contract*: Writes `message` to `Console.Out` and to the log file if one is open. Stdout + output is suppressed when `Silent` is true; the log file always receives the message. +- *Constraints*: None. + +**Context.WriteError**: Writes an error message and sets the error exit code. + +- *Type*: In-process .NET instance method. +- *Role*: Provider. +- *Contract*: Sets `_hasErrors` to true, writes `message` in red to `Console.Error`, and writes + to the log file if one is open. Stderr output is suppressed when `Silent` is true, but + `ExitCode` is set to 1 regardless. +- *Constraints*: Once set, `ExitCode` cannot return to 0 within the same invocation. + +**Context.ExitCode**: Derived property returning 0 or 1. + +- *Type*: In-process .NET property. +- *Role*: Provider. +- *Contract*: Returns 1 if `WriteError` has been called at least once; returns 0 otherwise. +- *Constraints*: Read-only. + +**Context.Dispose**: Releases the log file `StreamWriter`. + +- *Type*: In-process .NET method (`IDisposable`). +- *Role*: Provider. +- *Contract*: Disposes `_logWriter` and sets it to null; flushes any buffered content. Callers + must use a `using` statement to guarantee disposal. +- *Constraints*: Safe to call multiple times (idempotent after first call). + +### Design + +The `Cli` subsystem contains only the `Context` unit; there is no subsystem-level code of its +own. All behavior is provided by `Context`. The `Program` unit creates a `Context` at the start +of each invocation and passes it to all other units that produce output. + +The subsystem has no dependencies on other tool subsystems; it uses only .NET BCL types +(`Console`, `StreamWriter`). + +Error handling flows from `Context.Create` to `Program.Main`: argument parsing errors propagate +as `ArgumentException`; log-file errors propagate as `InvalidOperationException`. Both are +caught and handled in `Program.Main`, which writes the error to stderr and returns exit code 1. + +When `--log` is active, `Context` holds an open `StreamWriter` for the duration of the +invocation. The `Program.Main` call site wraps the `Context` in a `using` statement to ensure +the file handle is released and buffered content is flushed on exit. diff --git a/docs/design/template-dot-net-tool/cli/cli.md b/docs/design/template-dot-net-tool/cli/cli.md deleted file mode 100644 index 9b14817..0000000 --- a/docs/design/template-dot-net-tool/cli/cli.md +++ /dev/null @@ -1,68 +0,0 @@ -# Cli Subsystem - -The `Cli` subsystem provides the command-line interface for the Template DotNet Tool. -It is responsible for accepting user input from the command line and routing output to -the console and an optional log file. - -## Overview - -The `Cli` subsystem acts as the primary boundary between the user's shell invocation and -the tool's internal logic. It owns argument parsing, output formatting, and error tracking. -All other subsystems receive a `Context` object from the `Cli` subsystem to read parsed -flags and write output. - -## Units - -The `Cli` subsystem contains the following software unit: - -| Unit | File | Responsibility | -|-----------|------------------|---------------------------------------------------| -| `Context` | `Cli/Context.cs` | Argument parsing, output channels, and exit code. | - -## Interfaces - -The `Cli` subsystem exposes the following outbound interfaces to the rest of the tool: - -- **`Context.Create`**: Factory method constructing a `Context` from `string[] args`. - `--result` is accepted as a legacy alias for `--results`. -- **`Context.WriteLine`**: Writes a message to stdout and to the log file (if one is open). - Stdout output is suppressed when `--silent` is active; the log file always receives the - message regardless of `--silent`. -- **`Context.WriteError`**: Writes an error message to stderr and to the log file (if one is - open), and unconditionally sets the error exit code. Stderr output is suppressed when - `--silent` is active; the log file always receives the message and `ExitCode` is set to 1 - regardless of the `Silent` flag. -- **`Context.ExitCode`**: Returns 0 for success or 1 when errors have been reported. -- **`Context.HeadingDepth`**: Heading depth for markdown output (default 1); supplied via `--depth`. -- **`Context.Version`**: `true` when `-v` or `--version` was passed. -- **`Context.Help`**: `true` when `-?`, `-h`, or `--help` was passed. -- **`Context.Silent`**: `true` when `--silent` was passed. -- **`Context.Validate`**: `true` when `--validate` was passed. -- **`Context.ResultsFile`**: Path supplied after `--results` or `--result`, or `null`. -- **`Context.Dispose`**: Releases the log file `StreamWriter` if `--log` was specified. - Callers are responsible for disposal (use `using` statement). - -## Interactions - -The `Cli` subsystem has no dependencies on other tool subsystems. It uses only .NET base -class library types. The `Program` unit at system level creates the `Context` and passes it -to all subsystems that need to produce output. - -Subsystem verification uses `Program.Run` as an observable entry point alongside `Context.Create`. -This is intentional: the subsystem boundary is defined as the pair `(Context.Create, Program.Run)`, -and subsystem tests exercise both together to confirm end-to-end argument-to-behavior flow. - -## Error Handling - -`Context.Create` throws `ArgumentException` for unknown or malformed arguments. -`Program.Main` catches this exception, writes `"Error: {message}"` to stderr, and returns exit code 1. - -`Context.Create` also throws `InvalidOperationException` when `--log` specifies a file that cannot -be opened (e.g., access denied, invalid path). `Program.Main` catches this exception, writes -`"Error: {message}"` to stderr, and returns exit code 1. - -## Resource Lifecycle - -When `--log` is active, `Context` holds an open `StreamWriter` for the duration of the invocation. -Callers must ensure `Context` is disposed (via a `using` statement or explicit `Dispose` call) to -release the file handle and flush any buffered log content. diff --git a/docs/design/template-dot-net-tool/cli/context.md b/docs/design/template-dot-net-tool/cli/context.md index db718c7..6610a76 100644 --- a/docs/design/template-dot-net-tool/cli/context.md +++ b/docs/design/template-dot-net-tool/cli/context.md @@ -1,54 +1,90 @@ -# Context +### Context -The `Context` class handles command-line argument parsing and program output for the -Template DotNet Tool. It is the primary interface between the user's command-line invocation -and the tool's internal logic. +#### Purpose -## Overview +`Context` handles command-line argument parsing and program output for one tool invocation. Its +single responsibility is to parse the argument list, expose the parsed flags as read-only +properties, own the two output channels (console and log file), and derive the exit code from +whether any errors were reported. -`Context` is created once per tool invocation via the `Create` factory method. It parses -the argument list, opens any requested log file, and exposes the parsed flags as read-only -properties. It also owns the two output channels — console and log file — through its -`WriteLine` and `WriteError` methods. +#### Data Model -## Data Model +**_logWriter**: `StreamWriter?` — Log file writer; `null` when logging is not active. -| Field | Type | Description | -|----------------|-----------------|-----------------------------------------------------------------------------------| -| `_logWriter` | `StreamWriter?` | Log file writer; `null` when logging is disabled. | -| `_hasErrors` | `bool` | Set to `true` on the first `WriteError` call. | -| `Version` | `bool` | `true` when `-v` or `--version` was passed. | -| `Help` | `bool` | `true` when `-?`, `-h`, or `--help` was passed. | -| `Silent` | `bool` | `true` when `--silent` was passed. | -| `Validate` | `bool` | `true` when `--validate` was passed. | -| `ResultsFile` | `string?` | Path supplied after `--results` or `--result`, or `null`. | -| `HeadingDepth` | `int` | Heading depth for markdown output; valid range 1–6 (default 1); via `--depth`. | -| `ExitCode` | `int` | `1` if `_hasErrors`; `0` otherwise. | +**_hasErrors**: `bool` — Set to `true` on the first `WriteError` call; once set, cannot return +to `false` within the same invocation. -## Methods +**Version**: `bool` — `true` when `-v` or `--version` was present in the argument list. -### Create(string[] args) +**Help**: `bool` — `true` when `-?`, `-h`, or `--help` was present in the argument list. -Factory method. Delegates to the private `ArgumentParser` helper and opens the log file if -`--log` was supplied. +**Silent**: `bool` — `true` when `--silent` was present in the argument list. -**Throws:** `ArgumentException` — when an unknown argument or missing value is encountered. -`InvalidOperationException` — when a log file cannot be opened. +**Validate**: `bool` — `true` when `--validate` was present in the argument list. -### WriteLine(string message) +**ResultsFile**: `string?` — Path supplied after `--results` or `--result`, or `null` if +neither flag was present. -Writes `message` to `Console.Out` (unless `Silent`) and to `_logWriter` (if open). +**HeadingDepth**: `int` — Heading depth for markdown output; valid range 1–6, default 1; +supplied via `--depth`. -### WriteError(string message) +**ExitCode**: `int` (derived) — Returns 1 if `_hasErrors` is true; returns 0 otherwise. -Sets `_hasErrors = true`, writes `message` to `Console.Error` in red (unless `Silent`), -and to `_logWriter` (if open). +#### Key Methods -### Dispose() +**Create**: Factory method that parses arguments and returns a fully initialized `Context`. -Disposes `_logWriter` and sets it to `null`. +- *Parameters*: `string[] args` — raw command-line argument array. +- *Returns*: `Context` — a new instance with all flags set. +- *Preconditions*: `args` is not null. +- *Postconditions*: All flag properties reflect the parsed argument state; the log file is open + if `--log` was supplied. -## Interactions +Delegates to the private `ArgumentParser` helper to parse flags, then opens the log file by +calling `OpenLogFile` if `--log` was present. Throws `ArgumentException` for unknown or +malformed arguments; throws `InvalidOperationException` if the log file cannot be opened. -`Context` has no dependencies on other tool units. It uses only .NET base class library types -(`Console`, `StreamWriter`). +**WriteLine**: Writes a message to standard output and to the log file. + +- *Parameters*: `string message` — the message to write. +- *Returns*: `void`. +- *Preconditions*: None. +- *Postconditions*: Message is on stdout (unless `Silent`) and in the log file (if open). + +**WriteError**: Writes an error message, sets the error state, and records to the log file. + +- *Parameters*: `string message` — the error message. +- *Returns*: `void`. +- *Preconditions*: None. +- *Postconditions*: `_hasErrors` is true; message is on stderr in red (unless `Silent`) and in + the log file (if open). + +**Dispose**: Disposes the log file writer. + +- *Parameters*: None. +- *Returns*: `void`. +- *Preconditions*: None. +- *Postconditions*: `_logWriter` is disposed and set to null; any buffered log content is + flushed. + +#### Error Handling + +`Create` throws `ArgumentException` ("Unsupported argument '{arg}'") for any unrecognized flag +or missing required value. It throws `InvalidOperationException` +("Failed to open log file '{path}': {detail}") when the `--log` file cannot be opened. Both +exceptions propagate to `Program.Main`. + +`WriteLine` and `WriteError` do not throw; they write to whichever output channels are +available. + +`Dispose` does not throw; any disposal errors are silently ignored. + +#### Dependencies + +- **.NET BCL** — `Console`, `StreamWriter`, and `Path` are the only dependencies. No other + tool units are used. + +#### Callers + +- **Program** — creates `Context` via `Context.Create` and calls `WriteLine` and `WriteError`. +- **Validation** — receives `Context` from `Program` and calls `WriteLine` and `WriteError`. diff --git a/docs/design/template-dot-net-tool/program.md b/docs/design/template-dot-net-tool/program.md index 89cc980..a659d79 100644 --- a/docs/design/template-dot-net-tool/program.md +++ b/docs/design/template-dot-net-tool/program.md @@ -1,74 +1,76 @@ -# Program +## Program -The `Program` class is the main entry point for the Template DotNet Tool. It creates a `Context` -from command-line arguments, dispatches to the appropriate logic based on the flags, and returns -the exit code. +### Purpose -## Overview +`Program` is the entry point and execution orchestrator for the Template DotNet Tool. Its single +responsibility is to create a `Context` from the command-line arguments, dispatch to the +appropriate handler based on the parsed flags, and return the exit code. -`Program` owns the top-level execution flow. It delegates all argument interpretation to `Context` -and all validation logic to `Validation`. Its own responsibility is limited to reading the flags -that `Context` exposes and calling the correct handler. +### Data Model -## Data Model +**Version**: `string` (static property) — The tool version read from +`AssemblyInformationalVersionAttribute` on every access, falling back to `AssemblyVersion`, then +`"0.0.0"`. No caching is applied; callers that need the value more than once should store it +locally. -`Program` holds no instance state. Its single static property is: +### Key Methods -| Field | Type | Description | -|-----------|----------|----------------------------------------------------------------| -| `Version` | `string` | The tool version from `AssemblyInformationalVersionAttribute`. | +**Main**: Entry point for the tool process. -## Methods +- *Parameters*: `string[] args` — command-line arguments from the host environment. +- *Returns*: `int` — exit code; 0 for success, 1 for expected errors. +- *Preconditions*: None. +- *Postconditions*: Exit code reflects whether any errors were reported during execution. -### Main(string[] args) +Creates a `Context` using `Context.Create(args)`, calls `Run(context)`, and returns +`context.ExitCode`. Catches `ArgumentException` and `InvalidOperationException` — writes +`"Error: {message}"` to stderr and returns 1. Catches any other `Exception` — writes +`"Unexpected error: {message}"` to stderr and re-throws so the runtime can record it. -Entry point. Creates a `Context`, calls `Run`, and returns `context.ExitCode`. +**Run**: Dispatches execution based on parsed flags. -**Error handling:** catches `ArgumentException` and `InvalidOperationException` — writes -`"Error: {message}"` to stderr and returns exit code 1. Catches unexpected `Exception` — -writes `"Unexpected error: {message}"` to stderr and re-throws. +- *Parameters*: `Context context` — the parsed context. +- *Returns*: `void`. +- *Preconditions*: `context` is not null. +- *Postconditions*: Exactly one handler has been called. -**Returns:** `int` — 0 for success, non-zero for failure. +Inspects flags in priority order: (1) if `context.Version` is true, calls +`context.WriteLine(Version)` and returns; (2) calls `PrintBanner`; (3) if `context.Help` is +true, calls `PrintHelp` and returns; (4) if `context.Validate` is true, calls +`Validation.Run(context)`; (5) otherwise calls `RunToolLogic(context)`. -### Run(Context context) +**PrintBanner**: Writes the tool name, version, and copyright line to `context`. -Inspects the flags on `context` and dispatches: +- *Parameters*: `Context context` — output target. +- *Returns*: `void`. -- `Version` flag → prints `Version` string only, then returns (no banner). -- Otherwise: calls `PrintBanner`, then: - - `Help` flag → calls `PrintHelp`, then returns. - - `Validate` flag → calls `Validation.Run(context)`. - - No flags → calls `RunToolLogic`. +**PrintHelp**: Writes the usage synopsis and options table to `context`. -### PrintBanner(Context context) +- *Parameters*: `Context context` — output target. +- *Returns*: `void`. -Writes the tool name, version, and copyright line to `context`. - -### PrintHelp(Context context) - -Writes the usage/options table to `context`. - -### RunToolLogic(Context context) - -Placeholder for main tool logic. Currently writes a demo message: - -```text -Template DotNet Tool - Demo Functionality -This is a template project demonstrating best practices. +**RunToolLogic**: Placeholder for main tool logic; writes a demo message to `context`. +Downstream projects replace this method body with actual tool behavior. -Replace this with your actual tool implementation. -``` +- *Parameters*: `Context context` — output target. +- *Returns*: `void`. -Downstream projects replace this method body with actual tool behavior. +### Error Handling -### Version (property) +`Main` handles errors at two levels. Expected errors (`ArgumentException` and +`InvalidOperationException`) are written to stderr as `"Error: {message}"` and cause exit +code 1 without a stack trace. Unexpected errors (`Exception`) are written to stderr as +`"Unexpected error: {message}"` and re-thrown so the runtime can record them in event logs. +`Run`, `PrintBanner`, `PrintHelp`, and `RunToolLogic` do not catch exceptions; all errors +propagate to `Main`. -Reads `AssemblyInformationalVersionAttribute` from the executing assembly, falling back to -`AssemblyVersion`, then `"0.0.0"`. +### Dependencies -## Interactions +- **Context** — `Program` reads parsed flags from `Context` and calls `Context.WriteLine` and + `Context.WriteError` for all output. +- **Validation** — `Program.Run` calls `Validation.Run(context)` when the `--validate` flag is + set. -The `Program` unit uses the following dependencies: +### Callers -- **`Context`**: Reads flags; calls `WriteLine`/`WriteError`. -- **`Validation`**: Calls `Validation.Run` when validate flag is set. +N/A - entry point, called by the host environment. diff --git a/docs/design/template-dot-net-tool/self-test.md b/docs/design/template-dot-net-tool/self-test.md new file mode 100644 index 0000000..792da61 --- /dev/null +++ b/docs/design/template-dot-net-tool/self-test.md @@ -0,0 +1,48 @@ +## SelfTest + +### Overview + +The `SelfTest` subsystem provides the self-validation framework for the Template DotNet Tool. +It is invoked when the user passes `--validate` on the command line. The subsystem runs a +built-in suite of tests that exercise the tool's own capabilities, prints a pass/fail summary, +and optionally writes the results to a TRX or JUnit XML file for CI/CD integration. The +`SelfTest` subsystem contains one unit: `Validation`. + +### Interfaces + +**Validation.Run**: Runs all self-validation tests, prints a summary, and optionally writes a +results file. + +- *Type*: In-process .NET static method. +- *Role*: Provider. +- *Contract*: Accepts a `Context` argument. Prints a Markdown-formatted heading (depth + controlled by `context.HeadingDepth`) and a table of environment metadata, executes each test + runner (`RunVersionTest`, `RunHelpTest`), prints aggregate totals (`Total Tests:`, `Passed:`, + `Failed:`), and writes a results file if `context.ResultsFile` is set. Calls + `context.WriteError` for each failed test and for unsupported results file extensions, + causing `context.ExitCode` to return 1. +- *Constraints*: Throws `ArgumentNullException` if `context` is null. Each test runner wraps + its execution in a broad `catch (Exception)` handler so that one test failure does not prevent + remaining tests from running. + +### Design + +The `SelfTest` subsystem contains only the `Validation` unit. When `Program.Run` detects the +`--validate` flag, it calls `Validation.Run(context)`. The flow within `Validation.Run` is: + +1. `PrintValidationHeader` writes a Markdown heading and a table containing tool version, + machine name, OS description, .NET runtime description, and timestamp. +2. A `DemaConsulting.TestResults.TestResults` object is constructed to accumulate results. +3. Each test runner (`RunVersionTest`, `RunHelpTest`) creates a `TemporaryDirectory`, constructs + a log file path via `PathHelpers.SafePathCombine`, invokes `Program.Run` with controlled + arguments (capturing output to the log file via `--log`), reads the log, and asserts the + expected content is present. Pass or fail is recorded; any exception is caught and recorded + via `HandleTestException` so execution continues with the next test. +4. Totals are printed; `WriteError` is used for the failed count if any tests failed. +5. If `context.ResultsFile` is set, `WriteResultsFile` serializes the results. An unsupported + file extension causes `WriteError` to be called and no file is written. + +The `TemporaryDirectory` nested class manages temporary directory creation and deletion. Its +constructor wraps `IOException`, `UnauthorizedAccessException`, and `ArgumentException` in +`InvalidOperationException`. Its `Dispose` method attempts best-effort deletion; `IOException` +and `UnauthorizedAccessException` during cleanup are silently ignored. diff --git a/docs/design/template-dot-net-tool/self-test/self-test.md b/docs/design/template-dot-net-tool/self-test/self-test.md deleted file mode 100644 index a177ddb..0000000 --- a/docs/design/template-dot-net-tool/self-test/self-test.md +++ /dev/null @@ -1,49 +0,0 @@ -# SelfTest Subsystem - -The `SelfTest` subsystem provides the self-validation framework for the Template DotNet Tool. -It runs a built-in suite of tests to demonstrate the tool is functioning correctly in the -deployment environment. - -## Overview - -The `SelfTest` subsystem is invoked when the user passes `--validate` on the command line. -It exercises the tool's own capabilities and reports a pass/fail summary. It can also write -test results to a file in TRX or JUnit XML format for integration with CI/CD pipelines. - -## Units - -The `SelfTest` subsystem contains the following software unit: - -| Unit | File | Responsibility | -|--------------|--------------------------|----------------------------------------------------| -| `Validation` | `SelfTest/Validation.cs` | Orchestrating and executing self-validation tests. | - -## Interfaces - -The `SelfTest` subsystem exposes the following outbound interface to the rest of the tool: - -- **`Validation.Run`**: Runs all self-validation tests, prints a summary, and writes results. - -## Interactions - -The `SelfTest` subsystem uses the following dependencies: - -- **`Context`**: Output channel for header lines, test summaries, and errors. -- **`Program`**: `Program.Run` is called internally to exercise the tool. -- **`PathHelpers`**: `SafePathCombine` for constructing log file paths in tests. -- **`DemaConsulting.TestResults.IO`**: TrxSerializer and JUnitSerializer provide TRX and JUnit XML - serialization for results output. -- **`DemaConsulting.TestResults`**: `TestResults`, `TestResult`, and `TestOutcome` data-model types - used for accumulating and representing self-validation results. - -## Error Handling - -`Validation.Run` handles errors in two ways: - -- **Unsupported results file extension**: When `context.ResultsFile` has an extension other than - `.trx` or `.xml`, `WriteResultsFile` calls `context.WriteError` with a descriptive message - (e.g., `"Error: Unsupported results file format '.json'. Use .trx or .xml extension."`) and - returns without writing a file. This causes `context.ExitCode` to return 1. -- **Test runner exceptions**: Each test runner (`RunVersionTest`, `RunHelpTest`) wraps its - execution in a broad `catch (Exception)` handler. Any unexpected exception is recorded as a - test failure via `HandleTestException`, allowing remaining tests to continue executing. diff --git a/docs/design/template-dot-net-tool/self-test/validation.md b/docs/design/template-dot-net-tool/self-test/validation.md index c81b379..0ab861c 100644 --- a/docs/design/template-dot-net-tool/self-test/validation.md +++ b/docs/design/template-dot-net-tool/self-test/validation.md @@ -1,89 +1,111 @@ -# Validation +### Validation -The `Validation` class provides the self-validation framework for the Template DotNet Tool. -It runs a suite of internal tests that demonstrate the tool is functioning correctly in the -deployment environment. +#### Purpose -## Overview +`Validation` orchestrates the self-validation test suite. Its single responsibility is to run a +fixed set of internal tests that exercise the tool's own functionality, print a summary to the +provided `Context`, and optionally serialize the results to a file. It does not define the +tool's requirements; it verifies that the tool behaves correctly in the deployment environment. -`Validation.Run` prints a header, executes each test, accumulates results into a -`DemaConsulting.TestResults.TestResults` object, prints aggregate totals, and optionally writes -a results file in TRX or JUnit XML format. +#### Data Model -## Data Model +`Validation` holds no instance state. The class is `internal static`; all state is local to +`Run` and the private test methods. -`Validation` holds no instance state. All state is local to `Run` and the private test methods. +#### Key Methods -## Methods +**Run**: Entry point for the self-validation suite. -### Run(Context context) +- *Parameters*: `Context context` — the output channel and results configuration. +- *Returns*: `void`. +- *Preconditions*: `context` is not null. +- *Postconditions*: A summary has been printed; if `context.ResultsFile` was set, a results + file has been written or an error has been recorded for an unsupported extension. -Orchestrates the validation sequence: +Calls `PrintValidationHeader`, constructs a `TestResults` object named +`"Template DotNet Tool Self-Validation"`, calls `RunVersionTest` and `RunHelpTest`, prints +totals (using `WriteError` if any tests failed), and calls `WriteResultsFile` if +`context.ResultsFile` is set. -1. Calls `PrintValidationHeader` to emit a Markdown heading and a table with tool and - environment metadata. The heading level is controlled by `context.HeadingDepth` - (default `1`, producing a `#` heading; `--depth 2` produces `##`, etc.). -2. Constructs a `TestResults` object named `"Template DotNet Tool Self-Validation"`. -3. Calls each test runner (`RunVersionTest`, `RunHelpTest`). -4. Prints aggregate totals: `Total Tests:`, `Passed:`, and `Failed:`. - If `failedTests > 0`, the failed count is printed via `WriteError`, which sets - `context._hasErrors = true` and causes `context.ExitCode` to return `1`. -5. Calls `WriteResultsFile` if `context.ResultsFile` is set. +**RunVersionTest**: Verifies that `--version` produces a version string. -### RunVersionTest / RunHelpTest +- *Parameters*: `Context context`, `DemaConsulting.TestResults.TestResults testResults`. +- *Returns*: `void`. -Each test method: +Creates a `TemporaryDirectory`, constructs a log path with `PathHelpers.SafePathCombine`, +invokes `Program.Run` with `["--silent", "--log", logFile, "--version"]`, reads the log, and +asserts the content matches a semver-like regex (`\b\d+\.\d+\.\d+`). Records pass or fail. +Any exception is caught by a broad `catch (Exception)` and recorded via `HandleTestException`. -1. Creates a temporary directory via `TemporaryDirectory`. -2. Constructs a log-file path using `PathHelpers.SafePathCombine`. -3. Invokes `Program.Run` with the relevant arguments and captures the output log. -4. Validates the output against expected content. -5. Records pass or fail in the shared `TestResults`. -6. Prints its own pass/fail line to `context` upon completion. +**RunHelpTest**: Verifies that `--help` produces usage text. -Any exception thrown during steps 1–5 is caught by a broad `catch (Exception)` handler, which -records the failure via `HandleTestException` and continues. -This ensures the test framework remains robust and all remaining tests execute even if one test -fails unexpectedly. +- *Parameters*: `Context context`, `DemaConsulting.TestResults.TestResults testResults`. +- *Returns*: `void`. -### WriteResultsFile(Context context, TestResults testResults) +Creates a `TemporaryDirectory`, constructs a log path with `PathHelpers.SafePathCombine`, +invokes `Program.Run` with `["--silent", "--log", logFile, "--help"]`, reads the log, and +asserts the content contains both `"Usage:"` and `"Options:"`. Records pass or fail. Any +exception is caught by a broad `catch (Exception)` and recorded via `HandleTestException`. -Writes `testResults` to `context.ResultsFile`. The format is determined by the file extension: -`.trx` for TRX (MSTest), `.xml` for JUnit. Any other extension causes an error message to be -written to `context` and the method returns without creating a file. +**WriteResultsFile**: Serializes `testResults` to `context.ResultsFile`. -Any exception thrown during file write is caught by a broad `catch (Exception)` handler and reported to `context` via `WriteError`. +- *Parameters*: `Context context`, `DemaConsulting.TestResults.TestResults testResults`. +- *Returns*: `void`. -### Private Helpers +Determines the format from the file extension: `.trx` uses `TrxSerializer.Serialize`; `.xml` +uses `JUnitSerializer.Serialize`. Any other extension calls `context.WriteError` with a +descriptive message and returns without writing. File-write exceptions are caught by a broad +`catch (Exception)` and reported via `context.WriteError`. -#### CreateTestResult(string testName) +**CreateTestResult**: Creates a `TestResult` pre-populated with class and code-base metadata. -Creates a `TestResult` object pre-populated with the class name `"Validation"` and code base `"TemplateDotNetTool"`. +- *Parameters*: `string testName`. +- *Returns*: `DemaConsulting.TestResults.TestResult` with `ClassName = "Validation"` and + `CodeBase = "TemplateDotNetTool"`. -#### FinalizeTestResult(TestResult test, DateTime startTime, TestResults testResults) +**FinalizeTestResult**: Sets elapsed duration and appends the result to the collection. -Sets `test.Duration` to the elapsed time since `startTime` and appends `test` to `testResults.Results`. +- *Parameters*: `DemaConsulting.TestResults.TestResult test`, `DateTime startTime`, + `DemaConsulting.TestResults.TestResults testResults`. +- *Returns*: `void`. -#### HandleTestException(TestResult test, Context context, string testName, Exception ex) +**HandleTestException**: Records a test failure from a caught exception. -Sets `test.Outcome` to `Failed`, records the exception message as `test.ErrorMessage`, -and writes a failure line to `context` via `WriteError`. +- *Parameters*: `DemaConsulting.TestResults.TestResult test`, `Context context`, + `string testName`, `Exception ex`. +- *Returns*: `void`. -#### TemporaryDirectory (nested class) +Sets `test.Outcome` to `Failed`, records `ex.Message` as `test.ErrorMessage`, and calls +`context.WriteError` with a failure message. -Manages a temporary directory for test execution. Implements `IDisposable`. The constructor -calls `Directory.CreateDirectory` and wraps `IOException`, `UnauthorizedAccessException`, -and `ArgumentException` in `InvalidOperationException`. `Dispose` attempts a best-effort -deletion of the directory tree; `IOException` and `UnauthorizedAccessException` during -cleanup are silently ignored. +**TemporaryDirectory** (nested class): Manages a temporary directory for test execution. +Implements `IDisposable`. The constructor calls `Directory.CreateDirectory` and wraps +`IOException`, `UnauthorizedAccessException`, and `ArgumentException` in +`InvalidOperationException`. `Dispose` attempts best-effort deletion of the directory tree; +`IOException` and `UnauthorizedAccessException` during cleanup are silently ignored. -## Interactions +#### Error Handling -The `Validation` subsystem uses the following dependencies: +`Run` throws `ArgumentNullException` if `context` is null. Each test runner wraps its body in +a broad `catch (Exception)` to ensure test-suite robustness: any unexpected exception is +recorded as a test failure via `HandleTestException` and execution continues with the next test. +`WriteResultsFile` catches file-write exceptions and reports them via `context.WriteError`. -- **`Context`**: Output channel for header and summary lines. -- **`Program`**: `Program.Run` called to exercise the tool. -- **`PathHelpers`**: `SafePathCombine` for temp-dir file paths. -- **`TrxSerializer`**: Serializes TestResults to TRX format. -- **`JUnitSerializer`**: Serializes TestResults to JUnit XML format. -- **`DemaConsulting.TestResults`**: TestResults/TestResult/TestOutcome for test state. +An unsupported `context.ResultsFile` extension is treated as a user error: `WriteError` is +called with a descriptive message +(e.g., `"Error: Unsupported results file format '.json'. Use .trx or .xml extension."`) and +the method returns without writing a file, causing `context.ExitCode` to return 1. + +#### Dependencies + +- **Context** — output channel for header lines, test result lines, and summary totals. +- **Program** — `Program.Run` is called within each test runner to exercise the tool. +- **PathHelpers** — `SafePathCombine` constructs log file paths inside temporary directories. +- **DemaConsulting.TestResults** — `TestResults`, `TestResult`, and `TestOutcome` types for + accumulating and representing self-validation results. +- **DemaConsulting.TestResults.IO** — `TrxSerializer` and `JUnitSerializer` for serializing + results to `.trx` and `.xml` files. + +#### Callers + +- **Program** — calls `Validation.Run(context)` when the `--validate` flag is set. diff --git a/docs/design/template-dot-net-tool/template-dot-net-tool.md b/docs/design/template-dot-net-tool/template-dot-net-tool.md deleted file mode 100644 index afd55af..0000000 --- a/docs/design/template-dot-net-tool/template-dot-net-tool.md +++ /dev/null @@ -1,199 +0,0 @@ -# System Design - -This document describes the system-level design of the Template DotNet Tool, -including the overall architecture, external interfaces, and system-wide design -decisions that affect all subsystems. - -## System Architecture - -The Template DotNet Tool is a command-line application built on the .NET -platform that demonstrates DEMA Consulting best practices for .NET tool -development. The system consists of three primary subsystems working together -to provide a robust, testable, and maintainable command-line tool: - -### Major Components - -- **CLI Subsystem** — Command-line argument parsing and user interface - management, providing standardized input processing and output formatting -- **SelfTest Subsystem** — Automated validation framework enabling the tool to - verify its own functionality and report test results -- **Utilities Subsystem** — Shared utility functions providing common - functionality across all subsystems - -### Component Interactions - -The Program unit acts as the system orchestrator, coordinating interactions -between subsystems: - -1. **Initialization Phase** — Program creates Context from CLI subsystem to - parse command-line arguments and configure system behavior -2. **Execution Phase** — Program delegates to appropriate subsystem based on - parsed arguments (help display, version query, self-validation, or main tool - logic) -3. **Output Phase** — All subsystems use Context for consistent output - formatting and logging - -## External Interfaces - -### Command-Line Interface - -The system accepts command-line arguments following standard conventions: - -- **Version Query**: `-v`, `--version` — Display version information -- **Help Display**: `-?`, `-h`, `--help` — Show usage information -- **Silent Mode**: `--silent` — Suppress console output -- **Self-Validation**: `--validate` — Run internal test suite -- **Results Output**: `--results ` — Write test results to TRX or JUnit XML file -- **Results Output (alias)**: `--result ` — Write test results to file (legacy alias for `--results`) -- **Logging**: `--log ` — Write all output to log file -- **Heading Depth**: `--depth ` — Set the heading depth for markdown output - -### File System Interface - -The system interacts with the file system for: - -- **Log File Output** — Optional logging to user-specified file path -- **Test Results Output** — Optional test results in TRX or JUnit XML format for CI/CD integration -- **Path Operations** — Safe path combination and validation through Utilities subsystem - -### Standard I/O Interface - -The system uses standard console I/O streams: - -- **Standard Output** — Normal program output and information display -- **Standard Error** — Error messages and exception information -- **Color Output** — Red color coding for error messages when supported - -## Data Flow - -### Input Processing Flow - -1. **Command-line arguments** → CLI subsystem parses into Context object -2. **Context object** → Program uses to determine execution path -3. **User input validation** → Context throws typed exceptions for invalid arguments - -### Output Processing Flow - -1. **Program logic** → Generates output messages and status information -2. **Context formatting** → Applies consistent formatting and color coding -3. **Multi-target output** → Simultaneously writes to console and log file (if enabled) - -### Error Handling Flow - -1. **Exception generation** → Units throw typed exceptions with context information -2. **Exception catching** → Program catches and categorizes exceptions -3. **Error reporting** → Context formats and displays appropriate error messages -4. **Exit code setting** → Program returns appropriate exit codes for automation - -## System-Wide Design Constraints - -### .NET Runtime Requirements - -The system targets multiple .NET runtime versions to maximize compatibility: - -- **.NET 8.0** — Long-term support baseline -- **.NET 9.0** — Current standard runtime -- **.NET 10.0** — Latest runtime for forward compatibility - -### Dependency Injection Architecture - -All subsystems follow dependency injection patterns: - -- **Constructor injection** — All external dependencies injected via constructors -- **Interface abstraction** — Dependencies defined through interfaces where appropriate -- **Testability focus** — Design enables comprehensive unit testing with mocked dependencies - -### Error Handling Strategy - -System-wide error handling follows consistent patterns: - -- **Typed exceptions** — Use specific exception types (ArgumentException, InvalidOperationException) -- **Context preservation** — Exception messages include sufficient troubleshooting information -- **Audit logging** — When logging is enabled (for example, via `--log`), all errors are logged for - compliance and debugging purposes - -### Thread Safety - -The system operates as a single-threaded console application: - -- **No shared state** — Units avoid static mutable state -- **Immutable design** — Configuration objects use init-only properties -- **Resource cleanup** — IDisposable pattern for file handles and resources - -## Integration Patterns - -### Test Integration - -The system integrates with external testing infrastructure: - -- **TRX output format** — Compatible with MSTest and ReqStream requirements traceability -- **Exit code conventions** — Returns 0 for success, non-zero for failures -- **Structured logging** — Consistent output format for automated processing - -### CI/CD Integration - -The system supports automated build and deployment: - -- **Multi-framework compilation** — Single codebase targets multiple .NET versions -- **Self-validation capability** — Built-in testing for deployment verification -- **Logging integration** — File-based logging for automated analysis - -### Requirements Traceability - -The system design supports compliance and audit requirements: - -- **ReqStream integration** — Test output format compatible with requirements management -- **Documentation generation** — Design documents provide implementation traceability -- **Review evidence** — Structured design enables formal code review processes - -## Performance Characteristics - -### Startup Performance - -The system prioritizes fast startup for command-line usage: - -- **Minimal initialization** — Only essential components loaded at startup -- **Lazy evaluation** — Resource-intensive operations deferred until needed -- **Efficient argument parsing** — Linear-time parsing with early validation - -### Memory Usage - -The system maintains low memory footprint: - -- **Streaming I/O** — File operations use streaming for large outputs -- **Bounded allocation** — No unbounded collection growth -- **Prompt disposal** — Resources released immediately after use - -### Scalability Constraints - -As a command-line tool, the system has defined scalability boundaries: - -- **Single-user operation** — Designed for individual command execution -- **Stateless execution** — No persistent state between invocations -- **Process isolation** — Each invocation runs in separate process - -## Security Considerations - -### Input Validation - -All external inputs receive validation: - -- **Argument validation** — Command-line arguments checked for validity and safety -- **Path validation** — File paths validated to prevent directory traversal -- **Exception handling** — Input errors reported without exposing system internals - -### File System Access - -File system operations follow security best practices: - -- **Controlled access** — Only user-specified paths accessed -- **Permission handling** — Graceful degradation for insufficient permissions -- **Path sanitization** — All file paths validated and sanitized - -### Information Disclosure - -The system prevents unintended information disclosure: - -- **Error message filtering** — Exception details limited to necessary information -- **Log content control** — Sensitive information excluded from log files -- **Version information** — Only public version information exposed diff --git a/docs/design/template-dot-net-tool/utilities.md b/docs/design/template-dot-net-tool/utilities.md new file mode 100644 index 0000000..54c7579 --- /dev/null +++ b/docs/design/template-dot-net-tool/utilities.md @@ -0,0 +1,34 @@ +## Utilities + +### Overview + +The `Utilities` subsystem provides shared utility functions for the Template DotNet Tool. It +supplies reusable, independently testable helpers consumed by other subsystems. Its primary +responsibility is safe file-path manipulation, protecting callers from path-traversal +vulnerabilities when constructing paths from caller-supplied inputs. The `Utilities` subsystem +contains one unit: `PathHelpers`. + +### Interfaces + +**PathHelpers.SafePathCombine**: Combines a base path and a relative path, rejecting any result +that escapes the base directory. + +- *Type*: In-process .NET static method. +- *Role*: Provider. +- *Contract*: Accepts `string basePath` and `string relativePath`. Returns the combined path + produced by `Path.Combine(basePath, relativePath)` after verifying that the resolved result + remains within `basePath`. Preserves the caller's relative/absolute style in the return value. +- *Constraints*: Throws `ArgumentNullException` for null inputs; throws `ArgumentException` + when the combined path escapes the base directory; may propagate `NotSupportedException` or + `PathTooLongException` from underlying BCL path operations. + +### Design + +The `Utilities` subsystem contains only the `PathHelpers` unit. It has no dependencies on other +tool units or subsystems; it uses only .NET BCL types (`Path`, `ArgumentNullException`). + +`PathHelpers.SafePathCombine` is a pure utility method: it performs no file-system I/O, holds +no state, and throws immediately on invalid input. All calls to `SafePathCombine` in the +codebase originate from the `SelfTest` subsystem (`Validation`), which uses it to construct +log and result file paths inside temporary directories created during self-validation test +execution. diff --git a/docs/design/template-dot-net-tool/utilities/path-helpers.md b/docs/design/template-dot-net-tool/utilities/path-helpers.md index 228a4a9..44e248a 100644 --- a/docs/design/template-dot-net-tool/utilities/path-helpers.md +++ b/docs/design/template-dot-net-tool/utilities/path-helpers.md @@ -1,53 +1,54 @@ -# PathHelpers Design - -## Overview - -`PathHelpers` is a static utility class that provides a safe path-combination method. It -protects callers against path-traversal attacks by verifying the resolved combined path stays -within the base directory. Note that `Path.GetFullPath` normalizes `.`/`..` segments but does -not resolve symlinks or reparse points, so this check guards against string-level traversal -only. - -## Class Structure - -### SafePathCombine Method - -```csharp -internal static string SafePathCombine(string basePath, string relativePath) -``` - -Combines `basePath` and `relativePath` safely, ensuring the resulting path remains within -the base directory. - -**Validation steps:** - -1. Reject null inputs via `ArgumentNullException.ThrowIfNull`. -2. Combine the paths with `Path.Combine` to produce the candidate path (preserving the - caller's relative/absolute style). -3. Resolve both `basePath` and the candidate to absolute form with `Path.GetFullPath`. -4. Compute `Path.GetRelativePath(absoluteBase, absoluteCombined)` and reject the input if - the result is exactly `".."`, starts with `".."` followed by `Path.DirectorySeparatorChar` - or `Path.AltDirectorySeparatorChar`, or is itself rooted (absolute), which would indicate - the combined path escapes the base directory. -5. Return `combinedPath` (the pre-resolved combined path), preserving the caller's - relative/absolute style. The returned value is the candidate produced by `Path.Combine`, - not the resolved absolute path. - -## Design Decisions - -- **`Path.GetRelativePath` for containment check**: Using `GetRelativePath` to verify - containment handles root paths (e.g. `/`, `C:\`), platform case-sensitivity, and - directory-separator normalization natively. The containment test should treat `..` as an - escaping segment only when it is the entire relative result or is followed by a directory - separator, avoiding false positives for valid in-base names such as `..data`. -- **Post-combine canonical-path check**: Resolving paths after combining handles all traversal - patterns — `../`, embedded `/../`, absolute-path overrides, and platform edge cases — - without fragile pre-combine string inspection of `relativePath`. -- **ArgumentException on invalid input**: Callers receive a specific `ArgumentException` - identifying `relativePath` as the problematic parameter, making debugging straightforward. -- **No logging or error accumulation**: `SafePathCombine` is a pure utility method that throws - on invalid input; it does not interact with the `Context` or any output mechanism. -- **BCL pass-through exceptions**: `NotSupportedException` (unsupported path format) and - `PathTooLongException` (path exceeds system limit) may propagate from underlying BCL path - operations (`Path.Combine`, `Path.GetFullPath`). Callers that supply paths from external - sources should handle these exceptions alongside `ArgumentException`. +### PathHelpers + +#### Purpose + +`PathHelpers` is a static utility class that provides a safe path-combination method. Its +single responsibility is to combine two path segments while verifying that the result does not +escape the base directory, protecting callers from string-level path-traversal attacks. + +#### Data Model + +`PathHelpers` holds no instance state. The class is `internal static` with no fields or +properties. + +#### Key Methods + +**SafePathCombine**: Safely combines a base path and a relative path. + +- *Parameters*: `string basePath` — the base directory path; `string relativePath` — the + relative path to append. +- *Returns*: `string` — the pre-resolved combined path (preserves the caller's + relative/absolute style). +- *Preconditions*: Both `basePath` and `relativePath` are non-null. +- *Postconditions*: The returned path, when resolved to absolute form, is contained within + `basePath`. + +Validation steps: (1) reject null inputs via `ArgumentNullException.ThrowIfNull`; (2) call +`Path.Combine(basePath, relativePath)` to produce the candidate path; (3) resolve both +`basePath` and the candidate to absolute form with `Path.GetFullPath`; (4) compute +`Path.GetRelativePath(absoluteBase, absoluteCombined)` and reject if the result equals `".."`, +starts with `".."` followed by a directory separator character, or is itself rooted (absolute); +(5) return the pre-resolved `combinedPath` from step 2. + +The containment check uses `Path.GetRelativePath` rather than string inspection to handle root +paths, platform case-sensitivity, and directory-separator normalization natively. The `".."` +check treats a double-dot segment as escaping only when it is the entire relative result or is +followed by a directory separator, avoiding false positives for valid names such as `"..data"`. + +#### Error Handling + +`SafePathCombine` throws `ArgumentNullException` for null inputs. It throws `ArgumentException` +(`"Invalid path component: {relativePath}"`) when the combined path escapes the base directory. +`NotSupportedException` and `PathTooLongException` may propagate from underlying BCL path +operations (`Path.Combine`, `Path.GetFullPath`). No logging or error accumulation is performed; +callers receive exceptions directly. + +#### Dependencies + +- **.NET BCL** — `Path`, `ArgumentNullException`, and related types are the only dependencies. + No other tool units or subsystems are used. + +#### Callers + +- **Validation** — calls `PathHelpers.SafePathCombine` to construct log file paths and + temporary directory paths during self-validation test execution. diff --git a/docs/design/template-dot-net-tool/utilities/utilities.md b/docs/design/template-dot-net-tool/utilities/utilities.md deleted file mode 100644 index 64a0ac5..0000000 --- a/docs/design/template-dot-net-tool/utilities/utilities.md +++ /dev/null @@ -1,36 +0,0 @@ -# Utilities Subsystem - -The `Utilities` subsystem provides shared utility functions for the Template DotNet Tool. -It supplies reusable, independently testable helpers that are consumed by other subsystems. - -## Overview - -The `Utilities` subsystem contains general-purpose helpers that do not belong to any -specific feature subsystem. Its primary responsibility is safe file-path manipulation, -protecting callers from path-traversal vulnerabilities when constructing paths from -external inputs. - -## Units - -The `Utilities` subsystem contains the following software unit: - -| Unit | File | Responsibility | -|---------------|----------------------------|---------------------------------------------| -| `PathHelpers` | `Utilities/PathHelpers.cs` | Safe path combination and traversal checks. | - -## Interfaces - -The `Utilities` subsystem exposes the following outbound interface to the rest of the tool: - -- **`PathHelpers.SafePathCombine`**: Combines two path segments, rejecting traversal sequences - and absolute path overrides. - -`SafePathCombine` throws `ArgumentException` when the combined path escapes the base -directory, and `ArgumentNullException` for null inputs. `NotSupportedException` -(unsupported path format) and `PathTooLongException` (path exceeds system limit) may -propagate from the underlying BCL path operations. - -## Interactions - -`PathHelpers` has no dependencies on other tool units or subsystems. It uses only .NET base -class library types (`Path`, `ArgumentNullException`). diff --git a/docs/reqstream/ots/reviewmark.yaml b/docs/reqstream/ots/reviewmark.yaml index e67d299..72980ed 100644 --- a/docs/reqstream/ots/reviewmark.yaml +++ b/docs/reqstream/ots/reviewmark.yaml @@ -19,19 +19,33 @@ sections: tests: - ReviewMark_ReviewPlanGeneration - ReviewMark_ReviewReportGeneration - - id: Template-OTS-ReviewMark-Operations - title: >- - ReviewMark shall support index scanning, enforcement, elaboration, - lint, and directory override operations. + + - id: Template-OTS-ReviewMark-Lint + title: ReviewMark shall validate the review configuration file. justification: | - In addition to document generation, ReviewMark provides operational commands used - in the CI pipeline: scanning PDF evidence into an index catalogue, enforcing review - currency, elaborating review-set details for agent use, validating the configuration - file, and overriding the working directory for file resolution. + DemaConsulting.ReviewMark --lint is run in the CI lint pass to validate the + .reviewmark.yaml configuration before any build proceeds, making configuration + errors a build-breaking condition. tags: [ots] tests: - - ReviewMark_IndexScan - - ReviewMark_WorkingDirectoryOverride - - ReviewMark_Enforce - - ReviewMark_Elaborate - ReviewMark_Lint + + - id: Template-OTS-ReviewMark-Elaborate + title: ReviewMark shall elaborate review-set details for agent consumption. + justification: | + DemaConsulting.ReviewMark --elaborate is invoked by the formal-review agent + to expand review-set file lists and context, enabling automated agents to + perform targeted file reviews without manually resolving glob patterns. + tags: [ots] + tests: + - ReviewMark_Elaborate + + - id: Template-OTS-ReviewMark-Enforce + title: ReviewMark shall enforce that all review-sets have current review evidence. + justification: | + DemaConsulting.ReviewMark --enforce is run in the CI pipeline to verify that + every file in every review-set has been reviewed against current content, + making stale or missing review evidence a build-breaking condition. + tags: [ots] + tests: + - ReviewMark_Enforce diff --git a/docs/reqstream/template-dot-net-tool/template-dot-net-tool.yaml b/docs/reqstream/template-dot-net-tool.yaml similarity index 92% rename from docs/reqstream/template-dot-net-tool/template-dot-net-tool.yaml rename to docs/reqstream/template-dot-net-tool.yaml index 42860d7..7896f6b 100644 --- a/docs/reqstream/template-dot-net-tool/template-dot-net-tool.yaml +++ b/docs/reqstream/template-dot-net-tool.yaml @@ -94,16 +94,16 @@ sections: - TemplateDotNetTool_ValidateWithDepth_DepthThree_OutputsCorrectHeadingLevel - id: Template-System-ValidateFailure - title: The tool shall return a non-zero exit code when one or more self-validation tests fail. + title: The tool shall return a non-zero exit code when the results file cannot be written. justification: | - A non-zero exit code on validation failure enables CI/CD pipelines to detect - and surface qualification failures automatically, preventing the use of a - mis-behaving tool in compliance-critical workflows. + A non-zero exit code on results-write failure enables CI/CD pipelines to detect + and surface output errors automatically, preventing silent loss of validation + evidence in compliance-critical workflows. tests: - TemplateDotNetTool_ValidateWithBadExtension_ExtensionInvalid_ReturnsNonZero - id: Template-System-DefaultBehavior - title: The tool shall display a banner and run the default tool logic when invoked without any action flags. + title: The tool shall display a banner then run the default tool logic when invoked without any action flags. justification: | When a user invokes the tool without any recognized action flag, the tool shall display a banner identifying itself and run its primary logic so that the diff --git a/docs/reqstream/template-dot-net-tool/cli/cli.yaml b/docs/reqstream/template-dot-net-tool/cli.yaml similarity index 100% rename from docs/reqstream/template-dot-net-tool/cli/cli.yaml rename to docs/reqstream/template-dot-net-tool/cli.yaml diff --git a/docs/reqstream/template-dot-net-tool/platform-requirements.yaml b/docs/reqstream/template-dot-net-tool/platform-requirements.yaml index 728955e..b6ecc36 100644 --- a/docs/reqstream/template-dot-net-tool/platform-requirements.yaml +++ b/docs/reqstream/template-dot-net-tool/platform-requirements.yaml @@ -20,7 +20,7 @@ sections: - title: Platform Support requirements: - id: Template-Platform-Windows - title: The tool shall build and run on Windows platforms. + title: The tool shall execute successfully on Windows platforms. justification: | DEMA Consulting tools must support Windows as a major development platform. tests: @@ -29,7 +29,7 @@ sections: - "windows@TemplateTool_HelpDisplay" - id: Template-Platform-Linux - title: The tool shall build and run on Linux platforms. + title: The tool shall execute successfully on Linux platforms. justification: | DEMA Consulting tools must support Linux for CI/CD and containerized environments. tests: @@ -38,7 +38,7 @@ sections: - "ubuntu@TemplateTool_HelpDisplay" - id: Template-Platform-MacOS - title: The tool shall build and run on macOS platforms. + title: The tool shall execute successfully on macOS platforms. justification: | DEMA Consulting tools must support macOS for developers using Apple platforms. tests: diff --git a/docs/reqstream/template-dot-net-tool/self-test/self-test.yaml b/docs/reqstream/template-dot-net-tool/self-test.yaml similarity index 77% rename from docs/reqstream/template-dot-net-tool/self-test/self-test.yaml rename to docs/reqstream/template-dot-net-tool/self-test.yaml index 20a53ba..673bc0f 100644 --- a/docs/reqstream/template-dot-net-tool/self-test/self-test.yaml +++ b/docs/reqstream/template-dot-net-tool/self-test.yaml @@ -34,3 +34,14 @@ sections: - SelfTestSubsystem_ValidationWorkflow_WithTrxFile_GeneratesResults - SelfTestSubsystem_ValidationWorkflow_WithJUnitFile_GeneratesResults - SelfTestSubsystem_ValidationWorkflow_WithBothResultFiles_GeneratesBothResults + + - id: Template-SelfTest-InvalidExtension + title: >- + The tool shall report an error and set a non-zero exit code when an + unsupported results file extension is provided. + justification: | + Prevents silent loss of validation evidence by failing loudly when a results + file cannot be written due to an unrecognized extension. This enables CI/CD + pipelines to detect misconfigured output paths automatically. + tests: + - SelfTestSubsystem_ValidationWorkflow_WithUnsupportedExtension_EmitsErrorAndNoFile diff --git a/docs/reqstream/template-dot-net-tool/self-test/validation.yaml b/docs/reqstream/template-dot-net-tool/self-test/validation.yaml index dedb1e3..2ab68d5 100644 --- a/docs/reqstream/template-dot-net-tool/self-test/validation.yaml +++ b/docs/reqstream/template-dot-net-tool/self-test/validation.yaml @@ -36,3 +36,14 @@ sections: enabling broad integration with CI/CD pipelines that process JUnit output. tests: - Validation_Run_WithXmlResultsFile_WritesXmlFile + + - id: Template-Validation-InvalidExtension + title: >- + The Validation class shall report an error and set exit code 1 when the + results file path has an unsupported extension. + justification: | + Provides a clear failure signal when an unrecognized results file extension is + supplied, preventing silent evidence loss and enabling CI/CD pipelines to + surface output configuration errors automatically. + tests: + - Validation_Run_WithUnsupportedResultsFormat_DoesNotWriteFile diff --git a/docs/reqstream/template-dot-net-tool/utilities/utilities.yaml b/docs/reqstream/template-dot-net-tool/utilities.yaml similarity index 100% rename from docs/reqstream/template-dot-net-tool/utilities/utilities.yaml rename to docs/reqstream/template-dot-net-tool/utilities.yaml diff --git a/docs/user_guide/introduction.md b/docs/user_guide/introduction.md index b1a65ec..c5ee84a 100644 --- a/docs/user_guide/introduction.md +++ b/docs/user_guide/introduction.md @@ -191,3 +191,7 @@ templatetool --validate --results validation-results.trx ```bash templatetool --silent --log tool-output.log ``` + +## References + +N/A diff --git a/docs/verification/definition.yaml b/docs/verification/definition.yaml index b4d0664..f6a1ae2 100644 --- a/docs/verification/definition.yaml +++ b/docs/verification/definition.yaml @@ -11,14 +11,15 @@ resource-path: input-files: - docs/verification/title.txt - docs/verification/introduction.md - - docs/verification/template-dot-net-tool/template-dot-net-tool.md + - docs/verification/template-dot-net-tool.md - docs/verification/template-dot-net-tool/program.md - - docs/verification/template-dot-net-tool/cli/cli.md + - docs/verification/template-dot-net-tool/cli.md - docs/verification/template-dot-net-tool/cli/context.md - - docs/verification/template-dot-net-tool/self-test/self-test.md + - docs/verification/template-dot-net-tool/self-test.md - docs/verification/template-dot-net-tool/self-test/validation.md - - docs/verification/template-dot-net-tool/utilities/utilities.md + - docs/verification/template-dot-net-tool/utilities.md - docs/verification/template-dot-net-tool/utilities/path-helpers.md + - docs/verification/ots.md - docs/verification/ots/buildmark.md - docs/verification/ots/fileassert.md - docs/verification/ots/pandoc.md diff --git a/docs/verification/introduction.md b/docs/verification/introduction.md index bb057d4..18ebb89 100644 --- a/docs/verification/introduction.md +++ b/docs/verification/introduction.md @@ -13,24 +13,20 @@ test scenario. The document does not restate design; it explains how the design ## Scope -This document covers the verification design for the same software items described in the -*Template DotNet Tool Software Design Document*: - -- **TemplateDotNetTool** — the system as a whole -- **Program** — entry point and execution orchestrator -- **Cli** — command-line interface subsystem - - **Context** — argument parser and I/O owner -- **SelfTest** — self-validation subsystem - - **Validation** — self-validation test runner -- **Utilities** — shared utility subsystem - - **PathHelpers** — safe path combination utilities +This document covers the verification design for the following software items: -The following topics are out of scope: +**Local items:** -- Test infrastructure (xUnit framework, test helpers, Runner utility) -- Build pipeline and CI/CD configuration +- **TemplateDotNetTool** — system, subsystem, and unit verification: + - **Program** — entry point and execution orchestrator + - **Cli** subsystem + - **Context** — argument parser and I/O owner + - **SelfTest** subsystem + - **Validation** — self-validation test runner + - **Utilities** subsystem + - **PathHelpers** — safe path combination utilities -The following OTS items are also covered: +**OTS items:** - **BuildMark** — build-notes documentation tool - **FileAssert** — document assertion tool @@ -43,6 +39,14 @@ The following OTS items are also covered: - **WeasyPrint** — HTML-to-PDF conversion tool - **xUnit** — unit-testing framework +The following topics are out of scope: + +- Verification documents are not produced for the test projects themselves — they are the + means of verification, not subjects of it +- Build pipeline CI configuration is excluded +- The internal implementation of OTS software items is excluded; only integration and usage + are verified + ## Software Structure The following tree shows the software items covered by this document: @@ -70,6 +74,15 @@ OTS Items └── xUnit ``` +## Folder Layout + +The test folder structure mirrors the source subsystem breakdown: + +```text +test/ +└── DemaConsulting.TemplateDotNetTool.Tests/ — unit and integration tests +``` + ## Companion Artifact Structure In-house items have corresponding artifacts in parallel directory trees: @@ -86,3 +99,8 @@ OTS items have parallel artifacts in: - Verification: `docs/verification/ots/{ots-name}.md` (kebab-case) Review-sets: defined in `.reviewmark.yaml` + +## References + +- Template DotNet Tool Software Design Document +- Template DotNet Tool releases () diff --git a/docs/verification/ots.md b/docs/verification/ots.md new file mode 100644 index 0000000..c3ca567 --- /dev/null +++ b/docs/verification/ots.md @@ -0,0 +1,28 @@ +# OTS Software Verification + +This section provides verification evidence for all Off-The-Shelf (OTS) software items +used by the Template DotNet Tool. + +## Scope + +Each OTS item is verified by a combination of: + +- The OTS tool's own self-validation suite (where the tool supports `--validate`) +- Pipeline output assertions performed by FileAssert on documents produced by the tool + +Internal OTS tool design is out of scope; only integration and usage evidence is documented. + +## OTS Items + +The following OTS items have verification evidence in this section: + +- BuildMark (_buildmark.md_) — build-notes documentation tool +- FileAssert (_fileassert.md_) — document assertion tool +- Pandoc (_pandoc.md_) — Markdown-to-HTML conversion tool +- ReqStream (_reqstream.md_) — requirements traceability tool +- ReviewMark (_reviewmark.md_) — file review enforcement tool +- SarifMark (_sarifmark.md_) — SARIF report conversion tool +- SonarMark (_sonarmark.md_) — SonarCloud quality report tool +- VersionMark (_versionmark.md_) — tool-version documentation tool +- WeasyPrint (_weasyprint.md_) — HTML-to-PDF conversion tool +- xUnit (_xunit.md_) — unit-testing framework diff --git a/docs/verification/ots/buildmark.md b/docs/verification/ots/buildmark.md index da5e4bf..65cc1dc 100644 --- a/docs/verification/ots/buildmark.md +++ b/docs/verification/ots/buildmark.md @@ -1,15 +1,15 @@ -# BuildMark Verification +## BuildMark Verification This document provides the verification evidence for the `BuildMark` OTS software item. -## Required Functionality +### Required Functionality DemaConsulting.BuildMark queries the GitHub API to capture workflow run details and renders them as a markdown build-notes document included in the release artifacts. It runs as part of the same CI pipeline that produces the TRX test results, so a successful pipeline run is evidence that BuildMark executed without error. -## Verification Approach +### Verification Approach BuildMark is verified by two complementary layers of evidence. First, the CI pipeline runs `buildmark --validate --results artifacts/buildmark-self-validation.trx`, which exercises @@ -22,9 +22,9 @@ artifact, and FileAssert asserts the PDF exists, has content, and contains expec (`WeasyPrint_BuildNotesPdf`). A CI build failure at any step in this chain is evidence that BuildMark did not produce the required output. -## Test Scenarios +### Test Scenarios -### BuildMark_MarkdownReportGeneration +#### BuildMark_MarkdownReportGeneration **Scenario**: A CI pipeline run triggers BuildMark with live GitHub Actions metadata. @@ -33,7 +33,7 @@ in the release artifacts. **Requirement coverage**: `Template-OTS-BuildMark`. -### BuildMark_GitIntegration +#### BuildMark_GitIntegration **Scenario**: BuildMark self-validation reads version tags and commits from a mock Git history. @@ -41,7 +41,7 @@ in the release artifacts. **Requirement coverage**: `Template-OTS-BuildMark`. -### BuildMark_IssueTracking +#### BuildMark_IssueTracking **Scenario**: BuildMark self-validation processes mock GitHub issues and pull requests. @@ -49,7 +49,7 @@ in the release artifacts. **Requirement coverage**: `Template-OTS-BuildMark`. -### BuildMark_KnownIssuesReporting +#### BuildMark_KnownIssuesReporting **Scenario**: BuildMark self-validation includes open bugs in the generated report when requested. @@ -57,7 +57,7 @@ in the release artifacts. **Requirement coverage**: `Template-OTS-BuildMark`. -### BuildMark_RulesRouting +#### BuildMark_RulesRouting **Scenario**: BuildMark self-validation assigns mock items to report sections based on label and type rules. @@ -65,8 +65,3 @@ type rules. **Expected**: Exits 0 and correctly routes items to the expected sections. **Requirement coverage**: `Template-OTS-BuildMark`. - -## Requirements Coverage - -- **`Template-OTS-BuildMark`**: BuildMark_MarkdownReportGeneration, BuildMark_GitIntegration, - BuildMark_IssueTracking, BuildMark_KnownIssuesReporting, BuildMark_RulesRouting diff --git a/docs/verification/ots/fileassert.md b/docs/verification/ots/fileassert.md index 4027dde..5073c53 100644 --- a/docs/verification/ots/fileassert.md +++ b/docs/verification/ots/fileassert.md @@ -1,9 +1,9 @@ -# FileAssert Verification +## FileAssert Verification This document provides the verification evidence for the FileAssert OTS software item. Requirements for this OTS item are defined in the FileAssert OTS Software Requirements document. -## Required Functionality +### Required Functionality DemaConsulting.FileAssert validates HTML and PDF documents produced during the build, asserting that each document exists, has a non-trivial size, is structurally valid, and contains expected content. @@ -11,7 +11,7 @@ It provides OTS evidence for Pandoc and WeasyPrint and independently confirms fi functioning. Self-validation proves the tool itself is operational before ReqStream consumes the results. -## Verification Approach +### Verification Approach FileAssert is verified by two complementary layers of evidence. First, the CI pipeline runs `fileassert --validate --results artifacts/fileassert-self-validation.trx` after all documents @@ -25,9 +25,9 @@ incorrect results, causing `reqstream --enforce` to report missing test coverage build. A passing CI build therefore constitutes transitive evidence that FileAssert correctly asserted document content at each stage of the pipeline. -## Test Scenarios +### Test Scenarios -### FileAssert_Results +#### FileAssert_Results **Scenario**: FileAssert self-validation exercises the `--results` option, generating TRX test results containing both passing and failing outcomes. @@ -36,7 +36,7 @@ results containing both passing and failing outcomes. **Requirement coverage**: `Template-OTS-FileAssert`. -### FileAssert_File +#### FileAssert_File **Scenario**: FileAssert self-validation exercises a test configuration using a glob pattern to assert file existence. @@ -45,7 +45,7 @@ assert file existence. **Requirement coverage**: `Template-OTS-FileAssert`. -### FileAssert_Text +#### FileAssert_Text **Scenario**: FileAssert self-validation exercises a test configuration using a `contains` assertion to verify file text content. @@ -54,7 +54,7 @@ to verify file text content. **Requirement coverage**: `Template-OTS-FileAssert`. -### FileAssert_Html +#### FileAssert_Html **Scenario**: FileAssert self-validation exercises a test configuration using an XPath query to assert HTML document structure. @@ -63,7 +63,7 @@ assert HTML document structure. **Requirement coverage**: `Template-OTS-FileAssert`. -### FileAssert_Pdf +#### FileAssert_Pdf **Scenario**: FileAssert self-validation exercises a test configuration asserting PDF metadata fields and body text content. @@ -71,7 +71,3 @@ fields and body text content. **Expected**: Passes when the PDF contains the expected metadata and text. **Requirement coverage**: `Template-OTS-FileAssert`. - -## Requirements Coverage - -- **`Template-OTS-FileAssert`**: FileAssert_Results, FileAssert_File, FileAssert_Text, FileAssert_Html, FileAssert_Pdf diff --git a/docs/verification/ots/pandoc.md b/docs/verification/ots/pandoc.md index 2c0a350..d9c0a21 100644 --- a/docs/verification/ots/pandoc.md +++ b/docs/verification/ots/pandoc.md @@ -1,23 +1,23 @@ -# Pandoc Verification +## Pandoc Verification This document provides the verification evidence for the `Pandoc` OTS software item. -## Required Functionality +### Required Functionality DemaConsulting.PandocTool converts Markdown source documents to HTML as part of the documentation build pipeline. FileAssert validates that each generated HTML file exists, has a non-trivial size, contains a valid HTML title element, and includes expected document content. Passing FileAssert assertions for each document type proves Pandoc executed correctly and produced meaningful output. -## Verification Approach +### Verification Approach Pandoc is verified by self-validation evidence from the CI pipeline. Each scenario is a FileAssert assertion that runs after Pandoc converts a specific Markdown document to HTML. A passing pipeline run for all scenarios constitutes evidence that the requirement is satisfied. -## Test Scenarios +### Test Scenarios -### Pandoc_BuildNotesHtml +#### Pandoc_BuildNotesHtml **Scenario**: FileAssert asserts the build-notes HTML file exists, is non-trivially sized, contains a valid HTML title element, and includes expected document content. @@ -26,7 +26,7 @@ a valid HTML title element, and includes expected document content. **Requirement coverage**: `Template-OTS-Pandoc`. -### Pandoc_CodeQualityHtml +#### Pandoc_CodeQualityHtml **Scenario**: FileAssert asserts the code-quality HTML file exists, is non-trivially sized, contains a valid HTML title element, and includes expected document content. @@ -35,7 +35,7 @@ a valid HTML title element, and includes expected document content. **Requirement coverage**: `Template-OTS-Pandoc`. -### Pandoc_ReviewPlanHtml +#### Pandoc_ReviewPlanHtml **Scenario**: FileAssert asserts the review plan HTML file exists, is non-trivially sized, contains a valid HTML title element, and includes expected document content. @@ -44,7 +44,7 @@ a valid HTML title element, and includes expected document content. **Requirement coverage**: `Template-OTS-Pandoc`. -### Pandoc_ReviewReportHtml +#### Pandoc_ReviewReportHtml **Scenario**: FileAssert asserts the review report HTML file exists, is non-trivially sized, contains a valid HTML title element, and includes expected document content. @@ -53,7 +53,7 @@ contains a valid HTML title element, and includes expected document content. **Requirement coverage**: `Template-OTS-Pandoc`. -### Pandoc_DesignHtml +#### Pandoc_DesignHtml **Scenario**: FileAssert asserts the design document HTML file exists, is non-trivially sized, contains a valid HTML title element, and includes expected document content. @@ -62,7 +62,7 @@ contains a valid HTML title element, and includes expected document content. **Requirement coverage**: `Template-OTS-Pandoc`. -### Pandoc_VerificationHtml +#### Pandoc_VerificationHtml **Scenario**: FileAssert asserts the verification HTML file exists, is non-trivially sized, contains a valid HTML title element, and includes expected verification document content. @@ -71,7 +71,7 @@ a valid HTML title element, and includes expected verification document content. **Requirement coverage**: `Template-OTS-Pandoc`. -### Pandoc_UserGuideHtml +#### Pandoc_UserGuideHtml **Scenario**: FileAssert asserts the user guide HTML file exists, is non-trivially sized, contains a valid HTML title element, and includes expected document content. @@ -79,8 +79,3 @@ a valid HTML title element, and includes expected document content. **Expected**: FileAssert exits 0 for the user guide HTML document. **Requirement coverage**: `Template-OTS-Pandoc`. - -## Requirements Coverage - -- **`Template-OTS-Pandoc`**: Pandoc_BuildNotesHtml, Pandoc_CodeQualityHtml, Pandoc_ReviewPlanHtml, - Pandoc_ReviewReportHtml, Pandoc_DesignHtml, Pandoc_VerificationHtml, Pandoc_UserGuideHtml diff --git a/docs/verification/ots/reqstream.md b/docs/verification/ots/reqstream.md index 9ad191c..344f73b 100644 --- a/docs/verification/ots/reqstream.md +++ b/docs/verification/ots/reqstream.md @@ -1,8 +1,8 @@ -# ReqStream Verification +## ReqStream Verification This document provides the verification evidence for the `ReqStream` OTS software item. -## Required Functionality +### Required Functionality DemaConsulting.ReqStream processes requirements.yaml and the TRX test-result files to produce a requirements report, justifications document, and traceability matrix. When run with `--enforce`, it @@ -10,7 +10,7 @@ exits with a non-zero code if any requirement lacks test evidence, making unprov build-breaking condition. A successful pipeline run with `--enforce` proves all requirements are covered and that ReqStream is functioning. -## Verification Approach +### Verification Approach ReqStream is verified by two complementary layers of evidence. First, the CI pipeline runs `reqstream --validate --results artifacts/reqstream-self-validation.trx`, which exercises @@ -25,9 +25,9 @@ non-zero if any requirement lacks test evidence, which would also fail the build CI build proves ReqStream correctly processed the project's real requirements and found complete test coverage. -## Test Scenarios +### Test Scenarios -### ReqStream_RequirementsProcessing +#### ReqStream_RequirementsProcessing **Scenario**: ReqStream self-validation loads and processes a set of requirements YAML files. @@ -35,7 +35,7 @@ complete test coverage. **Requirement coverage**: `Template-OTS-ReqStream`. -### ReqStream_TraceMatrix +#### ReqStream_TraceMatrix **Scenario**: ReqStream self-validation generates a trace matrix from requirements and TRX test results. @@ -44,7 +44,7 @@ results. **Requirement coverage**: `Template-OTS-ReqStream`. -### ReqStream_ReportExport +#### ReqStream_ReportExport **Scenario**: ReqStream self-validation exports a requirements report to a markdown file. @@ -52,7 +52,7 @@ results. **Requirement coverage**: `Template-OTS-ReqStream`. -### ReqStream_TagsFiltering +#### ReqStream_TagsFiltering **Scenario**: ReqStream self-validation filters requirements by tags. @@ -60,7 +60,7 @@ results. **Requirement coverage**: `Template-OTS-ReqStream`. -### ReqStream_EnforcementMode +#### ReqStream_EnforcementMode **Scenario**: ReqStream self-validation exercises enforcement mode where all requirements have test coverage. @@ -69,7 +69,7 @@ coverage. **Requirement coverage**: `Template-OTS-ReqStream`. -### ReqStream_Lint +#### ReqStream_Lint **Scenario**: ReqStream self-validation exercises lint mode against a requirements file with deliberate issues. @@ -77,8 +77,3 @@ deliberate issues. **Expected**: Correctly identifies and reports the issues. **Requirement coverage**: `Template-OTS-ReqStream`. - -## Requirements Coverage - -- **`Template-OTS-ReqStream`**: ReqStream_RequirementsProcessing, ReqStream_TraceMatrix, - ReqStream_ReportExport, ReqStream_TagsFiltering, ReqStream_EnforcementMode, ReqStream_Lint diff --git a/docs/verification/ots/reviewmark.md b/docs/verification/ots/reviewmark.md index 937ad80..23cc7e1 100644 --- a/docs/verification/ots/reviewmark.md +++ b/docs/verification/ots/reviewmark.md @@ -1,16 +1,16 @@ -# ReviewMark Verification +## ReviewMark Verification This document provides the verification evidence for the ReviewMark OTS software item. Requirements for this OTS item are defined in the ReviewMark OTS Software Requirements document. -## Required Functionality +### Required Functionality DemaConsulting.ReviewMark reads the `.reviewmark.yaml` configuration and the review evidence store to produce a review plan and review report documenting file review coverage and currency. It runs in the same CI pipeline that produces the TRX test results, so a successful pipeline run is evidence that ReviewMark executed without error. -## Verification Approach +### Verification Approach ReviewMark is verified by two complementary layers of evidence. First, the CI pipeline runs `reviewmark --validate --results artifacts/reviewmark-self-validation.trx`, which exercises @@ -24,9 +24,9 @@ WeasyPrint renders both to PDF and FileAssert asserts their content (`WeasyPrint_ReviewPlanPdf`, `WeasyPrint_ReviewReportPdf`). A CI build failure at any step is evidence that ReviewMark did not produce the required review documents. -## Test Scenarios +### Test Scenarios -### ReviewMark_ReviewPlanGeneration +#### ReviewMark_ReviewPlanGeneration **Scenario**: ReviewMark self-validation uses `--definition` and `--plan` to generate a review plan from a test configuration. @@ -35,7 +35,7 @@ from a test configuration. **Requirement coverage**: `Template-OTS-ReviewMark`. -### ReviewMark_ReviewReportGeneration +#### ReviewMark_ReviewReportGeneration **Scenario**: ReviewMark self-validation uses `--definition` and `--report` to generate a review report from a test configuration and evidence store. @@ -44,53 +44,29 @@ report from a test configuration and evidence store. **Requirement coverage**: `Template-OTS-ReviewMark`. -### ReviewMark_IndexScan - -**Scenario**: ReviewMark self-validation uses `--index` to scan PDF evidence files and write an -`index.json` catalogue. - -**Expected**: Exits 0 and produces a correctly structured `index.json`. - -**Requirement coverage**: `Template-OTS-ReviewMark-Operations`. - -### ReviewMark_WorkingDirectoryOverride - -**Scenario**: ReviewMark self-validation uses `--dir` to override the working directory for file -operations. - -**Expected**: Exits 0 and resolves paths relative to the specified directory. - -**Requirement coverage**: `Template-OTS-ReviewMark-Operations`. - -### ReviewMark_Enforce +#### ReviewMark_Enforce **Scenario**: ReviewMark self-validation uses `--enforce` against a configuration with review issues. **Expected**: Exits with a non-zero exit code when review issues are present. -**Requirement coverage**: `Template-OTS-ReviewMark-Operations`. +**Requirement coverage**: `Template-OTS-ReviewMark-Enforce`. -### ReviewMark_Elaborate +#### ReviewMark_Elaborate **Scenario**: ReviewMark self-validation uses `--elaborate` to print a Markdown elaboration of a named review set. **Expected**: Exits 0 and prints the review-set ID, fingerprint, and file list. -**Requirement coverage**: `Template-OTS-ReviewMark-Operations`. +**Requirement coverage**: `Template-OTS-ReviewMark-Elaborate`. -### ReviewMark_Lint +#### ReviewMark_Lint **Scenario**: ReviewMark self-validation uses `--lint` to validate a definition file and report issues. **Expected**: Correctly reports structural and semantic issues found in the test definition. -**Requirement coverage**: `Template-OTS-ReviewMark-Operations`. - -## Requirements Coverage - -- **`Template-OTS-ReviewMark`**: ReviewMark_ReviewPlanGeneration, ReviewMark_ReviewReportGeneration -- **`Template-OTS-ReviewMark-Operations`**: ReviewMark_IndexScan, ReviewMark_WorkingDirectoryOverride, - ReviewMark_Enforce, ReviewMark_Elaborate, ReviewMark_Lint +**Requirement coverage**: `Template-OTS-ReviewMark-Lint`. diff --git a/docs/verification/ots/sarifmark.md b/docs/verification/ots/sarifmark.md index 10d3fcc..102cbd4 100644 --- a/docs/verification/ots/sarifmark.md +++ b/docs/verification/ots/sarifmark.md @@ -1,16 +1,16 @@ -# SarifMark Verification +## SarifMark Verification This document provides the verification evidence for the SarifMark OTS software item. Requirements for this OTS item are defined in the SarifMark OTS Software Requirements document. -## Required Functionality +### Required Functionality DemaConsulting.SarifMark reads the SARIF output produced by CodeQL code scanning and renders it as a human-readable markdown document included in the release artifacts. It runs in the same CI pipeline that produces the TRX test results, so a successful pipeline run is evidence that SarifMark executed without error. -## Verification Approach +### Verification Approach SarifMark is verified by two complementary layers of evidence. First, the CI pipeline runs `sarifmark --validate --results artifacts/sarifmark-self-validation.trx`, which exercises @@ -24,9 +24,9 @@ renders the result to PDF and FileAssert asserts the PDF contains expected conte (`WeasyPrint_CodeQualityPdf`). A CI build failure at any step is evidence that SarifMark did not produce the required output. -## Test Scenarios +### Test Scenarios -### SarifMark_SarifReading +#### SarifMark_SarifReading **Scenario**: SarifMark is invoked with a CodeQL SARIF results file as input. @@ -34,7 +34,7 @@ not produce the required output. **Requirement coverage**: `Template-OTS-SarifMark`. -### SarifMark_MarkdownReportGeneration +#### SarifMark_MarkdownReportGeneration **Scenario**: SarifMark renders the SARIF input as a markdown report included in the release artifacts. @@ -43,7 +43,7 @@ artifacts. **Requirement coverage**: `Template-OTS-SarifMark`. -### SarifMark_Enforcement +#### SarifMark_Enforcement **Scenario**: SarifMark self-validation processes a SARIF file containing known issues in enforcement mode. @@ -51,8 +51,3 @@ enforcement mode. **Expected**: Returns a non-zero exit code when issues are found. **Requirement coverage**: `Template-OTS-SarifMark-Enforcement`. - -## Requirements Coverage - -- **`Template-OTS-SarifMark`**: SarifMark_SarifReading, SarifMark_MarkdownReportGeneration -- **`Template-OTS-SarifMark-Enforcement`**: SarifMark_Enforcement diff --git a/docs/verification/ots/sonarmark.md b/docs/verification/ots/sonarmark.md index 57155a6..27e1375 100644 --- a/docs/verification/ots/sonarmark.md +++ b/docs/verification/ots/sonarmark.md @@ -1,16 +1,16 @@ -# SonarMark Verification +## SonarMark Verification This document provides the verification evidence for the SonarMark OTS software item. Requirements for this OTS item are defined in the SonarMark OTS Software Requirements document. -## Required Functionality +### Required Functionality DemaConsulting.SonarMark retrieves quality-gate and metrics data from SonarCloud and renders it as a markdown document included in the release artifacts. It runs in the same CI pipeline that produces the TRX test results, so a successful pipeline run is evidence that SonarMark executed without error. -## Verification Approach +### Verification Approach SonarMark is verified by two complementary layers of evidence. First, the CI pipeline runs `sonarmark --validate --results artifacts/sonarmark-self-validation.trx`, which exercises @@ -24,9 +24,9 @@ renders the result to PDF and FileAssert asserts the PDF contains expected conte (`WeasyPrint_CodeQualityPdf`). A CI build failure at any step is evidence that SonarMark did not retrieve and render quality data correctly. -## Test Scenarios +### Test Scenarios -### SonarMark_QualityGateRetrieval +#### SonarMark_QualityGateRetrieval **Scenario**: SonarMark queries the SonarCloud API for quality-gate status. @@ -34,7 +34,7 @@ not retrieve and render quality data correctly. **Requirement coverage**: `Template-OTS-SonarMark`. -### SonarMark_IssuesRetrieval +#### SonarMark_IssuesRetrieval **Scenario**: SonarMark queries the SonarCloud API for issues. @@ -42,7 +42,7 @@ not retrieve and render quality data correctly. **Requirement coverage**: `Template-OTS-SonarMark`. -### SonarMark_HotSpotsRetrieval +#### SonarMark_HotSpotsRetrieval **Scenario**: SonarMark queries the SonarCloud API for hot spots. @@ -50,15 +50,10 @@ not retrieve and render quality data correctly. **Requirement coverage**: `Template-OTS-SonarMark`. -### SonarMark_MarkdownReportGeneration +#### SonarMark_MarkdownReportGeneration **Scenario**: SonarMark renders quality-gate, issues, and hot-spots data as a markdown report. **Expected**: Exits 0 and produces a non-empty markdown quality report. **Requirement coverage**: `Template-OTS-SonarMark`. - -## Requirements Coverage - -- **`Template-OTS-SonarMark`**: SonarMark_QualityGateRetrieval, SonarMark_IssuesRetrieval, - SonarMark_HotSpotsRetrieval, SonarMark_MarkdownReportGeneration diff --git a/docs/verification/ots/versionmark.md b/docs/verification/ots/versionmark.md index 56b2ff9..6574a05 100644 --- a/docs/verification/ots/versionmark.md +++ b/docs/verification/ots/versionmark.md @@ -1,15 +1,15 @@ -# VersionMark Verification +## VersionMark Verification This document provides the verification evidence for the `VersionMark` OTS software item. -## Required Functionality +### Required Functionality DemaConsulting.VersionMark reads version metadata for each dotnet tool used in the pipeline and writes a versions markdown document included in the release artifacts. It runs in the same CI pipeline that produces the TRX test results, so a successful pipeline run is evidence that VersionMark executed without error. -## Verification Approach +### Verification Approach VersionMark is verified by two complementary layers of evidence. First, the CI pipeline runs `versionmark --validate --results *.trx` in each build job, exercising VersionMark's built-in @@ -23,9 +23,9 @@ compilation would be incomplete. WeasyPrint renders the result to PDF and FileAs its content (`WeasyPrint_BuildNotesPdf`). A CI build failure at any step is evidence that VersionMark did not execute correctly. -## Test Scenarios +### Test Scenarios -### VersionMark_CapturesVersions +#### VersionMark_CapturesVersions **Scenario**: VersionMark reads version metadata for each dotnet tool defined in the pipeline. @@ -33,7 +33,7 @@ VersionMark did not execute correctly. **Requirement coverage**: `Template-OTS-VersionMark`. -### VersionMark_GeneratesMarkdownReport +#### VersionMark_GeneratesMarkdownReport **Scenario**: VersionMark writes a versions markdown document to the release artifacts. @@ -41,7 +41,7 @@ VersionMark did not execute correctly. **Requirement coverage**: `Template-OTS-VersionMark`. -### VersionMark_LintPassesForValidConfig +#### VersionMark_LintPassesForValidConfig **Scenario**: VersionMark self-validation exercises lint mode against a valid `.versionmark.yaml` config. @@ -50,7 +50,7 @@ config. **Requirement coverage**: `Template-OTS-VersionMark`. -### VersionMark_LintReportsErrorsForInvalidConfig +#### VersionMark_LintReportsErrorsForInvalidConfig **Scenario**: VersionMark self-validation exercises lint mode against a malformed config with deliberate errors. @@ -58,8 +58,3 @@ deliberate errors. **Expected**: Correctly identifies and reports the configuration errors. **Requirement coverage**: `Template-OTS-VersionMark`. - -## Requirements Coverage - -- **`Template-OTS-VersionMark`**: VersionMark_CapturesVersions, VersionMark_GeneratesMarkdownReport, - VersionMark_LintPassesForValidConfig, VersionMark_LintReportsErrorsForInvalidConfig diff --git a/docs/verification/ots/weasyprint.md b/docs/verification/ots/weasyprint.md index 60d2ac1..fcf650e 100644 --- a/docs/verification/ots/weasyprint.md +++ b/docs/verification/ots/weasyprint.md @@ -1,9 +1,9 @@ -# WeasyPrint Verification +## WeasyPrint Verification This document provides the verification evidence for the WeasyPrint OTS software item. Requirements for this OTS item are defined in the WeasyPrint OTS Software Requirements document. -## Required Functionality +### Required Functionality DemaConsulting.WeasyPrintTool converts HTML documents to PDF as part of the documentation build pipeline. FileAssert validates that each generated PDF file exists, has a non-trivial size, contains @@ -11,15 +11,15 @@ at least one page, and includes expected document content in the rendered text. assertions for each document type proves WeasyPrint executed correctly and produced meaningful output. -## Verification Approach +### Verification Approach WeasyPrint is verified by self-validation evidence from the CI pipeline. Each scenario is a FileAssert assertion that runs after WeasyPrint converts a specific HTML document to PDF. A passing pipeline run for all scenarios constitutes evidence that the requirement is satisfied. -## Test Scenarios +### Test Scenarios -### WeasyPrint_BuildNotesPdf +#### WeasyPrint_BuildNotesPdf **Scenario**: FileAssert asserts the build-notes PDF file exists, is non-trivially sized, contains at least one page, and includes expected document content. @@ -28,7 +28,7 @@ at least one page, and includes expected document content. **Requirement coverage**: `Template-OTS-WeasyPrint`. -### WeasyPrint_CodeQualityPdf +#### WeasyPrint_CodeQualityPdf **Scenario**: FileAssert asserts the code-quality PDF file exists, is non-trivially sized, contains at least one page, and includes expected document content. @@ -37,7 +37,7 @@ at least one page, and includes expected document content. **Requirement coverage**: `Template-OTS-WeasyPrint`. -### WeasyPrint_ReviewPlanPdf +#### WeasyPrint_ReviewPlanPdf **Scenario**: FileAssert asserts the review plan PDF file exists, is non-trivially sized, contains at least one page, and includes expected document content. @@ -46,7 +46,7 @@ at least one page, and includes expected document content. **Requirement coverage**: `Template-OTS-WeasyPrint`. -### WeasyPrint_ReviewReportPdf +#### WeasyPrint_ReviewReportPdf **Scenario**: FileAssert asserts the review report PDF file exists, is non-trivially sized, contains at least one page, and includes expected document content. @@ -55,7 +55,7 @@ at least one page, and includes expected document content. **Requirement coverage**: `Template-OTS-WeasyPrint`. -### WeasyPrint_DesignPdf +#### WeasyPrint_DesignPdf **Scenario**: FileAssert asserts the design document PDF file exists, is non-trivially sized, contains at least one page, and includes expected document content. @@ -64,7 +64,7 @@ contains at least one page, and includes expected document content. **Requirement coverage**: `Template-OTS-WeasyPrint`. -### WeasyPrint_VerificationPdf +#### WeasyPrint_VerificationPdf **Scenario**: FileAssert asserts the verification PDF file exists, is non-trivially sized, contains at least one page, and includes expected verification document content. @@ -73,7 +73,7 @@ at least one page, and includes expected verification document content. **Requirement coverage**: `Template-OTS-WeasyPrint`. -### WeasyPrint_UserGuidePdf +#### WeasyPrint_UserGuidePdf **Scenario**: FileAssert asserts the user guide PDF file exists, is non-trivially sized, contains at least one page, and includes expected document content. @@ -81,9 +81,3 @@ at least one page, and includes expected document content. **Expected**: FileAssert exits 0 for the user guide PDF document. **Requirement coverage**: `Template-OTS-WeasyPrint`. - -## Requirements Coverage - -- **`Template-OTS-WeasyPrint`**: WeasyPrint_BuildNotesPdf, WeasyPrint_CodeQualityPdf, - WeasyPrint_ReviewPlanPdf, WeasyPrint_ReviewReportPdf, WeasyPrint_DesignPdf, - WeasyPrint_VerificationPdf, WeasyPrint_UserGuidePdf diff --git a/docs/verification/ots/xunit.md b/docs/verification/ots/xunit.md index ccdfc42..dab90a3 100644 --- a/docs/verification/ots/xunit.md +++ b/docs/verification/ots/xunit.md @@ -1,24 +1,24 @@ -# xUnit Verification +## xUnit Verification This document provides the verification evidence for the xUnit OTS software item. Requirements for this OTS item are defined in the xUnit OTS Software Requirements document. -## Required Functionality +### Required Functionality xUnit v3 (xunit.v3 and xunit.runner.visualstudio) is the unit-testing framework used by the project. It discovers and runs all test methods and writes TRX result files that feed into coverage reporting and requirements traceability. Passing tests confirm the framework is functioning correctly. -## Verification Approach +### Verification Approach xUnit is verified by self-validation evidence from the CI pipeline. Each scenario names a specific test method that xUnit must discover, execute, and record in a TRX result file. A passing pipeline run for all scenarios constitutes evidence that both requirements are satisfied. -## Test Scenarios +### Test Scenarios -### Context_Create_NoArguments_ReturnsDefaultContext +#### Context_Create_NoArguments_ReturnsDefaultContext **Scenario**: xUnit discovers and runs this test; the test verifies Context default construction. @@ -26,7 +26,7 @@ run for all scenarios constitutes evidence that both requirements are satisfied. **Requirement coverage**: `Template-OTS-xUnit-Execute`, `Template-OTS-xUnit-Report`. -### Context_Create_VersionFlag_SetsVersionTrue +#### Context_Create_VersionFlag_SetsVersionTrue **Scenario**: xUnit discovers and runs this test; the test verifies that passing the --version flag sets the Version property to true. @@ -35,7 +35,7 @@ sets the Version property to true. **Requirement coverage**: `Template-OTS-xUnit-Execute`, `Template-OTS-xUnit-Report`. -### Context_Create_SilentFlag_SetsSilentTrue +#### Context_Create_SilentFlag_SetsSilentTrue **Scenario**: xUnit discovers and runs this test; the test verifies that passing the --silent flag sets the Silent property to true. @@ -44,7 +44,7 @@ sets the Silent property to true. **Requirement coverage**: `Template-OTS-xUnit-Execute`, `Template-OTS-xUnit-Report`. -### Context_Create_LogFlag_OpensLogFile +#### Context_Create_LogFlag_OpensLogFile **Scenario**: xUnit discovers and runs this test; the test verifies that passing the --log flag opens a log file. @@ -53,7 +53,7 @@ opens a log file. **Requirement coverage**: `Template-OTS-xUnit-Execute`, `Template-OTS-xUnit-Report`. -### Context_Create_UnknownArgument_ThrowsArgumentException +#### Context_Create_UnknownArgument_ThrowsArgumentException **Scenario**: xUnit discovers and runs this test; the test verifies that an unrecognized argument raises an exception. @@ -62,7 +62,7 @@ raises an exception. **Requirement coverage**: `Template-OTS-xUnit-Execute`, `Template-OTS-xUnit-Report`. -### PathHelpers_SafePathCombine_ValidPaths_CombinesCorrectly +#### PathHelpers_SafePathCombine_ValidPaths_CombinesCorrectly **Scenario**: xUnit discovers and runs this test; the test verifies that SafePathCombine correctly joins valid path segments. @@ -71,7 +71,7 @@ joins valid path segments. **Requirement coverage**: `Template-OTS-xUnit-Execute`, `Template-OTS-xUnit-Report`. -### Program_Run_WithVersionFlag_DisplaysVersionOnly +#### Program_Run_WithVersionFlag_DisplaysVersionOnly **Scenario**: xUnit discovers and runs this test; the test verifies that the program prints only version information when invoked with the --version flag. @@ -80,7 +80,7 @@ version information when invoked with the --version flag. **Requirement coverage**: `Template-OTS-xUnit-Execute`, `Template-OTS-xUnit-Report`. -### Validation_Run_WithSilentContext_PrintsSummary +#### Validation_Run_WithSilentContext_PrintsSummary **Scenario**: xUnit discovers and runs this test; the test verifies that Validation.Run prints a summary even when the context is configured for silent operation. @@ -88,16 +88,3 @@ summary even when the context is configured for silent operation. **Expected**: xUnit executes the test, the test passes, and the result appears in the TRX output. **Requirement coverage**: `Template-OTS-xUnit-Execute`, `Template-OTS-xUnit-Report`. - -## Requirements Coverage - -- **`Template-OTS-xUnit-Execute`**: Context_Create_NoArguments_ReturnsDefaultContext, - Context_Create_VersionFlag_SetsVersionTrue, Context_Create_SilentFlag_SetsSilentTrue, - Context_Create_LogFlag_OpensLogFile, Context_Create_UnknownArgument_ThrowsArgumentException, - PathHelpers_SafePathCombine_ValidPaths_CombinesCorrectly, - Program_Run_WithVersionFlag_DisplaysVersionOnly, Validation_Run_WithSilentContext_PrintsSummary -- **`Template-OTS-xUnit-Report`**: Context_Create_NoArguments_ReturnsDefaultContext, - Context_Create_VersionFlag_SetsVersionTrue, Context_Create_SilentFlag_SetsSilentTrue, - Context_Create_LogFlag_OpensLogFile, Context_Create_UnknownArgument_ThrowsArgumentException, - PathHelpers_SafePathCombine_ValidPaths_CombinesCorrectly, - Program_Run_WithVersionFlag_DisplaysVersionOnly, Validation_Run_WithSilentContext_PrintsSummary diff --git a/docs/verification/template-dot-net-tool.md b/docs/verification/template-dot-net-tool.md new file mode 100644 index 0000000..3a83cfc --- /dev/null +++ b/docs/verification/template-dot-net-tool.md @@ -0,0 +1,87 @@ +# TemplateDotNetTool + +## Verification Approach + +System-level verification uses end-to-end integration tests that invoke the tool as a real +process via the `Runner.Run` helper in `IntegrationTests.cs`. Each test exercises the full +stack — argument parsing, dispatch, execution, and output — and validates both exit code and +combined console output. The tests treat the tool as a black box and assert only on observable +outputs; no internal implementation details are assumed. + +`Runner.Run` merges stdout and stderr into a single combined output string. Per-stream assertions +(e.g., "standard error is empty") are not possible at this level; all assertions are made +against the combined output. + +## Test Environment + +Integration tests run on .NET 8.0, .NET 9.0, and .NET 10.0 across Windows, Linux, and macOS. +All scenarios are expected to produce identical results on all supported runtime and platform +combinations. Temporary files and directories are created and cleaned up within each test. + +## Acceptance Criteria + +- All integration tests pass with zero failures across all supported runtimes and platforms. +- Exit code 0 is returned for all valid invocations. +- Exit code non-zero is returned for all invalid argument combinations. +- Results files are created at the specified paths when `--results` is used with `--validate`. +- Silent mode (`--silent`) produces empty combined output. + +## Test Scenarios + +**TemplateDotNetTool_VersionFlag_Provided_OutputsVersion**: The `--version` flag is passed as +the sole argument; the tool outputs the version string and exits with code 0. This scenario is +tested by `TemplateDotNetTool_VersionFlag_Provided_OutputsVersion`. + +**TemplateDotNetTool_HelpFlag_Provided_OutputsUsageInformation**: The `--help` flag is passed +as the sole argument; the combined output contains "Usage" and "Options" and the tool exits +with code 0. This scenario is tested by +`TemplateDotNetTool_HelpFlag_Provided_OutputsUsageInformation`. + +**TemplateDotNetTool_ValidateFlag_Provided_RunsValidation**: The `--validate` flag is passed as +the sole argument; the combined output contains "Total Tests:" and the tool exits with code 0, +confirming the self-validation suite runs and completes successfully. This scenario is tested by +`TemplateDotNetTool_ValidateFlag_Provided_RunsValidation`. + +**TemplateDotNetTool_ValidateWithTrxResults_Requested_GeneratesTrxFile**: The `--validate` flag +is combined with `--results .trx`; a TRX file containing a `.xml`; a JUnit XML file containing a `` flag is passed +pointing to a temporary file; the tool exits with code 0 and the log file is created containing +output that also appears in the combined console output. This scenario is tested by +`TemplateDotNetTool_LogFlag_Provided_WritesOutputToFile`. + +**TemplateDotNetTool_UnknownArgument_Provided_ReturnsError**: An unrecognized argument +(`--unknown`) is passed; the tool exits with a non-zero code and the combined output contains +an error message identifying the unknown argument. This scenario is tested by +`TemplateDotNetTool_UnknownArgument_Provided_ReturnsError`. + +**TemplateDotNetTool_ValidateWithDepth_DepthThree_OutputsCorrectHeadingLevel**: The `--validate` +flag is combined with `--depth 3`; the combined output contains `###` (heading at depth 3) and +the tool exits with code 0. This scenario is tested by +`TemplateDotNetTool_ValidateWithDepth_DepthThree_OutputsCorrectHeadingLevel`. + +**TemplateDotNetTool_NoArguments_Invoked_DisplaysBanner**: The tool is invoked with no +arguments; the combined output contains the tool name and copyright notice and the exit code is +0. This scenario is tested by `TemplateDotNetTool_NoArguments_Invoked_DisplaysBanner`. + +**TemplateDotNetTool_ResultAlias_LegacyFlag_WritesResultsFile**: The `--validate` flag is +combined with `--result .trx` (the legacy alias); a TRX file is created at the specified +path and the tool exits with code 0. This scenario is tested by +`TemplateDotNetTool_ResultAlias_LegacyFlag_WritesResultsFile`. + +**TemplateDotNetTool_ValidateWithBadExtension_ExtensionInvalid_ReturnsNonZero**: The `--validate` +flag is combined with `--results .bad` (unsupported extension); the tool exits with a +non-zero code and no file is created at the specified path. This scenario is tested by +`TemplateDotNetTool_ValidateWithBadExtension_ExtensionInvalid_ReturnsNonZero`. diff --git a/docs/verification/template-dot-net-tool/cli.md b/docs/verification/template-dot-net-tool/cli.md new file mode 100644 index 0000000..2109c90 --- /dev/null +++ b/docs/verification/template-dot-net-tool/cli.md @@ -0,0 +1,87 @@ +## Cli + +### Verification Approach + +The `Cli` subsystem is verified by integration tests defined in `CliSubsystemTests.cs`. Each +test exercises `Context.Create` and `Program.Run` together, treating the pair as the observable +subsystem interface. Tests pass controlled argument arrays and assert on captured console output, +file-system side-effects, and exit codes. `Validation` (part of the `SelfTest` subsystem) +executes its real logic in scenarios that exercise the `--validate` path; no mocking is applied +at any level. + +### Test Environment + +N/A - standard test environment. + +### Acceptance Criteria + +- All integration tests pass with zero failures. +- All supported flags produce the correct observable behavior when passed through `Context.Create` + and `Program.Run`. +- Invalid arguments produce exit code 1 and an error message. +- Log files and results files are created at the specified paths when the corresponding flags + are used. + +### Test Scenarios + +**CliSubsystem_VersionFlow_ContextAndProgram_DisplaysVersionAndExits**: Arguments `["--version"]` +are passed through `Context.Create` and `Program.Run`; standard output contains the version +string and exit code is 0. This scenario is tested by +`CliSubsystem_VersionFlow_ContextAndProgram_DisplaysVersionAndExits`. + +**CliSubsystem_VersionFlow_ContextAndProgram_DisplaysVersionAndExits_WithShortVFlag**: Arguments +`["-v"]` are passed through `Context.Create` and `Program.Run`; standard output contains the +version string and exit code is 0. This scenario is tested by +`CliSubsystem_VersionFlow_ContextAndProgram_DisplaysVersionAndExits_WithShortVFlag`. + +**CliSubsystem_HelpFlow_ContextAndProgram_DisplaysHelpAndExits**: Arguments `["--help"]` are +passed through `Context.Create` and `Program.Run`; standard output contains help text and exit +code is 0. This scenario is tested by +`CliSubsystem_HelpFlow_ContextAndProgram_DisplaysHelpAndExits`. + +**CliSubsystem_HelpFlow_ContextAndProgram_DisplaysHelpAndExits_WithShortQuestionFlag**: Arguments +`["-?"]` are passed; standard output contains help text and exit code is 0. This scenario is +tested by +`CliSubsystem_HelpFlow_ContextAndProgram_DisplaysHelpAndExits_WithShortQuestionFlag`. + +**CliSubsystem_HelpFlow_ContextAndProgram_DisplaysHelpAndExits_WithShortHFlag**: Arguments +`["-h"]` are passed; standard output contains help text and exit code is 0. This scenario is +tested by `CliSubsystem_HelpFlow_ContextAndProgram_DisplaysHelpAndExits_WithShortHFlag`. + +**CliSubsystem_ValidateFlow_ContextAndProgram_RunsValidationAndExits**: Arguments `["--validate"]` +are passed; standard output contains `"Total Tests:"` and exit code is 0. This scenario is +tested by `CliSubsystem_ValidateFlow_ContextAndProgram_RunsValidationAndExits`. + +**CliSubsystem_SilentFlow_ContextAndProgram_SuppressesOutput**: Arguments `["--version", "--silent"]` +are passed; standard output is empty and exit code is 0, confirming `--silent` suppresses all +console output. This scenario is tested by +`CliSubsystem_SilentFlow_ContextAndProgram_SuppressesOutput`. + +**CliSubsystem_ResultsFlow_ContextAndProgram_WritesResultsFile**: Arguments +`["--validate", "--results", ".trx"]` are passed; a results file is created at the +specified path and exit code is 0. This scenario is tested by +`CliSubsystem_ResultsFlow_ContextAndProgram_WritesResultsFile`. + +**CliSubsystem_LogFlow_ContextAndProgram_WritesLogFile**: Arguments `["--log", ".log"]` are +passed; a log file is created at the specified path and exit code is 0. This scenario is tested +by `CliSubsystem_LogFlow_ContextAndProgram_WritesLogFile`. + +**CliSubsystem_InvalidArgs_ContextAndProgram_RejectsUnknownArgumentsAndExitsNonZero**: Arguments +`["--unknown-flag"]` are passed directly to `Program.Main`; exit code is 1 and standard error +contains an error message including the unknown flag. This scenario is tested by +`CliSubsystem_InvalidArgs_ContextAndProgram_RejectsUnknownArgumentsAndExitsNonZero`. + +**CliSubsystem_ErrorOutput_ContextAndProgram_WritesErrorToStderr**: A `Context` is created with +no arguments and `WriteError` is called with a known message; standard error receives the +message and exit code is non-zero. This scenario is tested by +`CliSubsystem_ErrorOutput_ContextAndProgram_WritesErrorToStderr`. + +**CliSubsystem_ResultAliasFlow_ContextAndProgram_WritesResultsFile**: Arguments +`["--validate", "--result", ".trx"]` (legacy alias) are passed; a results file is created +at the specified path and exit code is 0. This scenario is tested by +`CliSubsystem_ResultAliasFlow_ContextAndProgram_WritesResultsFile`. + +**CliSubsystem_DepthFlow_ContextAndProgram_AdjustsHeadingDepth**: Arguments `["--depth", "2"]` +are passed; the heading depth is set to 2 and exit code is 0, confirming the `--depth` flag is +correctly parsed and propagated through `Context`. This scenario is tested by +`CliSubsystem_DepthFlow_ContextAndProgram_AdjustsHeadingDepth`. diff --git a/docs/verification/template-dot-net-tool/cli/cli.md b/docs/verification/template-dot-net-tool/cli/cli.md deleted file mode 100644 index 6cc479e..0000000 --- a/docs/verification/template-dot-net-tool/cli/cli.md +++ /dev/null @@ -1,134 +0,0 @@ -# Cli Subsystem Verification - -This document describes the subsystem-level verification design for the `Cli` subsystem. It -defines the integration test approach, subsystem boundary, mocking strategy, and test scenarios -that together verify the `Cli` subsystem requirements. - -## Verification Approach - -The `Cli` subsystem boundary at `Program` is verified by integration tests defined in -`CliSubsystemTests.cs`. Each test exercises `Context.Create` and `Program.Run` together, treating -the pair as the observable subsystem interface. Tests pass controlled argument arrays and assert -on captured console output, file system side-effects, and exit codes. - -## Dependencies and Mocking Strategy - -At the subsystem boundary, `Validation` (part of the `SelfTest` subsystem) is the only external -collaborator that `Program` calls. In scenarios that exercise the `--validate` path, `Validation` -executes its real logic rather than being stubbed. Scenarios that do not involve `--validate` do -not reach `Validation` at all. - -No mocking is applied at this level; all collaborators within and directly adjacent to the -subsystem use their real implementations. - -## Integration Test Scenarios - -The following integration test scenarios are defined in `CliSubsystemTests.cs`. - -### CliSubsystem_VersionFlow_ContextAndProgram_DisplaysVersionAndExits - -**Scenario**: Arguments `["--version"]` are passed through `Context.Create` and `Program.Run`. - -**Expected**: Standard output contains the version string; exit code is 0. - -### CliSubsystem_VersionFlow_ContextAndProgram_DisplaysVersionAndExits_WithShortVFlag - -**Scenario**: Arguments `["-v"]` are passed through `Context.Create` and `Program.Run`. - -**Expected**: Standard output contains the version string; exit code is 0. - -### CliSubsystem_HelpFlow_ContextAndProgram_DisplaysHelpAndExits - -**Scenario**: Arguments `["--help"]` are passed through `Context.Create` and `Program.Run`. - -**Expected**: Standard output contains help text; exit code is 0. - -### CliSubsystem_HelpFlow_ContextAndProgram_DisplaysHelpAndExits_WithShortQuestionFlag - -**Scenario**: Arguments `["-?"]` are passed through `Context.Create` and `Program.Run`. - -**Expected**: Standard output contains help text; exit code is 0. - -### CliSubsystem_HelpFlow_ContextAndProgram_DisplaysHelpAndExits_WithShortHFlag - -**Scenario**: Arguments `["-h"]` are passed through `Context.Create` and `Program.Run`. - -**Expected**: Standard output contains help text; exit code is 0. - -### CliSubsystem_ValidateFlow_ContextAndProgram_RunsValidationAndExits - -**Scenario**: Arguments `["--validate"]` are passed through `Context.Create` and `Program.Run`. - -**Expected**: Standard output contains `"Total Tests:"`; exit code is 0. - -### CliSubsystem_SilentFlow_ContextAndProgram_SuppressesOutput - -**Scenario**: Arguments `["--silent"]` are passed through `Context.Create` and `Program.Run`. - -**Expected**: Standard output is empty; exit code is 0. - -### CliSubsystem_ResultsFlow_ContextAndProgram_WritesResultsFile - -**Scenario**: Arguments `["--validate", "--results", ".trx"]` are passed through -`Context.Create` and `Program.Run`. - -**Expected**: A results file is created at the specified path; exit code is 0. - -### CliSubsystem_LogFlow_ContextAndProgram_WritesLogFile - -**Scenario**: Arguments `["--log", ".log"]` are passed through `Context.Create` and -`Program.Run`. - -**Expected**: A log file is created at the specified path; exit code is 0. - -### CliSubsystem_InvalidArgs_ContextAndProgram_RejectsUnknownArgumentsAndExitsNonZero - -**Scenario**: The private `Program.Main` entry point is invoked via reflection with -`["--unknown-flag"]`. This exercises the full argument-parsing and error-reporting path -including the `ArgumentException` handler in `Main`. - -**Expected**: Exit code is 1; standard error contains an error message including the unknown flag. - -### CliSubsystem_ErrorOutput_ContextAndProgram_WritesErrorToStderr - -**Scenario**: A `Context` is created with no arguments. `WriteError` is called directly with a -known message string. - -**Expected**: Standard error receives the error message; exit code is non-zero. - -### CliSubsystem_ResultAliasFlow_ContextAndProgram_WritesResultsFile - -**Scenario**: Arguments `["--validate", "--result", ".trx"]` (legacy alias) are passed -through `Context.Create` and `Program.Run`. - -**Expected**: A results file is created at the specified path; exit code is 0. - -### CliSubsystem_DepthFlow_ContextAndProgram_AdjustsHeadingDepth - -**Scenario**: Arguments `["--depth", "2"]` are passed through `Context.Create` and `Program.Run`. - -**Expected**: The heading depth is set to the specified value; exit code is 0. - -## Requirements Coverage - -- **`Template-Cli-Version`**: CliSubsystem_VersionFlow_ContextAndProgram_DisplaysVersionAndExits, - CliSubsystem_VersionFlow_ContextAndProgram_DisplaysVersionAndExits_WithShortVFlag -- **`Template-Cli-Help`**: CliSubsystem_HelpFlow_ContextAndProgram_DisplaysHelpAndExits, - CliSubsystem_HelpFlow_ContextAndProgram_DisplaysHelpAndExits_WithShortQuestionFlag, - CliSubsystem_HelpFlow_ContextAndProgram_DisplaysHelpAndExits_WithShortHFlag -- **`Template-Cli-Validate`**: CliSubsystem_ValidateFlow_ContextAndProgram_RunsValidationAndExits -- **`Template-Cli-Silent`**: CliSubsystem_SilentFlow_ContextAndProgram_SuppressesOutput -- **`Template-Cli-Results`**: CliSubsystem_ResultsFlow_ContextAndProgram_WritesResultsFile -- **`Template-Cli-Log`**: CliSubsystem_LogFlow_ContextAndProgram_WritesLogFile -- **`Template-Cli-InvalidArgs`**: CliSubsystem_InvalidArgs_ContextAndProgram_RejectsUnknownArgumentsAndExitsNonZero -- **`Template-Cli-ErrorOutput`**: CliSubsystem_ErrorOutput_ContextAndProgram_WritesErrorToStderr -- **`Template-Cli-ArgumentParsing`**: CliSubsystem_VersionFlow_ContextAndProgram_DisplaysVersionAndExits, - CliSubsystem_HelpFlow_ContextAndProgram_DisplaysHelpAndExits, - CliSubsystem_HelpFlow_ContextAndProgram_DisplaysHelpAndExits_WithShortQuestionFlag, - CliSubsystem_HelpFlow_ContextAndProgram_DisplaysHelpAndExits_WithShortHFlag, - CliSubsystem_ValidateFlow_ContextAndProgram_RunsValidationAndExits -- **`Template-Cli-OutputChannels`**: CliSubsystem_SilentFlow_ContextAndProgram_SuppressesOutput, - CliSubsystem_ErrorOutput_ContextAndProgram_WritesErrorToStderr -- **`Template-Cli-ExitCode`**: CliSubsystem_InvalidArgs_ContextAndProgram_RejectsUnknownArgumentsAndExitsNonZero -- **`Template-Cli-Depth`**: CliSubsystem_DepthFlow_ContextAndProgram_AdjustsHeadingDepth -- **`Template-Cli-ResultAlias`**: CliSubsystem_ResultAliasFlow_ContextAndProgram_WritesResultsFile diff --git a/docs/verification/template-dot-net-tool/cli/context.md b/docs/verification/template-dot-net-tool/cli/context.md index 31f121c..93d956b 100644 --- a/docs/verification/template-dot-net-tool/cli/context.md +++ b/docs/verification/template-dot-net-tool/cli/context.md @@ -1,269 +1,137 @@ -# Context Verification +### Context -This document describes the unit-level verification design for the `Context` unit. It defines the -test scenarios, dependency usage, and requirement coverage for `Cli/Context.cs`. +#### Verification Approach -## Verification Approach +`Context` is verified with unit tests defined in `ContextTests.cs`. Because `Context` depends +only on .NET BCL types (`Console`, `StreamWriter`), no mocking or test doubles are required. +Tests call `Context.Create` with controlled argument arrays, inspect the resulting properties +and exit codes, and verify output written to captured streams. -`Context` is verified with unit tests defined in `ContextTests.cs`. Because `Context` depends only -on .NET base class library types (`Console`, `StreamWriter`, `Path`), no mocking or test doubles -are required. Tests call `Context.Create` with controlled argument arrays, inspect the resulting -properties and exit codes, and verify output written to captured streams. +#### Test Environment -## Dependencies +N/A - standard test environment. -`Context` has no dependencies on other tool units. All dependencies are real .NET BCL types; -no mocking is needed at this level. +#### Acceptance Criteria -## Test Scenarios +- All unit tests pass with zero failures. +- All flag properties are correctly set for each recognized argument. +- `ArgumentException` is thrown for all unknown or malformed arguments. +- `ExitCode` is 1 after `WriteError` is called and 0 otherwise. +- Silent mode suppresses console output but does not affect the log file. -### Context_Create_NoArguments_ReturnsDefaultContext +#### Test Scenarios -**Scenario**: `Context.Create` is called with an empty argument array. +**Context_Create_NoArguments_ReturnsDefaultContext**: `Context.Create` is called with an empty +argument array; all boolean flags are false, `ResultsFile` is null, `HeadingDepth` is 1, and +exit code is 0. This scenario is tested by `Context_Create_NoArguments_ReturnsDefaultContext`. -**Expected**: All boolean flags are false; `ResultsFile` is null; `HeadingDepth` is 1; -exit code is 0. +**Context_Create_VersionFlag_SetsVersionTrue**: `Context.Create` is called with `["--version"]`; +the `Version` property is true. This scenario is tested by +`Context_Create_VersionFlag_SetsVersionTrue`. -**Requirement coverage**: Default context creation requirement. +**Context_Create_ShortVersionFlag_SetsVersionTrue**: `Context.Create` is called with `["-v"]`; +the `Version` property is true. This scenario is tested by +`Context_Create_ShortVersionFlag_SetsVersionTrue`. -### Context_Create_VersionFlag_SetsVersionTrue +**Context_Create_HelpFlag_SetsHelpTrue**: `Context.Create` is called with `["--help"]`; the +`Help` property is true. This scenario is tested by `Context_Create_HelpFlag_SetsHelpTrue`. -**Scenario**: `Context.Create` is called with `["--version"]`. +**Context_Create_ShortHelpFlag_H_SetsHelpTrue**: `Context.Create` is called with `["-h"]`; the +`Help` property is true. This scenario is tested by +`Context_Create_ShortHelpFlag_H_SetsHelpTrue`. -**Expected**: `Version` property is true. +**Context_Create_ShortHelpFlag_Question_SetsHelpTrue**: `Context.Create` is called with `["-?"]`; +the `Help` property is true. This scenario is tested by +`Context_Create_ShortHelpFlag_Question_SetsHelpTrue`. -**Requirement coverage**: Version flag parsing requirement. +**Context_Create_SilentFlag_SetsSilentTrue**: `Context.Create` is called with `["--silent"]`; +the `Silent` property is true. This scenario is tested by +`Context_Create_SilentFlag_SetsSilentTrue`. -### Context_Create_ShortVersionFlag_SetsVersionTrue +**Context_Create_ValidateFlag_SetsValidateTrue**: `Context.Create` is called with +`["--validate"]`; the `Validate` property is true. This scenario is tested by +`Context_Create_ValidateFlag_SetsValidateTrue`. -**Scenario**: `Context.Create` is called with `["-v"]`. +**Context_Create_ResultsFlag_SetsResultsFile**: `Context.Create` is called with +`["--results", "output.trx"]`; `ResultsFile` equals `"output.trx"`. This scenario is tested by +`Context_Create_ResultsFlag_SetsResultsFile`. -**Expected**: `Version` property is true. +**Context_Create_LogFlag_OpensLogFile**: `Context.Create` is called with +`["--log", ".log"]` and then `WriteLine` is called with a test message; the log file is +created and contains the test message. This scenario is tested by +`Context_Create_LogFlag_OpensLogFile`. -**Requirement coverage**: Short version flag parsing requirement. +**Context_Create_UnknownArgument_ThrowsArgumentException**: `Context.Create` is called with +`["--unknown"]`; an `ArgumentException` containing "Unsupported argument" is thrown. This +scenario is tested by `Context_Create_UnknownArgument_ThrowsArgumentException`. -### Context_Create_HelpFlag_SetsHelpTrue +**Context_Create_LogFlag_WithoutValue_ThrowsArgumentException**: `Context.Create` is called +with `["--log"]` (value missing); an `ArgumentException` is thrown. This scenario is tested by +`Context_Create_LogFlag_WithoutValue_ThrowsArgumentException`. -**Scenario**: `Context.Create` is called with `["--help"]`. +**Context_Create_ResultsFlag_WithoutValue_ThrowsArgumentException**: `Context.Create` is called +with `["--results"]` (value missing); an `ArgumentException` is thrown. This scenario is tested +by `Context_Create_ResultsFlag_WithoutValue_ThrowsArgumentException`. -**Expected**: `Help` property is true. +**Context_Create_ResultAliasFlag_SetsResultsFile**: `Context.Create` is called with +`["--result", "output.trx"]` (legacy alias); `ResultsFile` equals `"output.trx"`, identical to +the `--results` flag behavior. This scenario is tested by +`Context_Create_ResultAliasFlag_SetsResultsFile`. -**Requirement coverage**: Help flag (long form) parsing requirement. +**Context_Create_ResultAliasFlag_WithoutValue_ThrowsArgumentException**: `Context.Create` is +called with `["--result"]` (value missing); an `ArgumentException` is thrown. This scenario is +tested by `Context_Create_ResultAliasFlag_WithoutValue_ThrowsArgumentException`. -### Context_Create_ShortHelpFlag_H_SetsHelpTrue +**Context_Create_DepthFlag_SetsHeadingDepth**: `Context.Create` is called with +`["--depth", "3"]`; `HeadingDepth` equals 3. This scenario is tested by +`Context_Create_DepthFlag_SetsHeadingDepth`. -**Scenario**: `Context.Create` is called with `["-h"]`. +**Context_Create_NoDepthFlag_ReturnsDefaultHeadingDepth**: `Context.Create` is called with an +empty argument array; `HeadingDepth` equals 1 (the default; valid range is 1–6). This scenario +is tested by `Context_Create_NoDepthFlag_ReturnsDefaultHeadingDepth`. -**Expected**: `Help` property is true. +**Context_Create_DepthFlag_WithoutValue_ThrowsArgumentException**: `Context.Create` is called +with `["--depth"]` (value missing); an `ArgumentException` is thrown. This scenario is tested +by `Context_Create_DepthFlag_WithoutValue_ThrowsArgumentException`. -**Requirement coverage**: Help flag (-h) parsing requirement. +**Context_Create_DepthFlag_NonIntegerValue_ThrowsArgumentException**: `Context.Create` is +called with `["--depth", "abc"]`; an `ArgumentException` is thrown. This scenario is tested by +`Context_Create_DepthFlag_NonIntegerValue_ThrowsArgumentException`. -### Context_Create_ShortHelpFlag_Question_SetsHelpTrue +**Context_Create_DepthFlag_ZeroValue_ThrowsArgumentException**: `Context.Create` is called with +`["--depth", "0"]` (below the minimum of 1); an `ArgumentException` is thrown. This scenario +is tested by `Context_Create_DepthFlag_ZeroValue_ThrowsArgumentException`. -**Scenario**: `Context.Create` is called with `["-?"]`. +**Context_Create_DepthFlag_ExceedsMaxValue_ThrowsArgumentException**: `Context.Create` is +called with `["--depth", "7"]` (above the maximum of 6); an `ArgumentException` is thrown. +This scenario is tested by `Context_Create_DepthFlag_ExceedsMaxValue_ThrowsArgumentException`. -**Expected**: `Help` property is true. +**Context_WriteLine_NotSilent_WritesToConsole**: A non-silent `Context` calls `WriteLine` with +a test message; the message appears on standard output. This scenario is tested by +`Context_WriteLine_NotSilent_WritesToConsole`. -**Requirement coverage**: Help flag (-?) parsing requirement. +**Context_WriteLine_Silent_DoesNotWriteToConsole**: A silent `Context` (created with +`["--silent"]`) calls `WriteLine`; standard output receives nothing, confirming `--silent` +suppresses normal output. This scenario is tested by +`Context_WriteLine_Silent_DoesNotWriteToConsole`. + +**Context_WriteError_Silent_DoesNotWriteToConsole**: A silent `Context` calls `WriteError`; +standard error receives nothing, confirming `--silent` also suppresses error output. This +scenario is tested by `Context_WriteError_Silent_DoesNotWriteToConsole`. -### Context_Create_SilentFlag_SetsSilentTrue - -**Scenario**: `Context.Create` is called with `["--silent"]`. - -**Expected**: `Silent` property is true. - -**Requirement coverage**: Silent flag parsing requirement. - -### Context_Create_ValidateFlag_SetsValidateTrue - -**Scenario**: `Context.Create` is called with `["--validate"]`. - -**Expected**: `Validate` property is true. - -**Requirement coverage**: Validate flag parsing requirement. - -### Context_Create_ResultsFlag_SetsResultsFile - -**Scenario**: `Context.Create` is called with `["--results", "output.trx"]`. - -**Expected**: `ResultsFile` property equals `"output.trx"`. - -**Requirement coverage**: Results file path parsing requirement. - -### Context_Create_LogFlag_OpensLogFile - -**Scenario**: `Context.Create` is called with `["--log", ".log"]`; `WriteLine` is then called -with a test message. - -**Expected**: The log file is created; the test message is written to it. - -**Requirement coverage**: Log file opening and writing requirement. - -### Context_Create_UnknownArgument_ThrowsArgumentException - -**Scenario**: `Context.Create` is called with an unrecognized argument (e.g., `["--unknown"]`). - -**Expected**: An `ArgumentException` is thrown containing the text "Unsupported argument". - -**Requirement coverage**: Unknown argument rejection requirement. - -### Context_Create_LogFlag_WithoutValue_ThrowsArgumentException - -**Scenario**: `Context.Create` is called with `["--log"]` (value missing). - -**Expected**: An `ArgumentException` is thrown. - -**Requirement coverage**: Log flag missing-value validation requirement. - -### Context_Create_ResultsFlag_WithoutValue_ThrowsArgumentException - -**Scenario**: `Context.Create` is called with `["--results"]` (value missing). - -**Expected**: An `ArgumentException` is thrown. - -**Requirement coverage**: Results flag missing-value validation requirement. - -### Context_Create_ResultAliasFlag_SetsResultsFile - -**Scenario**: `Context.Create` is called with `["--result", "output.trx"]` (legacy alias). - -**Expected**: `ResultsFile` property equals `"output.trx"`, identical to the `--results` flag. - -**Requirement coverage**: Results alias flag parsing requirement. - -### Context_Create_ResultAliasFlag_WithoutValue_ThrowsArgumentException - -**Scenario**: `Context.Create` is called with `["--result"]` (value missing). - -**Expected**: An `ArgumentException` is thrown. - -**Requirement coverage**: Results alias flag missing-value validation requirement. - -### Context_Create_DepthFlag_SetsHeadingDepth - -**Scenario**: `Context.Create` is called with `["--depth", "3"]`. - -**Expected**: `HeadingDepth` property equals 3. - -**Requirement coverage**: Depth flag parsing requirement. - -### Context_Create_NoDepthFlag_ReturnsDefaultHeadingDepth - -**Scenario**: `Context.Create` is called with an empty argument array. - -**Expected**: `HeadingDepth` property equals 1 (the default; minimum is 1, maximum is 6). - -**Requirement coverage**: Default heading depth requirement. - -### Context_Create_DepthFlag_WithoutValue_ThrowsArgumentException - -**Scenario**: `Context.Create` is called with `["--depth"]` (value missing). - -**Expected**: An `ArgumentException` is thrown. - -**Requirement coverage**: Depth flag missing-value validation requirement. - -### Context_Create_DepthFlag_NonIntegerValue_ThrowsArgumentException - -**Scenario**: `Context.Create` is called with `["--depth", "abc"]`. - -**Expected**: An `ArgumentException` is thrown. - -**Requirement coverage**: Depth flag non-integer validation requirement. - -### Context_Create_DepthFlag_ZeroValue_ThrowsArgumentException - -**Scenario**: `Context.Create` is called with `["--depth", "0"]` (below minimum of 1). - -**Expected**: An `ArgumentException` is thrown. - -**Requirement coverage**: Depth flag minimum-value validation requirement (minimum is 1). - -### Context_Create_DepthFlag_ExceedsMaxValue_ThrowsArgumentException - -**Scenario**: `Context.Create` is called with `["--depth", "7"]` (above the maximum of 6). - -**Expected**: An `ArgumentException` is thrown. - -**Requirement coverage**: Depth flag maximum-value validation requirement (maximum is 6). - -### Context_WriteLine_NotSilent_WritesToConsole - -**Scenario**: A non-silent `Context` is created and `WriteLine` is called with a test message. - -**Expected**: The test message appears on standard output. - -**Requirement coverage**: Normal output writing requirement. - -### Context_WriteLine_Silent_DoesNotWriteToConsole - -**Scenario**: A silent `Context` (created with `["--silent"]`) calls `WriteLine`. - -**Expected**: Standard output receives nothing. - -**Requirement coverage**: Silent mode suppression requirement. - -### Context_WriteError_Silent_DoesNotWriteToConsole - -**Scenario**: A silent `Context` calls `WriteError`. - -**Expected**: Standard error receives nothing. - -**Requirement coverage**: Silent mode error suppression requirement. - -### Context_WriteError_SetsErrorExitCode - -**Scenario**: A `Context` calls `WriteError`. - -**Expected**: `ExitCode` is 1 after the call. - -**Requirement coverage**: Error exit code setting requirement. - -### Context_WriteError_NotSilent_WritesToConsole - -**Scenario**: A non-silent `Context` calls `WriteError` with a test message. - -**Expected**: The test message appears on standard error. - -**Requirement coverage**: Error output writing requirement. - -### Context_WriteError_WritesToLogFile - -**Scenario**: A `Context` created with `["--silent", "--log", ".log"]` calls `WriteError` -with a test message. - -**Expected**: The test message appears in the log file. - -**Requirement coverage**: Error log writing requirement. - -## Requirements Coverage - -| Requirement | Test Scenario | -|--------------------------------|---------------------------------------------------------------------| -| Default context creation | Context_Create_NoArguments_ReturnsDefaultContext | -| --version flag parsing | Context_Create_VersionFlag_SetsVersionTrue | -| -v flag parsing | Context_Create_ShortVersionFlag_SetsVersionTrue | -| --help flag parsing | Context_Create_HelpFlag_SetsHelpTrue | -| -h flag parsing | Context_Create_ShortHelpFlag_H_SetsHelpTrue | -| -? flag parsing | Context_Create_ShortHelpFlag_Question_SetsHelpTrue | -| --silent flag parsing | Context_Create_SilentFlag_SetsSilentTrue | -| --validate flag parsing | Context_Create_ValidateFlag_SetsValidateTrue | -| --results flag parsing | Context_Create_ResultsFlag_SetsResultsFile | -| --log flag and file writing | Context_Create_LogFlag_OpensLogFile | -| Unknown argument rejection | Context_Create_UnknownArgument_ThrowsArgumentException | -| --log missing value | Context_Create_LogFlag_WithoutValue_ThrowsArgumentException | -| --results missing value | Context_Create_ResultsFlag_WithoutValue_ThrowsArgumentException | -| --result alias parsing | Context_Create_ResultAliasFlag_SetsResultsFile | -| --result alias missing value | Context_Create_ResultAliasFlag_WithoutValue_ThrowsArgumentException | -| --depth flag parsing | Context_Create_DepthFlag_SetsHeadingDepth | -| Default heading depth | Context_Create_NoDepthFlag_ReturnsDefaultHeadingDepth | -| --depth missing value | Context_Create_DepthFlag_WithoutValue_ThrowsArgumentException | -| --depth non-integer value | Context_Create_DepthFlag_NonIntegerValue_ThrowsArgumentException | -| --depth zero value (min 1) | Context_Create_DepthFlag_ZeroValue_ThrowsArgumentException | -| --depth exceeds maximum (max 6)| Context_Create_DepthFlag_ExceedsMaxValue_ThrowsArgumentException | -| Normal output writing | Context_WriteLine_NotSilent_WritesToConsole | -| Silent mode output suppression | Context_WriteLine_Silent_DoesNotWriteToConsole | -| Silent mode error suppression | Context_WriteError_Silent_DoesNotWriteToConsole | -| Error exit code | Context_WriteError_SetsErrorExitCode | -| Error output to stderr | Context_WriteError_NotSilent_WritesToConsole | -| Error writing to log file | Context_WriteError_WritesToLogFile | +**Context_WriteError_SetsErrorExitCode**: A `Context` calls `WriteError`; `ExitCode` is 1 +after the call. This scenario is tested by `Context_WriteError_SetsErrorExitCode`. + +**Context_WriteError_NotSilent_WritesToConsole**: A non-silent `Context` calls `WriteError` +with a test message; the message appears on standard error. This scenario is tested by +`Context_WriteError_NotSilent_WritesToConsole`. + +**Context_WriteError_WritesToLogFile**: A `Context` created with +`["--silent", "--log", ".log"]` calls `WriteError` with a test message; the message appears +in the log file, confirming errors are always written to the log regardless of `--silent`. This +scenario is tested by `Context_WriteError_WritesToLogFile`. + +**Context_Create_LogFlag_InvalidPath_ThrowsInvalidOperationException**: +`Context.Create` is called with `["--log", "/invalid/\x00path.log"]`; an +`InvalidOperationException` is thrown because the log file cannot be opened. This +scenario is tested by `Context_Create_LogFlag_InvalidPath_ThrowsInvalidOperationException`. diff --git a/docs/verification/template-dot-net-tool/program.md b/docs/verification/template-dot-net-tool/program.md index 611bec2..e1e4d07 100644 --- a/docs/verification/template-dot-net-tool/program.md +++ b/docs/verification/template-dot-net-tool/program.md @@ -1,107 +1,60 @@ -# Program Verification +## Program -This document describes the unit-level verification design for the `Program` unit. It defines the -test scenarios, dependency usage, and requirement coverage for `Program.cs`. - -## Verification Approach +### Verification Approach `Program` is verified with unit tests defined in `ProgramTests.cs`. Because `Program` directly -instantiates `Context` from real arguments and calls `Validation.Run` when needed, no mocking is -required. The tests pass controlled argument arrays and assert on captured console output and exit -codes. - -## Dependencies - -| Dependency | Usage in Tests | -|--------------|--------------------------------------------------------------------------| -| `Context` | Used directly (not mocked) — created from the argument array under test. | -| `Validation` | Used directly (not mocked) — called when the validate flag is set. | - -No test doubles are introduced at the `Program` level; all collaborators execute their real logic. - -## Test Scenarios - -### Program_Run_WithVersionFlag_DisplaysVersionOnly - -**Scenario**: `Program.Run` is called with a context created from `["--version"]`. - -**Expected**: Standard output contains the version string; the word "Copyright" does not appear; -the banner prefix "Template DotNet Tool version" does not appear; exit code is 0. - -**Requirement coverage**: `Template-Program-Version`, `Template-Program-ExitCode`. - -### Program_Run_WithHelpFlag_DisplaysUsageInformation - -**Scenario**: `Program.Run` is called with a context created from `["--help"]`. - -**Expected**: Standard output contains "Usage:", "Options:", "--version", and "--help"; exit code -is 0. - -**Requirement coverage**: `Template-Program-Help`, `Template-Program-ExitCode`. - -### Program_Run_WithValidateFlag_RunsValidation - -**Scenario**: `Program.Run` is called with a context created from `["--validate"]`. - -**Expected**: Standard output contains "Total Tests:"; exit code is 0. - -**Requirement coverage**: `Template-Program-Validate`, `Template-Program-ExitCode`. - -### Program_Run_NoArguments_DisplaysDefaultBehavior - -**Scenario**: `Program.Run` is called with a context created from an empty argument array. - -**Expected**: Standard output contains the tool name and copyright notice; exit code is 0. - -**Requirement coverage**: `Template-Program-ExitCode`. - -### Program_Version_ReturnsNonEmptyString - -**Scenario**: The `Program.Version` static property is read. - -**Expected**: The returned string is non-empty and non-null. - -**Requirement coverage**: `Template-Program-Version`. - -### Program_Main_WithInvalidArgs_ReturnsNonZeroExitCode - -**Scenario**: `Program.Main` is invoked with `["--invalid-argument"]`. - -**Expected**: Exit code is 1. - -**Requirement coverage**: `Template-Program-ExitCode`. +instantiates `Context` from real arguments and calls `Validation.Run` when needed, no mocking +is required. Tests pass controlled argument arrays and assert on captured console output and +exit codes. -### Program_Run_WithShortVersionFlag_DisplaysVersion +### Test Environment -**Scenario**: `Program.Run` is called with a context created from `["-v"]`. +N/A - standard test environment. -**Expected**: Standard output contains the version string; exit code is 0. +### Acceptance Criteria -**Requirement coverage**: `Template-Program-Version`. +- All unit tests pass with zero failures. +- Exit code 0 is returned for all valid flag combinations. +- Exit code 1 is returned when invalid arguments are supplied to `Program.Main`. +- `Program.Version` returns a non-empty, non-null string. -### Program_Run_WithShortHelpFlag_DisplaysUsage +### Test Scenarios -**Scenario**: `Program.Run` is called with a context created from `["-h"]`. +**Program_Run_WithVersionFlag_DisplaysVersionOnly**: `Program.Run` is called with a context +created from `["--version"]`; the output contains the version string, "Copyright" does not +appear, the banner prefix does not appear, and exit code is 0, confirming version-only output +with no banner. This scenario is tested by `Program_Run_WithVersionFlag_DisplaysVersionOnly`. -**Expected**: Standard output contains "Usage:" and "Options:"; exit code is 0. +**Program_Run_WithHelpFlag_DisplaysUsageInformation**: `Program.Run` is called with a context +from `["--help"]`; the output contains "Usage:", "Options:", "--version", and "--help" and exit +code is 0. This scenario is tested by `Program_Run_WithHelpFlag_DisplaysUsageInformation`. -**Requirement coverage**: `Template-Program-Help`. +**Program_Run_WithValidateFlag_RunsValidation**: `Program.Run` is called with a context from +`["--validate"]`; the output contains "Total Tests:" and exit code is 0, confirming the +self-validation suite is invoked and completes successfully. This scenario is tested by +`Program_Run_WithValidateFlag_RunsValidation`. -### Program_Run_WithQuestionMarkFlag_DisplaysUsage +**Program_Run_NoArguments_DisplaysDefaultBehavior**: `Program.Run` is called with an empty +argument array; the output contains the tool name and copyright notice and exit code is 0. +This scenario is tested by `Program_Run_NoArguments_DisplaysDefaultBehavior`. -**Scenario**: `Program.Run` is called with a context created from `["-?"]`. +**Program_Version_ReturnsNonEmptyString**: The `Program.Version` static property is read; the +returned string is non-empty and non-null, confirming the version is resolvable from the +assembly attributes. This scenario is tested by `Program_Version_ReturnsNonEmptyString`. -**Expected**: Standard output contains "Usage:" and "Options:"; exit code is 0. +**Program_Main_WithInvalidArgs_ReturnsNonZeroExitCode**: `Program.Main` is invoked with +`["--invalid-argument"]`; the exit code is 1, confirming that invalid arguments are caught and +handled without re-throwing. This scenario is tested by +`Program_Main_WithInvalidArgs_ReturnsNonZeroExitCode`. -**Requirement coverage**: `Template-Program-Help`. +**Program_Run_WithShortVersionFlag_DisplaysVersion**: `Program.Run` is called with a context +from `["-v"]`; the output contains the version string and exit code is 0. This scenario is +tested by `Program_Run_WithShortVersionFlag_DisplaysVersion`. -## Requirements Coverage +**Program_Run_WithShortHelpFlag_DisplaysUsage**: `Program.Run` is called with a context from +`["-h"]`; the output contains "Usage:" and "Options:" and exit code is 0. This scenario is +tested by `Program_Run_WithShortHelpFlag_DisplaysUsage`. -- **`Template-Program-Version`**: Program_Run_WithVersionFlag_DisplaysVersionOnly, - Program_Run_WithShortVersionFlag_DisplaysVersion, - Program_Version_ReturnsNonEmptyString -- **`Template-Program-Help`**: Program_Run_WithHelpFlag_DisplaysUsageInformation, - Program_Run_WithShortHelpFlag_DisplaysUsage, - Program_Run_WithQuestionMarkFlag_DisplaysUsage -- **`Template-Program-Validate`**: Program_Run_WithValidateFlag_RunsValidation -- **`Template-Program-ExitCode`**: Program_Main_WithInvalidArgs_ReturnsNonZeroExitCode +**Program_Run_WithQuestionMarkFlag_DisplaysUsage**: `Program.Run` is called with a context from +`["-?"]`; the output contains "Usage:" and "Options:" and exit code is 0. This scenario is +tested by `Program_Run_WithQuestionMarkFlag_DisplaysUsage`. diff --git a/docs/verification/template-dot-net-tool/self-test.md b/docs/verification/template-dot-net-tool/self-test.md new file mode 100644 index 0000000..5916194 --- /dev/null +++ b/docs/verification/template-dot-net-tool/self-test.md @@ -0,0 +1,51 @@ +## SelfTest + +### Verification Approach + +The `SelfTest` subsystem is verified by integration tests defined in +`SelfTestSubsystemTests.cs`. Each test exercises `Validation.Run` with a real `Context` to +confirm that the subsystem produces correct output and result files across the supported +result-format options. No mocking is applied; `Context`, `Program`, and `PathHelpers` all +execute their real implementations. Temporary directories are used for result file output to +keep tests isolated and leave no permanent file-system side-effects. + +### Test Environment + +N/A - standard test environment. + +### Acceptance Criteria + +- All integration tests pass with zero failures. +- `Validation.Run` completes successfully with exit code 0 when no result file is requested. +- TRX and JUnit XML result files are generated with the correct XML root elements when valid + file paths are supplied. +- An unsupported result file extension produces exit code 1 and no file is created. + +### Test Scenarios + +**SelfTestSubsystem_ValidationWorkflow_NoResultFiles_CompletesSuccessfully**: +`Validation.Run` is called with a context that does not specify any results file; validation +completes without error, exit code is 0, and the validate flag is set. This scenario is tested +by `SelfTestSubsystem_ValidationWorkflow_NoResultFiles_CompletesSuccessfully`. + +**SelfTestSubsystem_ValidationWorkflow_WithTrxFile_GeneratesResults**: `Validation.Run` is +called with a context whose `ResultsFile` points to a temporary `.trx` path; a TRX file +containing the `.trx` pointing to a -temporary file. - -**Expected**: Exit code 0; a TRX file is created at the specified path; the file contains a -`.xml` pointing to a -temporary file. - -**Expected**: Exit code 0; an XML file is created at the specified path; the file contains a -`` flag is passed pointing to a temporary file. - -**Expected**: Exit code 0; the specified log file is created and contains output that also appears -on standard output. - -### TemplateDotNetTool_UnknownArgument_Provided_ReturnsError - -**Scenario**: An unrecognized argument (e.g., `--unknown`) is passed. - -**Expected**: Exit code non-zero; combined output contains an error message indicating the -unrecognized argument. - -### TemplateDotNetTool_ValidateWithDepth_DepthThree_OutputsCorrectHeadingLevel - -**Scenario**: The `--validate` flag is combined with `--depth 3`. - -**Expected**: Exit code 0; combined output contains `###` (heading at depth 3). - -### TemplateDotNetTool_NoArguments_Invoked_DisplaysBanner - -**Scenario**: The tool is invoked with no arguments. - -**Expected**: Exit code 0; combined output contains the tool name and copyright notice. - -### TemplateDotNetTool_ResultAlias_LegacyFlag_WritesResultsFile - -**Scenario**: The `--validate` flag is combined with `--result .trx` (the legacy alias). - -**Expected**: Exit code 0; a TRX file is created at the specified path. - -### TemplateDotNetTool_ValidateWithBadExtension_ExtensionInvalid_ReturnsNonZero - -**Scenario**: The `--validate` flag is combined with `--results .bad` (unsupported extension). - -**Expected**: Exit code non-zero; no file is created at the specified path. - -## Requirements Coverage - -The following list maps each system-level requirement category to the integration test scenarios -that verify it. - -- **Version display**: TemplateDotNetTool_VersionFlag_Provided_OutputsVersion -- **Help display**: TemplateDotNetTool_HelpFlag_Provided_OutputsUsageInformation -- **Self-validation**: TemplateDotNetTool_ValidateFlag_Provided_RunsValidation -- **TRX results output**: TemplateDotNetTool_ValidateWithTrxResults_Requested_GeneratesTrxFile -- **JUnit results output**: TemplateDotNetTool_ValidateWithXmlResults_Requested_GeneratesJUnitFile -- **Silent mode**: TemplateDotNetTool_SilentFlag_Provided_SuppressesOutput -- **Log file output**: TemplateDotNetTool_LogFlag_Provided_WritesOutputToFile -- **Invalid argument rejection**: TemplateDotNetTool_UnknownArgument_Provided_ReturnsError -- **`Template-System-Depth`**: TemplateDotNetTool_ValidateFlag_Provided_RunsValidation, - TemplateDotNetTool_ValidateWithDepth_DepthThree_OutputsCorrectHeadingLevel -- **`Template-System-ValidateFailure`**: - TemplateDotNetTool_ValidateWithBadExtension_ExtensionInvalid_ReturnsNonZero -- **`Template-System-DefaultBehavior`**: TemplateDotNetTool_NoArguments_Invoked_DisplaysBanner -- **`Template-System-ResultAlias`**: TemplateDotNetTool_ResultAlias_LegacyFlag_WritesResultsFile diff --git a/docs/verification/template-dot-net-tool/utilities.md b/docs/verification/template-dot-net-tool/utilities.md new file mode 100644 index 0000000..d1fde70 --- /dev/null +++ b/docs/verification/template-dot-net-tool/utilities.md @@ -0,0 +1,47 @@ +## Utilities + +### Verification Approach + +The `Utilities` subsystem is verified by integration tests defined in +`UtilitiesSubsystemTests.cs`. Each test exercises `PathHelpers` through realistic +path-combination workflows to confirm that valid paths are resolved correctly, traversal attacks +are rejected, and the resulting paths can be used for actual directory creation. `PathHelpers` +has no dependencies on other tool units, so no mocking is required. + +### Test Environment + +N/A - standard test environment. + +### Acceptance Criteria + +- All integration tests pass with zero failures. +- Valid relative paths are combined correctly with no exception. +- Path traversal patterns (e.g., `../`) cause `ArgumentException` to be thrown. +- Absolute paths supplied as the relative argument cause `ArgumentException` to be thrown. +- Paths produced by `SafePathCombine` can be passed directly to `Directory.CreateDirectory`. + +### Test Scenarios + +**UtilitiesSubsystem_PathResolutionWorkflow_ValidPaths_ResolvesCorrectly**: Multiple valid +relative path arguments are combined with a base path using `PathHelpers.SafePathCombine`; all +results are correctly combined paths that remain within the base directory and no exception is +thrown. This scenario is tested by +`UtilitiesSubsystem_PathResolutionWorkflow_ValidPaths_ResolvesCorrectly`. + +**UtilitiesSubsystem_PathTraversalValidation_DangerousPaths_ThrowsException**: Multiple path +traversal patterns (e.g., `../`, `subfolder/../../../`) are passed to `SafePathCombine`; an +`ArgumentException` is thrown for each traversal pattern and no traversal succeeds. This +scenario is tested by +`UtilitiesSubsystem_PathTraversalValidation_DangerousPaths_ThrowsException`. + +**UtilitiesSubsystem_AbsolutePathRejection_ThrowsException**: Absolute paths (e.g., +`/etc/passwd` and on Windows `C:\Windows\System32\file.txt`) are passed as the relative path +argument to `SafePathCombine`; an `ArgumentException` is thrown for each and no injection +succeeds. This scenario is tested by +`UtilitiesSubsystem_AbsolutePathRejection_ThrowsException`. + +**UtilitiesSubsystem_DirectoryCreationWorkflow_ValidPaths_CreatesDirectories**: +`PathHelpers.SafePathCombine` is used to compute a nested path, and the resulting path is +passed to `Directory.CreateDirectory`; the directory is created at the expected location within +the base directory. This scenario is tested by +`UtilitiesSubsystem_DirectoryCreationWorkflow_ValidPaths_CreatesDirectories`. diff --git a/docs/verification/template-dot-net-tool/utilities/path-helpers.md b/docs/verification/template-dot-net-tool/utilities/path-helpers.md index 181e202..1a5360b 100644 --- a/docs/verification/template-dot-net-tool/utilities/path-helpers.md +++ b/docs/verification/template-dot-net-tool/utilities/path-helpers.md @@ -1,135 +1,76 @@ -# PathHelpers Verification +### PathHelpers -This document describes the unit-level verification design for the `PathHelpers` unit. It defines -the test scenarios, dependency usage, and requirement coverage for `Utilities/PathHelpers.cs`. - -## Verification Approach +#### Verification Approach `PathHelpers` is verified with unit tests defined in `PathHelpersTests.cs`. Because `PathHelpers` -performs pure path manipulation using only .NET BCL types, no mocking or test doubles are needed. -Tests call `PathHelpers.SafePathCombine` directly with controlled base and relative path arguments -and assert on the returned string or the thrown exception. - -## Dependencies - -`PathHelpers` has no dependencies on other tool units. All path operations use .NET BCL types -(`Path`, `string`); no mocking is needed at this level. - -## Test Scenarios - -### PathHelpers_SafePathCombine_ValidPaths_CombinesCorrectly - -**Scenario**: A relative path (e.g., `"subfolder/file.txt"`) is combined with a base path. - -**Expected**: The returned path equals the expected combined result; no exception is thrown. - -**Requirement coverage**: `Template-PathHelpers-SafeCombine` (valid path combination). - -### PathHelpers_SafePathCombine_PathTraversalWithDoubleDots_ThrowsArgumentException - -**Scenario**: A relative path starting with `"../"` is passed to `SafePathCombine`. - -**Expected**: An `ArgumentException` is thrown containing the text "Invalid path component". - -**Boundary / error path**: Directory traversal attempt via leading `../`. - -**Requirement coverage**: `Template-PathHelpers-SafeCombine` (traversal rejection). - -### PathHelpers_SafePathCombine_DoubleDotsInMiddle_ThrowsArgumentException - -**Scenario**: A relative path containing `"subfolder/../../../etc/passwd"` is passed to -`SafePathCombine`. - -**Expected**: An `ArgumentException` is thrown. - -**Boundary / error path**: Directory traversal attempt via embedded `../` sequence. - -**Requirement coverage**: `Template-PathHelpers-SafeCombine` (embedded traversal rejection). - -### PathHelpers_SafePathCombine_AbsolutePath_ThrowsArgumentException - -**Scenario**: An absolute path is passed as the relative argument to `SafePathCombine`. -Sub-cases: - -- Unix-style: `"/etc/passwd"` (tested on all platforms). -- Windows-style: `"C:\Windows\System32\file.txt"` (tested only when `OperatingSystem.IsWindows()` is true). - -**Expected**: An `ArgumentException` is thrown for each sub-case. - -**Boundary / error path**: Absolute path used where a relative path is required. - -**Requirement coverage**: `Template-PathHelpers-SafeCombine` (absolute path rejection). - -### PathHelpers_SafePathCombine_CurrentDirectoryReference_CombinesCorrectly - -**Scenario**: A relative path starting with `"./"` (e.g., `"./subfolder/file.txt"`) is combined -with a base path. - -**Expected**: The returned path equals the expected combined result; no exception is thrown. - -**Requirement coverage**: `Template-PathHelpers-SafeCombine` (current-directory prefix). - -### PathHelpers_SafePathCombine_NestedPaths_CombinesCorrectly - -**Scenario**: A deeply nested relative path (e.g., `"a/b/c/d/file.txt"`) is combined with a base -path. - -**Expected**: The returned path equals the expected combined result; no exception is thrown. - -**Requirement coverage**: `Template-PathHelpers-SafeCombine` (nested path combination). - -### PathHelpers_SafePathCombine_EmptyRelativePath_ReturnsBasePath - -**Scenario**: An empty string is passed as the relative path argument. - -**Expected**: The returned path equals the base path; no exception is thrown. - -**Boundary / error path**: Empty relative path edge case. - -**Requirement coverage**: `Template-PathHelpers-SafeCombine` (empty relative path). - -### PathHelpers_SafePathCombine_DotDotPrefixedName_CombinesCorrectly - -**Scenario**: A relative path whose filename starts with `".."` but is not a traversal sequence -(e.g., `"..data/file.txt"`) is combined with a base path. - -**Expected**: The returned path equals the expected combined result; no exception is thrown. - -**Boundary / error path**: Filename beginning with `".."` must not be misidentified as a traversal. - -**Requirement coverage**: `Template-PathHelpers-SafeCombine` (dot-dot-prefixed filename). - -### PathHelpers_SafePathCombine_NullBasePath_ThrowsArgumentNullException - -**Scenario**: `null` is passed as the `basePath` argument to `SafePathCombine`. - -**Expected**: An `ArgumentNullException` is thrown. - -**Boundary / error path**: Null guard on `basePath`. - -**Requirement coverage**: `Template-PathHelpers-SafeCombine` (null input rejection). - -### PathHelpers_SafePathCombine_NullRelativePath_ThrowsArgumentNullException - -**Scenario**: `null` is passed as the `relativePath` argument to `SafePathCombine`. - -**Expected**: An `ArgumentNullException` is thrown. - -**Boundary / error path**: Null guard on `relativePath`. - -**Requirement coverage**: `Template-PathHelpers-SafeCombine` (null input rejection). - -## Requirements Coverage - -The `Template-PathHelpers-SafeCombine` requirement is verified by the following test scenarios: - -- (valid path combination): PathHelpers_SafePathCombine_ValidPaths_CombinesCorrectly -- (leading traversal rejection): PathHelpers_SafePathCombine_PathTraversalWithDoubleDots_ThrowsArgumentException -- (embedded traversal rejection): PathHelpers_SafePathCombine_DoubleDotsInMiddle_ThrowsArgumentException -- (absolute path rejection): PathHelpers_SafePathCombine_AbsolutePath_ThrowsArgumentException -- (current-directory prefix): PathHelpers_SafePathCombine_CurrentDirectoryReference_CombinesCorrectly -- (nested path combination): PathHelpers_SafePathCombine_NestedPaths_CombinesCorrectly -- (empty relative path): PathHelpers_SafePathCombine_EmptyRelativePath_ReturnsBasePath -- (dot-dot filename, not traversal): PathHelpers_SafePathCombine_DotDotPrefixedName_CombinesCorrectly -- (null basePath rejection): PathHelpers_SafePathCombine_NullBasePath_ThrowsArgumentNullException -- (null relativePath rejection): PathHelpers_SafePathCombine_NullRelativePath_ThrowsArgumentNullException +performs pure path manipulation using only .NET BCL types, no mocking or test doubles are +required. Tests call `PathHelpers.SafePathCombine` directly with controlled base and relative +path arguments and assert on the returned string or the thrown exception type and message. + +#### Test Environment + +N/A - standard test environment. + +#### Acceptance Criteria + +- All unit tests pass with zero failures. +- Valid relative paths are combined correctly and the returned path equals the expected result. +- Path traversal patterns cause `ArgumentException` with message containing "Invalid path + component". +- Absolute paths supplied as the relative argument cause `ArgumentException`. +- Null inputs cause `ArgumentNullException`. +- A filename beginning with `".."` that is not a traversal sequence is accepted correctly. + +#### Test Scenarios + +**PathHelpers_SafePathCombine_ValidPaths_CombinesCorrectly**: A relative path +(`"subfolder/file.txt"`) is combined with a base path; the returned path equals the expected +combined result and no exception is thrown, confirming basic path combination works. This +scenario is tested by `PathHelpers_SafePathCombine_ValidPaths_CombinesCorrectly`. + +**PathHelpers_SafePathCombine_PathTraversalWithDoubleDots_ThrowsArgumentException**: A relative +path starting with `"../"` is passed; an `ArgumentException` containing "Invalid path +component" is thrown, rejecting the leading traversal attempt. This scenario is tested by +`PathHelpers_SafePathCombine_PathTraversalWithDoubleDots_ThrowsArgumentException`. + +**PathHelpers_SafePathCombine_DoubleDotsInMiddle_ThrowsArgumentException**: A relative path +containing `"subfolder/../../../etc/passwd"` is passed; an `ArgumentException` is thrown, +rejecting the embedded traversal sequence. This scenario is tested by +`PathHelpers_SafePathCombine_DoubleDotsInMiddle_ThrowsArgumentException`. + +**PathHelpers_SafePathCombine_AbsolutePath_ThrowsArgumentException**: An absolute path is +passed as the relative argument — Unix-style `/etc/passwd` on all platforms and Windows-style +`C:\Windows\System32\file.txt` when running on Windows; an `ArgumentException` is thrown for +each sub-case. This scenario is tested by +`PathHelpers_SafePathCombine_AbsolutePath_ThrowsArgumentException`. + +**PathHelpers_SafePathCombine_CurrentDirectoryReference_CombinesCorrectly**: A relative path +starting with `"./"` (e.g., `"./subfolder/file.txt"`) is combined with a base path; the +returned path equals the expected combined result and no exception is thrown. This scenario is +tested by `PathHelpers_SafePathCombine_CurrentDirectoryReference_CombinesCorrectly`. + +**PathHelpers_SafePathCombine_NestedPaths_CombinesCorrectly**: A deeply nested relative path +(`"a/b/c/d/file.txt"`) is combined with a base path; the returned path equals the expected +combined result and no exception is thrown. This scenario is tested by +`PathHelpers_SafePathCombine_NestedPaths_CombinesCorrectly`. + +**PathHelpers_SafePathCombine_EmptyRelativePath_ReturnsBasePath**: An empty string is passed as +the relative path; the returned path equals the base path and no exception is thrown, +confirming the empty-string edge case is handled correctly. This scenario is tested by +`PathHelpers_SafePathCombine_EmptyRelativePath_ReturnsBasePath`. + +**PathHelpers_SafePathCombine_DotDotPrefixedName_CombinesCorrectly**: A relative path whose +filename starts with `".."` but is not a traversal sequence (e.g., `"..data/file.txt"`) is +combined with a base path; the returned path equals the expected combined result and no +exception is thrown, confirming such filenames are not misidentified as traversal. This scenario +is tested by `PathHelpers_SafePathCombine_DotDotPrefixedName_CombinesCorrectly`. + +**PathHelpers_SafePathCombine_NullBasePath_ThrowsArgumentNullException**: `null` is passed as +the `basePath` argument; an `ArgumentNullException` is thrown, confirming the null guard on +`basePath`. This scenario is tested by +`PathHelpers_SafePathCombine_NullBasePath_ThrowsArgumentNullException`. + +**PathHelpers_SafePathCombine_NullRelativePath_ThrowsArgumentNullException**: `null` is passed +as the `relativePath` argument; an `ArgumentNullException` is thrown, confirming the null guard +on `relativePath`. This scenario is tested by +`PathHelpers_SafePathCombine_NullRelativePath_ThrowsArgumentNullException`. diff --git a/docs/verification/template-dot-net-tool/utilities/utilities.md b/docs/verification/template-dot-net-tool/utilities/utilities.md deleted file mode 100644 index 70fce29..0000000 --- a/docs/verification/template-dot-net-tool/utilities/utilities.md +++ /dev/null @@ -1,60 +0,0 @@ -# Utilities Subsystem Verification - -This document describes the subsystem-level verification design for the `Utilities` subsystem. It -defines the integration test approach, subsystem boundary, mocking strategy, and test scenarios -that together verify the `Utilities` subsystem requirements. - -## Verification Approach - -The `Utilities` subsystem is verified by integration tests defined in `UtilitiesSubsystemTests.cs`. -Each test exercises the `PathHelpers` unit through realistic path-combination workflows, confirming -that valid paths are resolved correctly, traversal attacks are rejected, and the resulting paths -can be used for actual directory creation. - -## Dependencies and Mocking Strategy - -`PathHelpers` depends only on .NET BCL types for path manipulation. No external dependencies -require mocking at the subsystem level. Temporary directories are used when directory-creation -scenarios require a writable base path. - -## Integration Test Scenarios - -The following integration test scenarios are defined in `UtilitiesSubsystemTests.cs`. - -### UtilitiesSubsystem_PathResolutionWorkflow_ValidPaths_ResolvesCorrectly - -**Scenario**: Multiple valid relative path arguments are combined with a base path using -`PathHelpers.SafePathCombine`. - -**Expected**: All results are correctly combined paths that remain within the base directory; -no exception is thrown. - -### UtilitiesSubsystem_PathTraversalValidation_DangerousPaths_ThrowsException - -**Scenario**: Multiple path traversal patterns (e.g., `../`, `subfolder/../../../`) are passed to -`PathHelpers.SafePathCombine`. - -**Expected**: An `ArgumentException` is thrown for each traversal pattern; no traversal succeeds. - -### UtilitiesSubsystem_AbsolutePathRejection_ThrowsException - -**Scenario**: Absolute paths (e.g., `/etc/passwd` and on Windows `C:\Windows\System32\file.txt`) -are passed as the relative path argument to `PathHelpers.SafePathCombine`. - -**Expected**: An `ArgumentException` is thrown for each absolute path; no injection succeeds. - -### UtilitiesSubsystem_DirectoryCreationWorkflow_ValidPaths_CreatesDirectories - -**Scenario**: `PathHelpers.SafePathCombine` is used to compute a nested path, and the resulting -path is passed to `Directory.CreateDirectory`. - -**Expected**: The directory is created at the expected location within the base directory. - -## Requirements Coverage - -The `Template-Utilities-SafePaths` requirement is verified by the following test scenarios: - -- (valid path resolution): UtilitiesSubsystem_PathResolutionWorkflow_ValidPaths_ResolvesCorrectly -- (traversal rejection): UtilitiesSubsystem_PathTraversalValidation_DangerousPaths_ThrowsException -- (absolute path rejection): UtilitiesSubsystem_AbsolutePathRejection_ThrowsException -- (directory creation): UtilitiesSubsystem_DirectoryCreationWorkflow_ValidPaths_CreatesDirectories diff --git a/requirements.yaml b/requirements.yaml index c6350a0..ffa543e 100644 --- a/requirements.yaml +++ b/requirements.yaml @@ -1,13 +1,13 @@ --- # Root requirements file - includes all system, unit, platform, and OTS requirements includes: - - docs/reqstream/template-dot-net-tool/template-dot-net-tool.yaml + - docs/reqstream/template-dot-net-tool.yaml - docs/reqstream/template-dot-net-tool/program.yaml - - docs/reqstream/template-dot-net-tool/cli/cli.yaml + - docs/reqstream/template-dot-net-tool/cli.yaml - docs/reqstream/template-dot-net-tool/cli/context.yaml - - docs/reqstream/template-dot-net-tool/self-test/self-test.yaml + - docs/reqstream/template-dot-net-tool/self-test.yaml - docs/reqstream/template-dot-net-tool/self-test/validation.yaml - - docs/reqstream/template-dot-net-tool/utilities/utilities.yaml + - docs/reqstream/template-dot-net-tool/utilities.yaml - docs/reqstream/template-dot-net-tool/utilities/path-helpers.yaml - docs/reqstream/template-dot-net-tool/platform-requirements.yaml - docs/reqstream/ots/xunit.yaml