Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .reviewmark.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ reviews:
- "docs/reqstream/file-assert/selftest/validation.yaml" # requirements
- "docs/design/file-assert/selftest/validation.md" # design
- "docs/verification/file-assert/selftest/validation.md" # verification
- "src/**/SelfTest/Validation.cs" # implementation
- "src/**/SelfTest/Validation*.cs" # implementation
- "test/**/SelfTest/ValidationTests.cs" # unit tests

# FileAssert-Utilities-PathHelpers Review (one per unit)
Expand Down
26 changes: 19 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -279,11 +279,17 @@ Running self-validation produces a report demonstrating that FileAssert is funct
✓ FileAssert_VersionDisplay - Passed
✓ FileAssert_HelpDisplay - Passed
✓ FileAssert_Results - Passed
✓ FileAssert_Exists - Passed
✓ FileAssert_Contains - Passed

Total Tests: 5
Passed: 5
✓ FileAssert_File - Passed
✓ FileAssert_Text - Passed
✓ FileAssert_Html - Passed
✓ FileAssert_Xml - Passed
✓ FileAssert_Yaml - Passed
✓ FileAssert_Json - Passed
✓ FileAssert_Pdf - Passed
✓ FileAssert_Zip - Passed

Total Tests: 11
Passed: 11
Failed: 0
```

Expand All @@ -292,8 +298,14 @@ Each test in the report proves:
- **`FileAssert_VersionDisplay`** - `--version` outputs a valid version string.
- **`FileAssert_HelpDisplay`** - `--help` outputs usage and options information.
- **`FileAssert_Results`** - `--results` generates test results with pass and fail outcomes.
- **`FileAssert_Exists`** - `--config` verifies file-existence via glob pattern.
- **`FileAssert_Contains`** - `--config` verifies file content with contains assertions.
- **`FileAssert_File`** - `--config` verifies file-existence via glob pattern.
- **`FileAssert_Text`** - `--config` verifies file content with contains assertions.
- **`FileAssert_Html`** - `--config` verifies HTML XPath assertions.
- **`FileAssert_Xml`** - `--config` verifies XML XPath assertions.
- **`FileAssert_Yaml`** - `--config` verifies YAML key-path assertions.
- **`FileAssert_Json`** - `--config` verifies JSON key-path assertions.
- **`FileAssert_Pdf`** - `--config` verifies PDF metadata and text assertions.
- **`FileAssert_Zip`** - `--config` verifies zip archive entry assertions.

See the [User Guide][link-guide] for more details on the self-validation tests.

Expand Down
26 changes: 22 additions & 4 deletions docs/design/file-assert/selftest/validation.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,12 @@ file.

#### Class Structure

`Validation` is implemented as a `partial class` spread across 9 source files:
`Validation.cs`, `ValidationFile.cs`, `ValidationText.cs`, `ValidationHtml.cs`,
`ValidationXml.cs`, `ValidationYaml.cs`, `ValidationJson.cs`, `ValidationPdf.cs`,
and `ValidationZip.cs`. The orchestrating `Run()` method and shared utilities remain
in `Validation.cs`; each per-type test method lives in its own file.

##### Run Method

```csharp
Expand All @@ -31,8 +37,14 @@ Entry point for self-validation. Executes the following steps:
| `FileAssert_VersionDisplay` |
| `FileAssert_HelpDisplay` |
| `FileAssert_Results` |
| `FileAssert_Exists` |
| `FileAssert_Contains` |
| `FileAssert_File` |
| `FileAssert_Text` |
| `FileAssert_Html` |
| `FileAssert_Xml` |
| `FileAssert_Yaml` |
| `FileAssert_Json` |
| `FileAssert_Pdf` |
| `FileAssert_Zip` |

Each test is dispatched via `RunValidationTest`, which handles the common boilerplate:

Expand Down Expand Up @@ -108,8 +120,14 @@ N/A — `Validation` is a `static` class with no instance fields. All state is l
| `RunVersionTest(Context, TestResults)` *(private)* |
| `RunHelpTest(Context, TestResults)` *(private)* |
| `RunResultsTest(Context, TestResults)` *(private)* |
| `RunExistsTest(Context, TestResults)` *(private)* |
| `RunContainsTest(Context, TestResults)` *(private)* |
| `RunFileTest(Context, TestResults)` *(private)* |
| `RunTextTest(Context, TestResults)` *(private)* |
| `RunHtmlTest(Context, TestResults)` *(private)* |
| `RunXmlTest(Context, TestResults)` *(private)* |
| `RunYamlTest(Context, TestResults)` *(private)* |
| `RunJsonTest(Context, TestResults)` *(private)* |
| `RunPdfTest(Context, TestResults)` *(private)* |
| `RunZipTest(Context, TestResults)` *(private)* |
| `RunValidationTest(Context, TestResults, string, Func<string?>)` *(private)* |
| `WriteResultsFile(Context, TestResults)` *(private)* |
| `CreateTestResult(string)` *(private)* |
Expand Down
10 changes: 8 additions & 2 deletions docs/reqstream/file-assert/selftest.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,14 @@ sections:
- FileAssert-Validation-Run
- FileAssert-Validation-Results
- FileAssert-Validation-ResultsTest
- FileAssert-Validation-ExistsTest
- FileAssert-Validation-ContainsTest
- FileAssert-Validation-FileTest
- FileAssert-Validation-TextTest
- FileAssert-Validation-HtmlTest
- FileAssert-Validation-XmlTest
- FileAssert-Validation-YamlTest
- FileAssert-Validation-JsonTest
- FileAssert-Validation-PdfTest
- FileAssert-Validation-ZipTest
- FileAssert-Validation-NullContext
- FileAssert-Validation-Depth
tests:
Expand Down
64 changes: 59 additions & 5 deletions docs/reqstream/file-assert/selftest/validation.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -36,23 +36,77 @@ sections:
tests:
- Validation_Run_WithSilentContext_LogContainsFileAssertResults

- id: FileAssert-Validation-ExistsTest
- id: FileAssert-Validation-FileTest
title: The Validation class shall include a self-test that verifies file-existence checking.
justification: |
Demonstrates that glob-pattern file-existence assertions work correctly in the
deployment environment, providing evidence that the core file-matching feature
Comment thread
Malcolmnixon marked this conversation as resolved.
is functioning as expected.
tests:
- Validation_Run_WithSilentContext_LogContainsFileAssertExists
- Validation_Run_WithSilentContext_LogContainsFileAssertFile

- id: FileAssert-Validation-ContainsTest
title: The Validation class shall include a self-test that verifies file-content checking.
- id: FileAssert-Validation-TextTest
title: The Validation class shall include a self-test that verifies text-content checking.
justification: |
Demonstrates that text-contains assertions work correctly in the deployment
environment, providing evidence that the file content inspection feature
is functioning as expected.
tests:
- Validation_Run_WithSilentContext_LogContainsFileAssertContains
- Validation_Run_WithSilentContext_LogContainsFileAssertText

- id: FileAssert-Validation-HtmlTest
title: The Validation class shall include a self-test that verifies HTML assertion checking.
justification: |
Demonstrates that HTML XPath assertions work correctly in the deployment
environment, providing evidence that the HTML content inspection feature
is functioning as expected.
tests:
- Validation_Run_WithSilentContext_LogContainsFileAssertHtml

- id: FileAssert-Validation-XmlTest
title: The Validation class shall include a self-test that verifies XML assertion checking.
justification: |
Demonstrates that XML XPath assertions work correctly in the deployment
environment, providing evidence that the XML content inspection feature
is functioning as expected.
tests:
- Validation_Run_WithSilentContext_LogContainsFileAssertXml

- id: FileAssert-Validation-YamlTest
title: The Validation class shall include a self-test that verifies YAML assertion checking.
justification: |
Demonstrates that YAML dot-path assertions work correctly in the deployment
environment, providing evidence that the YAML content inspection feature
is functioning as expected.
tests:
- Validation_Run_WithSilentContext_LogContainsFileAssertYaml

- id: FileAssert-Validation-JsonTest
title: The Validation class shall include a self-test that verifies JSON assertion checking.
justification: |
Demonstrates that JSON dot-path assertions work correctly in the deployment
environment, providing evidence that the JSON content inspection feature
is functioning as expected.
tests:
- Validation_Run_WithSilentContext_LogContainsFileAssertJson

- id: FileAssert-Validation-PdfTest
title: The Validation class shall include a self-test that verifies PDF assertion checking.
justification: |
Demonstrates that PDF metadata and text assertions work correctly in the
deployment environment, providing evidence that the PDF content inspection
feature is functioning as expected.
tests:
- Validation_Run_WithSilentContext_LogContainsFileAssertPdf

- id: FileAssert-Validation-ZipTest
title: The Validation class shall include a self-test that verifies ZIP assertion checking.
justification: |
Demonstrates that ZIP nested file assertions work correctly in the
deployment environment, providing evidence that the ZIP archive inspection
feature is functioning as expected.
tests:
- Validation_Run_WithSilentContext_LogContainsFileAssertZip

- id: FileAssert-Validation-NullContext
title: The Validation class shall reject a null context argument.
Expand Down
36 changes: 24 additions & 12 deletions docs/user_guide/introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,20 +130,26 @@ Example validation report:

| Information | Value |
| :------------------ | :------------------------------------------------- |
| Tool Version | 1.0.0 |
| Machine Name | BUILD-SERVER |
| OS Version | Ubuntu 22.04.3 LTS |
| DotNet Runtime | .NET 10.0.0 |
| Time Stamp | 2024-01-15 10:30:00 UTC |
| Tool Version | <version> |
| Machine Name | <machine-name> |
| OS Version | <os-version> |
| DotNet Runtime | <dotnet-runtime-version> |
| Time Stamp | <timestamp> |

✓ FileAssert_VersionDisplay - Passed
✓ FileAssert_HelpDisplay - Passed
✓ FileAssert_Results - Passed
✓ FileAssert_Exists - Passed
✓ FileAssert_Contains - Passed

Total Tests: 5
Passed: 5
✓ FileAssert_File - Passed
✓ FileAssert_Text - Passed
✓ FileAssert_Html - Passed
✓ FileAssert_Xml - Passed
✓ FileAssert_Yaml - Passed
✓ FileAssert_Json - Passed
✓ FileAssert_Pdf - Passed
✓ FileAssert_Zip - Passed

Total Tests: 11
Passed: 11
Failed: 0
```

Expand All @@ -154,8 +160,14 @@ Each test proves specific functionality works correctly:
- **`FileAssert_VersionDisplay`** - `--version` outputs a valid version string.
- **`FileAssert_HelpDisplay`** - `--help` outputs usage and options information.
- **`FileAssert_Results`** - `--results` generates test results with pass and fail outcomes.
- **`FileAssert_Exists`** - `--config` verifies file-existence via glob pattern.
- **`FileAssert_Contains`** - `--config` verifies file content with contains assertions.
- **`FileAssert_File`** - `--config` verifies file-existence via glob pattern.
- **`FileAssert_Text`** - `--config` verifies file content with contains assertions.
- **`FileAssert_Html`** - `--config` verifies HTML XPath assertions.
- **`FileAssert_Xml`** - `--config` verifies XML XPath assertions.
- **`FileAssert_Yaml`** - `--config` verifies YAML key-path assertions.
- **`FileAssert_Json`** - `--config` verifies JSON key-path assertions.
- **`FileAssert_Pdf`** - `--config` verifies PDF metadata and text assertions.
- **`FileAssert_Zip`** - `--config` verifies zip archive entry assertions.

## Silent Mode

Expand Down
44 changes: 40 additions & 4 deletions docs/verification/file-assert/selftest/validation.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,17 +86,53 @@ is written to `context` indicating the unsupported format.

**Expected**: The log contains FileAssert results output.

##### Validation_Run_WithSilentContext_LogContainsFileAssertExists
##### Validation_Run_WithSilentContext_LogContainsFileAssertFile

**Scenario**: `Validation.Run` is called with a context that has logging enabled.

**Expected**: The log contains output from the FileAssert "exists" self-validation test.
**Expected**: The log contains output from the `FileAssert_File` self-validation test.

##### Validation_Run_WithSilentContext_LogContainsFileAssertContains
##### Validation_Run_WithSilentContext_LogContainsFileAssertText

**Scenario**: `Validation.Run` is called with a context that has logging enabled.

**Expected**: The log contains output from the FileAssert "contains" self-validation test.
**Expected**: The log contains output from the `FileAssert_Text` self-validation test.

##### Validation_Run_WithSilentContext_LogContainsFileAssertHtml

**Scenario**: `Validation.Run` is called with a context that has logging enabled.

**Expected**: The log contains output from the `FileAssert_Html` self-validation test.

##### Validation_Run_WithSilentContext_LogContainsFileAssertXml

**Scenario**: `Validation.Run` is called with a context that has logging enabled.

**Expected**: The log contains output from the `FileAssert_Xml` self-validation test.

##### Validation_Run_WithSilentContext_LogContainsFileAssertYaml

**Scenario**: `Validation.Run` is called with a context that has logging enabled.

**Expected**: The log contains output from the `FileAssert_Yaml` self-validation test.

##### Validation_Run_WithSilentContext_LogContainsFileAssertJson

**Scenario**: `Validation.Run` is called with a context that has logging enabled.

**Expected**: The log contains output from the `FileAssert_Json` self-validation test.

##### Validation_Run_WithSilentContext_LogContainsFileAssertPdf

**Scenario**: `Validation.Run` is called with a context that has logging enabled.

**Expected**: The log contains output from the `FileAssert_Pdf` self-validation test.

##### Validation_Run_WithSilentContext_LogContainsFileAssertZip

**Scenario**: `Validation.Run` is called with a context that has logging enabled.

**Expected**: The log contains output from the `FileAssert_Zip` self-validation test.

##### Validation_Run_WithDepth_UsesSpecifiedHeadingDepth

Expand Down
Loading
Loading