diff --git a/README.md b/README.md index c5aae6e..b4d121b 100644 --- a/README.md +++ b/README.md @@ -120,6 +120,25 @@ Tests: Validation Passed ``` +Each test in the report proves a specific command works correctly: + +- **SpdxTool_AddPackage** - `add-package` command adds a package with relationships to an SPDX file. +- **SpdxTool_AddRelationship** - `add-relationship` command adds a relationship between SPDX elements. +- **SpdxTool_Validate** - `validate` command accepts valid and rejects invalid SPDX files. +- **SpdxTool_CopyPackage** - `copy-package` command copies a package with relationships between SPDX files. +- **SpdxTool_Diagram** - `diagram` command generates a Mermaid diagram from an SPDX file. +- **SpdxTool_FindPackage** - `find-package` command locates a package by name in an SPDX file. +- **SpdxTool_GetVersion** - `get-version` command retrieves a package version from an SPDX file. +- **SpdxTool_Hash** - `hash` command generates and verifies file hashes. +- **SpdxTool_Ntia** - `validate` command enforces NTIA minimum SBOM element requirements. +- **SpdxTool_Query** - `query` command extracts values from program output. +- **SpdxTool_RenameId** - `rename-id` command renames an element identifier throughout an SPDX file. +- **SpdxTool_RunNuGetWorkflow** - `run-workflow` command executes a workflow from a NuGet package. +- **SpdxTool_ToMarkdown** - `to-markdown` command generates a Markdown summary from an SPDX file. +- **SpdxTool_UpdatePackage** - `update-package` command updates all fields of a package in an SPDX file. + +For detailed descriptions of each validation test, see the [User Guide][user-guide]. + On validation failure the tool will exit with a non-zero exit code. This report may be useful in regulated industries requiring evidence of tool validation. @@ -158,6 +177,7 @@ Additional information can be found at: [command-line-docs]: https://github.com/demaconsulting/SpdxTool/blob/main/docs/spdx-tool-command-line.md [workflow-docs]: https://github.com/demaconsulting/SpdxTool/blob/main/docs/spdx-tool-workflow-files.md +[user-guide]: https://github.com/demaconsulting/SpdxTool/blob/main/docs/guide/guide.md [contributing]: https://github.com/demaconsulting/SpdxTool/blob/main/CONTRIBUTING.md [code-of-conduct]: https://github.com/demaconsulting/SpdxTool/blob/main/CODE_OF_CONDUCT.md [architecture]: https://github.com/demaconsulting/SpdxTool/blob/main/ARCHITECTURE.md diff --git a/docs/guide/guide.md b/docs/guide/guide.md index 166cf59..984296e 100644 --- a/docs/guide/guide.md +++ b/docs/guide/guide.md @@ -932,6 +932,68 @@ Tests: Validation Passed ``` +## Validation Tests + +Each test exercises a specific SpdxTool command end-to-end and verifies the expected result: + +* **SpdxTool_AddPackage** - Creates an SPDX document with one package, then runs the `add-package` + command via a workflow to add a second package with a `BUILD_TOOL_OF` relationship. Verifies + that the resulting document contains both packages and the expected relationship. + +* **SpdxTool_AddRelationship** - Creates an SPDX document with two packages, then runs the + `add-relationship` command to add a `CONTAINS` relationship with a comment between them. + Verifies that the resulting document contains the relationship with the correct type and comment. + +* **SpdxTool_Validate** - Creates a valid SPDX document and verifies that the `validate` command + accepts it (exit code 0). Then creates an invalid SPDX document (package missing its SPDXID) + and verifies that the `validate` command rejects it with an appropriate error message. + +* **SpdxTool_CopyPackage** - Creates two SPDX documents each with one package, then runs the + `copy-package` command to copy a package from one document to the other with a `CONTAINED_BY` + relationship. Verifies that the target document contains both packages and the expected relationship. + +* **SpdxTool_Diagram** - Creates an SPDX document with two packages connected by a `DEPENDS_ON` + relationship, then runs the `diagram` command to generate a Mermaid diagram. Verifies that the + output file contains the expected `erDiagram` syntax, package names, and relationship type. + +* **SpdxTool_FindPackage** - Creates an SPDX document with two packages, then runs the + `find-package` command in a workflow to locate a package by name and print its ID. Verifies + that the log output contains the expected SPDX ID. + +* **SpdxTool_GetVersion** - Creates an SPDX document with two packages, then runs the + `get-version` command in a workflow to retrieve a package version by its SPDX ID and print it. + Verifies that the log output contains the expected version string. + +* **SpdxTool_Hash** - Creates a file with known content and runs the `hash generate` command to + produce a SHA-256 hash file. Verifies the hash value is correct. Then runs the `hash verify` + command with the correct hash (should pass) and with a corrupted hash (should fail). + +* **SpdxTool_Ntia** - Creates an SPDX document with a package missing the required supplier field. + Verifies that the `validate` command without the `ntia` flag passes, but with the `ntia` flag + fails with an error citing the missing supplier. Then creates an NTIA-compliant document and + verifies that `validate ntia` passes. + +* **SpdxTool_Query** - Runs the `query` command in a workflow to execute `dotnet --version` and + extract the version number using a regular expression pattern. Verifies that the log output + contains a version string in the expected format. + +* **SpdxTool_RenameId** - Creates an SPDX document with a package identified as `SPDXRef-Package-1`, + then runs the `rename-id` command to rename it to `SPDXRef-Package-2`. Verifies that the + resulting document contains the package under its new identifier. + +* **SpdxTool_RunNuGetWorkflow** - Runs the `run-workflow` command to execute the + `GetDotNetVersion.yaml` workflow from the `DemaConsulting.SpdxWorkflows` NuGet package. Verifies + that the workflow completes successfully (exit code 0). + +* **SpdxTool_ToMarkdown** - Creates an SPDX document with two packages in a `CONTAINS` + relationship, then runs the `to-markdown` command to generate a Markdown summary. Verifies that + the output file contains the expected title, section headings, and package information. + +* **SpdxTool_UpdatePackage** - Creates an SPDX document with one package, then runs the + `update-package` command to update all fields of the package including name, version, download + location, supplier, originator, homepage, copyright, summary, description, and license. + Verifies that all fields in the resulting document match the updated values. + ## Validation Failure On validation failure: