Skip to content

Plausibility test bom refs#14

Closed
CBeck-96 wants to merge 25 commits intoFesto-se:mainfrom
CBeck-96:plausibility_test_bom_refs
Closed

Plausibility test bom refs#14
CBeck-96 wants to merge 25 commits intoFesto-se:mainfrom
CBeck-96:plausibility_test_bom_refs

Conversation

@CBeck-96
Copy link
Copy Markdown
Collaborator

closes #5

@github-actions github-actions bot added enhancement New feature or request unittests labels Jun 13, 2023
Copy link
Copy Markdown
Collaborator

@mmarseu mmarseu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this check be part of the validate command? It is certainly a type of validation.
If it remains a separate command, available_commands.md should be updated.

IMO, the check that all bom-refs are unique is missing. Your code depends on this assumption but doesn't make sure it is correct. Again, this would be better suited for the validate command, because it is a requirement in the spec. It just can't be expressed in JSON schema.

Comment thread cdxev/__main__.py Outdated
_STATUS_APP_ERROR = 2
_STATUS_USAGE_ERROR = 3
_STATUS_VALIDATION_ERROR = 4
_STATUS_plausibility_ERROR = 5
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wrong case.

Comment thread cdxev/__main__.py Outdated
choices=["stdout", "warnings-ng"],
default="stdout",
)
add_output_argument(parser)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not use this argument for the report. For one, it is confusing because other commands use the same argument for outputting an SBOM, not a log file. Secondly, the help message is incorrect for your use case.

Also, the interaction between --report-format and --output is not clear to me. What if the user specifies --report-format stdout but still passes an output file? What if it is the other way around: --report-format warnings-ng but no output?

Comment thread cdxev/__main__.py Outdated
if args.output is None:
output = Path("./issues.json")
else:
output = args.output
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the user passes a path to a file in a folder that doesn't exist or a path to a folder, writing to this path will crash.

Comment thread cdxev/plausibility_check.py Outdated
Comment on lines +39 to +41
if report_format == "warnings-ng":
warnings_ng_handler = WarningsNgReporter(file, output)
logger.addHandler(warnings_ng_handler)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From a design-perspective I don't find it intuitive that the command logic should interpret the command-line arguments to decide where to log it's errors. This seems better suited for __main__.py, if you can manage to make that work.

Comment thread cdxev/plausibility_check.py Outdated

# check compositions
for composition in sbom.get("compositions", []):
for reference in composition.get("assemblies", []):
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're missing the compositions[].dependencies array which also contains refs.

Comment thread cdxev/plausibility_check.py Outdated
Comment on lines +140 to +142
# check if the dependency tree is connected, i.e. that the product
# decribed by the sbom depends directly or indirectly on every component.
# also checks that every component is depended on
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a misinterpretation of CycloneDX. Not all components in an SBOM are required to be part of the same dependency tree.

That is why dependencies and assemblies are both a thing in CycloneDX. There are products assembled from multiple components, where there is no depends-on relationship but rather an is-part-of relationship.

@italvi
Copy link
Copy Markdown
Collaborator

italvi commented Jun 14, 2023

Shouldn't this check be part of the validate command? It is certainly a type of validation. If it remains a separate command, available_commands.md should be updated.

IMO, the check that all bom-refs are unique is missing. Your code depends on this assumption but doesn't make sure it is correct. Again, this would be better suited for the validate command, because it is a requirement in the spec. It just can't be expressed in JSON schema.

Agree, though this should not be a default check when executing validate and be controlled through a flag.

Regarding the uniqueness of bom-ref within whole (for dependsOn the uniqueItems property is already used) SBOM you are correct, seems like jsonschema does not support unique key-value pairs, yet. But easy to loop through the SBOM and do this check ourselves.

@CBeck-96 CBeck-96 force-pushed the plausibility_test_bom_refs branch from abe4be1 to 9f40972 Compare July 1, 2023 23:45
Comment thread cdxev/__main__.py Outdated
Comment on lines +303 to +308
metavar="<plausability-check>",
choices=["yes", "y"],
help=(
"y/yes if the plausibility of the bom-refs in the"
"sbom should also be checked"
),
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this additional choice? Why not providing the flag plausibility-check means true?

Comment thread cdxev/__main__.py Outdated


def invoke_validate(args: argparse.Namespace) -> int:
logger_validate = logging.getLogger(__name__)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why introduce a new logger and not use the existing?

Comment thread tests/test_validate.py Outdated
Comment on lines +23 to +25
path_to_second_sbom = (
path_to_folder_with_test_sboms + "sub_programm_T5.0.3.96_20220217T101458_cdx.json"
)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why introduce a second SBOM, if you still make changes to it? In this case, just adjust your first sbom.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

adapting the tests for plausibility to this one is, and in doing so changing it to fit the requirements, would be work, that seemed unnecessary

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed

Comment thread tests/test_main.py Outdated
def test_get_validate(
self, mock_validate: unittest.mock.Mock, mock_read: unittest.mock.Mock
) -> None:
error_return = {"error"}
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

introduced for using it once?

Comment thread cdxev/validator/helper.py Outdated
errors.append(
create_error_orphaned_bom_ref(
affected.get("ref", ""),
"vulnerabilitie " + vulnerability.get("id", ""),
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

vulnerability

@CBeck-96 CBeck-96 requested review from italvi and mmarseu July 4, 2023 12:23
@mmarseu
Copy link
Copy Markdown
Collaborator

mmarseu commented Jul 2, 2024

Has this PR been abandoned? It's been in my todo list forever but now I'm not even sure there is a point in reviewing this anymore.

@CBeck-96 CBeck-96 closed this Jul 3, 2024
@CBeck-96
Copy link
Copy Markdown
Collaborator Author

CBeck-96 commented Jul 3, 2024

Considering there are other priorities and demand seems low, i will close it for the time beeing and tackle the subject at a more oppurtune time (propably version 1.0 or higher).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request unittests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a plausibility check for bom-refs

3 participants