Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UI fix : Fix missing function check descriptions #1126

Merged
merged 7 commits into from
Apr 28, 2022

Conversation

jaisnan
Copy link
Contributor

@jaisnan jaisnan commented Apr 28, 2022

Description of changes:

Fixes output for cases where there are checks with unreachable functions with missing definitions.

Before -

Check x: _ZN5alloc5alloc18handle_alloc_error17ha7321ad4cddc1663E.assertion.1
	 - Status: SUCCESS
	 - Description: "assertion false"
	 - Location: Unknown File in function _ZN5alloc5alloc18handle_alloc_error17ha7321ad4cddc1663E

After -

Check x: _ZN5alloc5alloc18handle_alloc_error17ha7321ad4cddc1663E.assertion.1
	 - Status: UNDETERMINED
	 - Description: "Function with missing definition is unreachable"
	 - Location: Unknown File in function _ZN5alloc5alloc18handle_alloc_error17ha7321ad4cddc1663E
  1. We replace the description for all such checks
  2. It changes the status of all checks to UNDETERMINED if any such check has a FAILURE status.

Resolved issues:

Addresses -

  1. Failure to cast back to &str from &[u8] #241
  2. Failures when iterating over results #556
  3. Multiple failures in overlapping traits example #564
  4. Failing verification for proptest example using format! #576

Call-outs:

Only a temporary fix , in the future, we'll need to resolve the mangled names.

Testing:

  • How is this change tested?

  • Is this a refactor change?

Checklist

  • Each commit message has a non-empty body, explaining why the change was made
  • Methods or procedures are documented
  • Regression or unit tests are included, or existing tests cover the modified code
  • My PR is restricted to a single feature or bugfix

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.

@jaisnan jaisnan requested a review from a team as a code owner April 28, 2022 06:10
Copy link
Contributor

@danielsn danielsn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you give some before/after examples of the UI

// SPDX-License-Identifier: Apache-2.0 OR MIT

// kani-flags: --enable-unstable --cbmc-args --unwind 4 --object-bits 9

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a comment explaining what this test is testing?

// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
// SPDX-License-Identifier: Apache-2.0 OR MIT

#[kani::proof]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, comment what is this test testing

scripts/cbmc_json_parser.py Show resolved Hide resolved
"""
prop_class = prop["property"].rsplit(".", 3)
# Do nothing if prop_class is diff than cbmc's convention
class_id = prop_class[-2] if len(prop_class) > 1 else None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we warn/log this error?

Copy link
Contributor Author

@jaisnan jaisnan Apr 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe in some developer log but to the user, this error should make no difference at all. This convention/tag schema is mostly for our consumption at the end of it.

scripts/cbmc_json_parser.py Outdated Show resolved Hide resolved
1. Searches for any check which has unknown file location or missing defition and replaces description
2. If any of these checks has a failure status, then we turn all the sucesses into undetermined.
"""
has_unknown_location_checks = False
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would call this has_reachable_undefined_functions.

"""
has_unknown_location_checks = False
for property in properties:
if GlobalMessages.ASSERTION_FALSE in property["description"] and extract_property_class(
Copy link
Contributor

@zhassan-aws zhassan-aws Apr 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a comment that this logic is intended to capture assertions that CBMC generates for functions with missing definitions?

One way to check whether this logic matches other checks is to run the regressions without the --generate-function-body-options assert-false-assume-false that we pass to goto-instrument (which would disable the undefined function checks) and see if any other checks are captured by this logic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After running the suggested experiment, no unexpected tests were found to change their result so we can safely assume that this logic doesn't capture false positives.

scripts/cbmc_json_parser.py Outdated Show resolved Hide resolved
tests/ui/missing-function/replaced-description/expected Outdated Show resolved Hide resolved
Copy link
Contributor

@zhassan-aws zhassan-aws left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM other than one small typo. Thanks!

scripts/cbmc_json_parser.py Outdated Show resolved Hide resolved
typo fixed

Co-authored-by: Zyad Hassan <[email protected]>
@jaisnan jaisnan merged commit 7cc6568 into model-checking:main Apr 28, 2022
@jaisnan jaisnan deleted the assertion-false-ui-fix branch April 28, 2022 21:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants