Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove dependency on jvm-verifier #993

Closed
atomb opened this issue Jan 7, 2021 · 4 comments · Fixed by #1005
Closed

Remove dependency on jvm-verifier #993

atomb opened this issue Jan 7, 2021 · 4 comments · Fixed by #1005
Assignees
Labels
tech debt Issues that document or involve technical debt
Milestone

Comments

@atomb
Copy link
Contributor

atomb commented Jan 7, 2021

Now that Crucible-based Java verification has better functionality and reasonable performance relative to jvm-verifier (e.g., see #399), it's getting close to time to cut the dependency on jvm-verifier.

@atomb atomb added the tech debt Issues that document or involve technical debt label Jan 7, 2021
@atomb atomb self-assigned this Jan 7, 2021
@RyanGlScott
Copy link
Contributor

I've found myself needing to change code in jvm-verifier recently, so naturally, this issue is relevant to my interests. Replacing all of jvm-verifier with crucible-jvm is a pretty tall order (at least for a neophyte like me), since the two libraries' APIs differ significantly in some places. Compare jvm-verifier's runSimulator to crucible-jvm's executeCrucibleJVM, for instance.

I was initially hopeful that I could get away with only replacing a small subset of jvm-verifier's uses in SAW with crucible-jvm. The function from jvm-verifier I'm looking to change is tryLookupClass, which has an identical counterpart in crucible-jvm. Unfortunately, it's not as simple as swapping out the former for the latter, since the two functions use different Codebase data types, so any function that depends on tryLookupClass would now need to be updated to use the crucible-jvm version of Codebase. I attempted to do this, but this following all of the dependencies eventually led me back to the use of runSimulator! I gave up at that point.

@brianhuffman
Copy link
Contributor

One approach would be to start by removing all the deprecated jvm-verifier-based saw-script commands, and then proceed to remove all the dead code that results from that. Once all that stuff's gone, I imagine it would be a bit less work to remove the package dependency.

@brianhuffman
Copy link
Contributor

#1005 removes the dependency of the saw-script and saw-remote-api packages on jvm-verifier. However, some of the integration tests (including test0001) rely on some java programs and .jar files that currently reside in the jvm-verifier repository. We'll need to decide what to do about that.

@atomb atomb added this to the 0.8 milestone Jan 15, 2021
@kquick
Copy link
Member

kquick commented Jan 15, 2021

Plan: Remove JSS tests, and move jar files into saw repo examples directory.

RyanGlScott added a commit that referenced this issue Jan 27, 2021
This allows SAW to deal with JDK 9 or later, which packages its standard
library not in a JAR file, but in a JIMAGE file. Extracting `.class` files
from JIMAGE files proves to be surprisingly tricky, and I've carefully
documented the intricacies of doing so in
`Note [Loading classes from JIMAGE files]` in `SAWScript.JavaCodebase`.
This fixes #861.

This depends on #1030 to work.

Remaining tasks:

* Ideally, the code in `SAWScript.JavaCodebase` would be upstreamed to
  `crucible-jvm`, where the all-important `Codebase` data type lives.
  Unfortunately, some parts of SAW (e.g., `java_verify` still rely on
  the `jvm-verifier` library, which defines a separate `Codebase` type. SAW is
  in the process of phasing out the use of `jvm-verifier` in favor of
  `crucible-jvm` (see #993), but until that happens, I needed to introduce
  some ugly hacks in order to make everything typecheck.

  In particular, the (hopefully temporary) `SAWScript.JavaCodebase` module
  defines a shim version of `Codebase` that puts the experimental new things
  that I added in an `ExperimentalCodebase` constructor, but preserving the
  ability to use the `jvm-verifier` version of `Codebase` in the
  `LegacyCodebase` constructor. If JDK 8 or earlier is used, then
  `LegacyCodebase` is chosen, and if JDK 9 or later is used, then
  `ExperimentalCodebase` is chosen.

* Unfortunately, `java_verify` doesn't work with `ExperimentalCodebase`. Nor
  would we necessarily want to make this happen, as that would require
  upstreaming changes to `jvm-verifier`, which we are in the process of phasing
  out. As a result, this is blocked on #993.

* The CI should be updated to test more versions of the JDK than just 8.

Other things:

* I removed the dependency on the `xdg-basedir`, as it was unused. This
  dependency was likely added quite some time ago, and it appears that
  `saw-script` switched over to using XDG-related functionality from the
  `directory` library since then. I opted to use `directory` to find the
  `.cache` directory as well, so I have made that clear in the `.cabal` file.
RyanGlScott added a commit that referenced this issue Jan 29, 2021
This allows SAW to deal with JDK 9 or later, which packages its standard
library not in a JAR file, but in a JIMAGE file. Extracting `.class` files
from JIMAGE files proves to be surprisingly tricky, and I've carefully
documented the intricacies of doing so in
`Note [Loading classes from JIMAGE files]` in `SAWScript.JavaCodebase`.
This fixes #861.

Remaining tasks:

* Ideally, the code in `SAWScript.JavaCodebase` would be upstreamed to
  `crucible-jvm`, where the all-important `Codebase` data type lives.
  Unfortunately, some parts of SAW (e.g., `java_verify` still rely on
  the `jvm-verifier` library, which defines a separate `Codebase` type. SAW is
  in the process of phasing out the use of `jvm-verifier` in favor of
  `crucible-jvm` (see #993), but until that happens, I needed to introduce
  some ugly hacks in order to make everything typecheck.

  In particular, the (hopefully temporary) `SAWScript.JavaCodebase` module
  defines a shim version of `Codebase` that puts the experimental new things
  that I added in an `ExperimentalCodebase` constructor, but preserving the
  ability to use the `jvm-verifier` version of `Codebase` in the
  `LegacyCodebase` constructor. If JDK 8 or earlier is used, then
  `LegacyCodebase` is chosen, and if JDK 9 or later is used, then
  `ExperimentalCodebase` is chosen.

* Unfortunately, `java_verify` doesn't work with `ExperimentalCodebase`. Nor
  would we necessarily want to make this happen, as that would require
  upstreaming changes to `jvm-verifier`, which we are in the process of phasing
  out. As a result, this is blocked on #993.

* The CI should be updated to test more versions of the JDK than just 8.

Other things:

* I removed the dependency on the `xdg-basedir`, as it was unused. This
  dependency was likely added quite some time ago, and it appears that
  `saw-script` switched over to using XDG-related functionality from the
  `directory` library since then. I opted to use `directory` to find the
  `.cache` directory as well, so I have made that clear in the `.cabal` file.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tech debt Issues that document or involve technical debt
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants