test: add vm-level fuzzing #5462
Conversation
matklad
force-pushed
the
m/runner-fuzzer
branch
from
8642aab to
2f08b2e
Compare
| } | ||
|
|
||
| fn create_context(input: Vec<u8>) -> VMContext { | ||
| VMContext { |
There was a problem hiding this comment.
Aren't some of these potentially controlled by the user? Would be ideal if the user-influenced parts were Arbitrary too. Potentially as a separate fuzz target?
There was a problem hiding this comment.
Yeah, that's a good idea. Let me issue this... #5465
|
pytest and nightly changes LGTM |
matklad
force-pushed
the
m/runner-fuzzer
branch
3 times, most recently
from
641529e to
d200ea8
Compare
| def run(dir: str, fuzz_target: str) -> int: | ||
| args = ('cargo', 'fuzz', 'run', fuzz_target, '--', '-len_control=0' | ||
| '-prefer_small=0', '-max_len=4000000', '-rss_limit_mb=10240') | ||
| os.environ['RUSTC_BOOTSTRAP'] = '1' |
There was a problem hiding this comment.
Fuzzing is an unstable features -- it can break with new compiler releases. So we need to opt-into using unstable features.
The official way to do this is via running a nightly toolchain, but that means that you need a separate nightly toolchain, and you need to manually correlate it with the stable version (you don't want to use current nightly, you want the nightly from the night where the stable release was cut).
The unofficial way to do this is to set this magical env var, which just unlocks all nightly features on stable. It is used by the rust compiler itself for bootstrapping: compiler internally uses nightly Rust features, but it can be build with the previous release.
There was a problem hiding this comment.
To spell this explicitly, by butting fuzzing into our CI we make it our responsibility to deal with any breakage related to compiler version upgrades. This in contrast to all over Rust functionality we use -- if something stable&documented breaks with an upgrade, we can and should complain upstream about that.
| [package] | ||
| name = "near-vm-runner-fuzz" | ||
| version = "0.0.0" | ||
| authors = ["Automatically generated"] |
There was a problem hiding this comment.
It is. Though as authors field is now optional, it's better to remove it here (and upstream in the generator itself as well, cc @nagisa ).
matklad
force-pushed
the
m/runner-fuzzer
branch
from
246e897 to
01bc34a
Compare
Finally got to publishing fuzzing infra we have been using in ad-hoc ways before. This just runs the contracts to make sure that VM doesn't crash. The code is setup to easily compare execution with different VM Kinds.
Note that at the moment fuzzing does not generally exercise host function calls. I need to finish bytecodealliance/wasm-tools#286 for that 😅