Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split into library and binary parts and implement fuzzer #74

Merged
merged 33 commits into from
Sep 25, 2024
Merged

Conversation

realchonk
Copy link
Owner

@realchonk realchonk commented Sep 5, 2024

TODO:

  • use cargo workspaces
  • remove all dependencies on fuser from the library
  • remove anyhow dependency from library
  • update ChangeLog once everything is done

@realchonk realchonk requested a review from asomers September 5, 2024 22:53
Cargo.toml Outdated
@@ -32,3 +32,7 @@ rstest = { version = "0.19.0", default-features = false }
rstest_reuse = "0.7.0"
tempfile = "3.0"
xattr = "1.3.1"

[[bin]]
name = "fuse-ufs-fuser"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does the "-fuser" part indicate? Simply "fuse-ufs" would be more consistent with precedent.

Copy link
Owner Author

@realchonk realchonk Sep 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There will be a fuse-ufs-fuse2 for OpenBSD in the future.
The right binary will be chosen when building with make and installed as fuse-ufs.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Up until now, make hasn't been necessary for building; it's just been a convenience for development. It would be unfortunate if it suddenly becomes necessary. Plus, it would not be easy to integrate with things like the ports system. May I suggest two alternatives?

  • Define a fuser feature which is on-by-default, but which must be turned off to build on OpenBSD.
  • Have a single src/bin/fuse-ufs file which is a thin wrapper around either src/bin/fuse-ufs/fuser.rs or src/bin/fuse-ufs/fuse2, #[cfg()]-gated.

Copy link
Owner Author

@realchonk realchonk Sep 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this idea. I'll do this, once I have workspace-ified this project.
I think having fuser and fuse2 features would be useful, with only fuser being enabled by default.
On OpenBSD only fuse2 would work, on all other platforms, you could built with both (if you wanted that for some reason).
It sucks that per-platform features don't work

@realchonk realchonk changed the title Split into library and binary parts Split into library and binary parts and implement fuzzer Sep 11, 2024
@realchonk realchonk marked this pull request as ready for review September 11, 2024 18:50
@realchonk
Copy link
Owner Author

@asomers Feel free to review this now.

rufs/Cargo.toml Show resolved Hide resolved
fuse-ufs/fuse-ufs-fuser Outdated Show resolved Hide resolved
@@ -0,0 +1,13 @@
#![cfg_attr(fuzzing, allow(dead_code, unused_imports, unused_mut))]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you really need to disable these lints for fuzzing? This looks more like something temporary that got left behind.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are required to silence warnings which occur due to parts of the code not being called while fuzzing.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to remove those unused parts of code.

)
}
};
// FIXME: Choose based on hash of input or so, to excercise BE as well with introducing non-determinism
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that when fuzzing, you should guess the endianess just like you do when not fuzzing. You can generate multiple corpuses, right? One based on BE and one based on LE?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue as with the superblock check below - wasted fuzz cases due to guessing an invalid magic.

let mut file = Decoder::new(file, config);

let superblock: Superblock = file.decode_at(SBLOCK_UFS2 as u64)?;
#[cfg(not(fuzzing))]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why disable this check when fuzzing?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the fuzzer doesn't have to guess a valid superblock, which would lead to a lot of wasted fuzz cases as they'd fail at this stage without reaching crashes/panics deeper in the code. It's common practice to disable these sorts of checks while fuzzing for this reason.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fuzzer doesn't generate totally random input, right? It starts from a valid disk image and then randomly mutates it?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that it starts from a valid disk image, would you really expect to lost very many runs at this stage? And even if you disable the magic check here, wouldn't a run with a corrupt superblock still likely fail during SuperBlock::check ?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is no longer relevant, as I just always run the checks, even when fuzzing.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uh, what do you mean by "I just always run the checks"? Do you have a separate fork or something? Or is there a commit that you forgot to push? Because the magic check is still disabled during fuzzing.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I forgot to remove this check. I did the match on the magic.

rufs/Cargo.toml Show resolved Hide resolved
@realchonk realchonk mentioned this pull request Sep 22, 2024
@realchonk
Copy link
Owner Author

@asomers do you have any objections besides the (incomplete) fuzzing?
If not, I'd really like to finally merge this and improve the fuzzer later.

@realchonk realchonk merged commit 0693c99 into main Sep 25, 2024
5 checks passed
@realchonk realchonk deleted the refactor branch September 25, 2024 13:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants