Conversation
|
@CrockAgile: Thank you for submitting a pull request! Before we can merge it, you'll need to sign the Apollo Contributor License Agreement here: https://contribute.apollographql.com/ |
|
@apollo-cla never got the email, so guess I'll just wait 😴 update: it went to spam because the amazon SES isn't setup to use certificates, but I got it! |
|
@CrockAgile: thanks for this! to help me understand it, could you motivate the change for me? I'm not 100% sure I follow what's at stake or whether this fixes/improves something for folks |
|
@aaronArinder yeah sure! first motivation is just to update BNF to the latest version (I'm a maintainer, so selfishly working to keep more users on the recent version) other motivation is that BNF is being used in this code based for fuzz testing. the existing random generation in 0.5 was not really designed with fuzzing in mind tho, and did not prioritize exploring new paths to increase coverage BNF 0.6 has multiple strategies now, one of which being |
| bnf = "0.6" | ||
| env_logger = "0.11.0" | ||
| log = "0.4" | ||
| # Required until https://github.com/shnewto/bnf/pull/175, remove when bnf 0.6 is out |
There was a problem hiding this comment.
also addresses this leftover TODO due to rand version differences in bnf 0.5, which have been addressed in bnf 0.6
|
@aaronArinder anything I can do to explain this better? just let me know |
|
@CrockAgile: it's more of a priority thing, just waiting until I have more time to look at it. There's an internal advocate for this change, too, so I might rely on them to help out too |
goto-bus-stop
left a comment
There was a problem hiding this comment.
LGTM, thanks for doing this! & extra for the improvement to the fuzzer :)
|
@mergify copy dev |
✅ Pull request copies have been createdDetails
|
|
Our test suite can't run fully from a fork, so the PR will be merged thru #8935. Thanks again! |
Summary
bnfto 0.6 and remove therandversion pin that was required until bnf 0.6 (see #7897).connector_selection_parsefuzz target: switch to bnf'sCoverageGuidedstrategy and run multiple generations per fuzz input so we exercise more grammar productions per seed.Motivation
randand uses rand 0.9, so the fuzz crate no longer needs a pinnedranddependency.connector_selection_parsetarget previously used a single grammar generation per input;CoverageGuidedonly helps when the same strategy is reused across generations. Running several generations per input (same seed, same strategy) lets the strategy prefer previously unused productions, improving grammar coverage for the JSON selection parser fuzz run.Changes
fuzz/Cargo.toml: bumpbnffrom0.5.0to0.6, removerand = "=0.8.5"and the comment about bnf PR fix(deps): update rust crate anyhow to 1.0.48 #175.fuzz/fuzz_targets/connector_selection_parse.rs: usebnf::CoverageGuidedandbnf::rand(no directranddep); generate up to 8 selections per fuzz input viagenerate_seeded_with_strategyand parse each; keepCorpus::Rejectwhen no generation succeeds (e.g. non-terminating expansion).Checklist
Complete the checklist (and note appropriate exceptions) before the PR is marked ready-for-review.
Exceptions
cargo check -p router-fuzzand by running theconnector_selection_parsefuzz target.Notes
Footnotes
It may be appropriate to bring upcoming changes to the attention of other (impacted) groups. Please endeavour to do this before seeking PR approval. The mechanism for doing this will vary considerably, so use your judgement as to how and when to do this. ↩
Configuration is an important part of many changes. Where applicable please try to document configuration examples. ↩
A lot of (if not most) features benefit from built-in observability and
debug-level logs. Please read this guidance on metrics best-practices. ↩Tick whichever testing boxes are applicable. If you are adding Manual Tests, please document the manual testing (extensively) in the Exceptions. ↩