-
Notifications
You must be signed in to change notification settings - Fork 38.1k
tests: Dry run bench_bitcoin as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code #14092
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
in this case you'll probably want to remove the RUN_BENCH logic from travis as it does pretty much the same (introduced in #13811) |
|
@laanwj Yes, that is my plan if @MarcoFalke is OK with it :-) |
|
I don't see any downside to doing this. So Concept ACK, but I can't review build system changes. |
src/Makefile.test.include
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does scaling=0 actually run everything at least once? If not, just set to a small value larger than zero.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@MarcoFalke Yes it does, see:
Lines 115 to 118 in 78dae8c
| uint64_t num_iters = static_cast<uint64_t>(p.second.num_iters_for_one_second * scaling); | |
| if (0 == num_iters) { | |
| num_iters = 1; | |
| } |
1d0e186 to
fcb34f4
Compare
ab4bea1 to
e4f0372
Compare
…time) as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code
e4f0372 to
dfef0df
Compare
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
|
Concept ACK dfef0df, can't review the build system changes, though. |
|
Tested by merging dfef0df and then applying the changes from #14011 (so that make check actually runs on macOS). Got the bench output: Added an Probably worth getting @theuni to glance over the test.include changes pre-merge. |
…low for quick identification of assertion/sanitizer failures in benchmarking code dfef0df tests: Dry run bench_bitcoin (-evals=1 -scaling=0: <1 second running time) as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code (practicalswift) 00c6306 Remove RUN_BENCH logic (practicalswift) Pull request description: Dry run `bench_bitcoin` (`-evals=1 -scaling=0`: <1 second running time) as part `make check` to allow for quick identification of assertion/sanitizer failures or crashes in benchmarking code. This is already tested in Travis but it is nice to have it locally too. The cost is near zero. Tree-SHA512: 1f51b86b34bf97f75785f2694891d80f1bfb3e050211e6f6c35d8d9bc80c75bdebaa5ebfa51855ac0cf76d8773c3026bc576f60d0227afb0e646d728b83abde7
…" to allow for quick identification of assertion/sanitizer failures in benchmarking code dfef0df tests: Dry run bench_bitcoin (-evals=1 -scaling=0: <1 second running time) as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code (practicalswift) 00c6306 Remove RUN_BENCH logic (practicalswift) Pull request description: Dry run `bench_bitcoin` (`-evals=1 -scaling=0`: <1 second running time) as part `make check` to allow for quick identification of assertion/sanitizer failures or crashes in benchmarking code. This is already tested in Travis but it is nice to have it locally too. The cost is near zero. Tree-SHA512: 1f51b86b34bf97f75785f2694891d80f1bfb3e050211e6f6c35d8d9bc80c75bdebaa5ebfa51855ac0cf76d8773c3026bc576f60d0227afb0e646d728b83abde7
…" to allow for quick identification of assertion/sanitizer failures in benchmarking code dfef0df tests: Dry run bench_bitcoin (-evals=1 -scaling=0: <1 second running time) as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code (practicalswift) 00c6306 Remove RUN_BENCH logic (practicalswift) Pull request description: Dry run `bench_bitcoin` (`-evals=1 -scaling=0`: <1 second running time) as part `make check` to allow for quick identification of assertion/sanitizer failures or crashes in benchmarking code. This is already tested in Travis but it is nice to have it locally too. The cost is near zero. Tree-SHA512: 1f51b86b34bf97f75785f2694891d80f1bfb3e050211e6f6c35d8d9bc80c75bdebaa5ebfa51855ac0cf76d8773c3026bc576f60d0227afb0e646d728b83abde7
…" to allow for quick identification of assertion/sanitizer failures in benchmarking code dfef0df tests: Dry run bench_bitcoin (-evals=1 -scaling=0: <1 second running time) as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code (practicalswift) 00c6306 Remove RUN_BENCH logic (practicalswift) Pull request description: Dry run `bench_bitcoin` (`-evals=1 -scaling=0`: <1 second running time) as part `make check` to allow for quick identification of assertion/sanitizer failures or crashes in benchmarking code. This is already tested in Travis but it is nice to have it locally too. The cost is near zero. Tree-SHA512: 1f51b86b34bf97f75785f2694891d80f1bfb3e050211e6f6c35d8d9bc80c75bdebaa5ebfa51855ac0cf76d8773c3026bc576f60d0227afb0e646d728b83abde7
…" to allow for quick identification of assertion/sanitizer failures in benchmarking code dfef0df tests: Dry run bench_bitcoin (-evals=1 -scaling=0: <1 second running time) as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code (practicalswift) 00c6306 Remove RUN_BENCH logic (practicalswift) Pull request description: Dry run `bench_bitcoin` (`-evals=1 -scaling=0`: <1 second running time) as part `make check` to allow for quick identification of assertion/sanitizer failures or crashes in benchmarking code. This is already tested in Travis but it is nice to have it locally too. The cost is near zero. Tree-SHA512: 1f51b86b34bf97f75785f2694891d80f1bfb3e050211e6f6c35d8d9bc80c75bdebaa5ebfa51855ac0cf76d8773c3026bc576f60d0227afb0e646d728b83abde7
…" to allow for quick identification of assertion/sanitizer failures in benchmarking code dfef0df tests: Dry run bench_bitcoin (-evals=1 -scaling=0: <1 second running time) as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code (practicalswift) 00c6306 Remove RUN_BENCH logic (practicalswift) Pull request description: Dry run `bench_bitcoin` (`-evals=1 -scaling=0`: <1 second running time) as part `make check` to allow for quick identification of assertion/sanitizer failures or crashes in benchmarking code. This is already tested in Travis but it is nice to have it locally too. The cost is near zero. Tree-SHA512: 1f51b86b34bf97f75785f2694891d80f1bfb3e050211e6f6c35d8d9bc80c75bdebaa5ebfa51855ac0cf76d8773c3026bc576f60d0227afb0e646d728b83abde7
…" to allow for quick identification of assertion/sanitizer failures in benchmarking code dfef0df tests: Dry run bench_bitcoin (-evals=1 -scaling=0: <1 second running time) as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code (practicalswift) 00c6306 Remove RUN_BENCH logic (practicalswift) Pull request description: Dry run `bench_bitcoin` (`-evals=1 -scaling=0`: <1 second running time) as part `make check` to allow for quick identification of assertion/sanitizer failures or crashes in benchmarking code. This is already tested in Travis but it is nice to have it locally too. The cost is near zero. Tree-SHA512: 1f51b86b34bf97f75785f2694891d80f1bfb3e050211e6f6c35d8d9bc80c75bdebaa5ebfa51855ac0cf76d8773c3026bc576f60d0227afb0e646d728b83abde7
…" to allow for quick identification of assertion/sanitizer failures in benchmarking code dfef0df tests: Dry run bench_bitcoin (-evals=1 -scaling=0: <1 second running time) as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code (practicalswift) 00c6306 Remove RUN_BENCH logic (practicalswift) Pull request description: Dry run `bench_bitcoin` (`-evals=1 -scaling=0`: <1 second running time) as part `make check` to allow for quick identification of assertion/sanitizer failures or crashes in benchmarking code. This is already tested in Travis but it is nice to have it locally too. The cost is near zero. Tree-SHA512: 1f51b86b34bf97f75785f2694891d80f1bfb3e050211e6f6c35d8d9bc80c75bdebaa5ebfa51855ac0cf76d8773c3026bc576f60d0227afb0e646d728b83abde7
…ke check" to allow for quick identification of assertion/sanitizer failures in benchmarking code
…" to allow for quick identification of assertion/sanitizer failures in benchmarking code dfef0df tests: Dry run bench_bitcoin (-evals=1 -scaling=0: <1 second running time) as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code (practicalswift) 00c6306 Remove RUN_BENCH logic (practicalswift) Pull request description: Dry run `bench_bitcoin` (`-evals=1 -scaling=0`: <1 second running time) as part `make check` to allow for quick identification of assertion/sanitizer failures or crashes in benchmarking code. This is already tested in Travis but it is nice to have it locally too. The cost is near zero. Tree-SHA512: 1f51b86b34bf97f75785f2694891d80f1bfb3e050211e6f6c35d8d9bc80c75bdebaa5ebfa51855ac0cf76d8773c3026bc576f60d0227afb0e646d728b83abde7
…" to allow for quick identification of assertion/sanitizer failures in benchmarking code dfef0df tests: Dry run bench_bitcoin (-evals=1 -scaling=0: <1 second running time) as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code (practicalswift) 00c6306 Remove RUN_BENCH logic (practicalswift) Pull request description: Dry run `bench_bitcoin` (`-evals=1 -scaling=0`: <1 second running time) as part `make check` to allow for quick identification of assertion/sanitizer failures or crashes in benchmarking code. This is already tested in Travis but it is nice to have it locally too. The cost is near zero. Tree-SHA512: 1f51b86b34bf97f75785f2694891d80f1bfb3e050211e6f6c35d8d9bc80c75bdebaa5ebfa51855ac0cf76d8773c3026bc576f60d0227afb0e646d728b83abde7
…" to allow for quick identification of assertion/sanitizer failures in benchmarking code dfef0df tests: Dry run bench_bitcoin (-evals=1 -scaling=0: <1 second running time) as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code (practicalswift) 00c6306 Remove RUN_BENCH logic (practicalswift) Pull request description: Dry run `bench_bitcoin` (`-evals=1 -scaling=0`: <1 second running time) as part `make check` to allow for quick identification of assertion/sanitizer failures or crashes in benchmarking code. This is already tested in Travis but it is nice to have it locally too. The cost is near zero. Tree-SHA512: 1f51b86b34bf97f75785f2694891d80f1bfb3e050211e6f6c35d8d9bc80c75bdebaa5ebfa51855ac0cf76d8773c3026bc576f60d0227afb0e646d728b83abde7
…" to allow for quick identification of assertion/sanitizer failures in benchmarking code dfef0df tests: Dry run bench_bitcoin (-evals=1 -scaling=0: <1 second running time) as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code (practicalswift) 00c6306 Remove RUN_BENCH logic (practicalswift) Pull request description: Dry run `bench_bitcoin` (`-evals=1 -scaling=0`: <1 second running time) as part `make check` to allow for quick identification of assertion/sanitizer failures or crashes in benchmarking code. This is already tested in Travis but it is nice to have it locally too. The cost is near zero. Tree-SHA512: 1f51b86b34bf97f75785f2694891d80f1bfb3e050211e6f6c35d8d9bc80c75bdebaa5ebfa51855ac0cf76d8773c3026bc576f60d0227afb0e646d728b83abde7
…" to allow for quick identification of assertion/sanitizer failures in benchmarking code dfef0df tests: Dry run bench_bitcoin (-evals=1 -scaling=0: <1 second running time) as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code (practicalswift) 00c6306 Remove RUN_BENCH logic (practicalswift) Pull request description: Dry run `bench_bitcoin` (`-evals=1 -scaling=0`: <1 second running time) as part `make check` to allow for quick identification of assertion/sanitizer failures or crashes in benchmarking code. This is already tested in Travis but it is nice to have it locally too. The cost is near zero. Tree-SHA512: 1f51b86b34bf97f75785f2694891d80f1bfb3e050211e6f6c35d8d9bc80c75bdebaa5ebfa51855ac0cf76d8773c3026bc576f60d0227afb0e646d728b83abde7
…" to allow for quick identification of assertion/sanitizer failures in benchmarking code dfef0df tests: Dry run bench_bitcoin (-evals=1 -scaling=0: <1 second running time) as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code (practicalswift) 00c6306 Remove RUN_BENCH logic (practicalswift) Pull request description: Dry run `bench_bitcoin` (`-evals=1 -scaling=0`: <1 second running time) as part `make check` to allow for quick identification of assertion/sanitizer failures or crashes in benchmarking code. This is already tested in Travis but it is nice to have it locally too. The cost is near zero. Tree-SHA512: 1f51b86b34bf97f75785f2694891d80f1bfb3e050211e6f6c35d8d9bc80c75bdebaa5ebfa51855ac0cf76d8773c3026bc576f60d0227afb0e646d728b83abde7
…" to allow for quick identification of assertion/sanitizer failures in benchmarking code dfef0df tests: Dry run bench_bitcoin (-evals=1 -scaling=0: <1 second running time) as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code (practicalswift) 00c6306 Remove RUN_BENCH logic (practicalswift) Pull request description: Dry run `bench_bitcoin` (`-evals=1 -scaling=0`: <1 second running time) as part `make check` to allow for quick identification of assertion/sanitizer failures or crashes in benchmarking code. This is already tested in Travis but it is nice to have it locally too. The cost is near zero. Tree-SHA512: 1f51b86b34bf97f75785f2694891d80f1bfb3e050211e6f6c35d8d9bc80c75bdebaa5ebfa51855ac0cf76d8773c3026bc576f60d0227afb0e646d728b83abde7
…" to allow for quick identification of assertion/sanitizer failures in benchmarking code dfef0df tests: Dry run bench_bitcoin (-evals=1 -scaling=0: <1 second running time) as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code (practicalswift) 00c6306 Remove RUN_BENCH logic (practicalswift) Pull request description: Dry run `bench_bitcoin` (`-evals=1 -scaling=0`: <1 second running time) as part `make check` to allow for quick identification of assertion/sanitizer failures or crashes in benchmarking code. This is already tested in Travis but it is nice to have it locally too. The cost is near zero. Tree-SHA512: 1f51b86b34bf97f75785f2694891d80f1bfb3e050211e6f6c35d8d9bc80c75bdebaa5ebfa51855ac0cf76d8773c3026bc576f60d0227afb0e646d728b83abde7
…" to allow for quick identification of assertion/sanitizer failures in benchmarking code dfef0df tests: Dry run bench_bitcoin (-evals=1 -scaling=0: <1 second running time) as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code (practicalswift) 00c6306 Remove RUN_BENCH logic (practicalswift) Pull request description: Dry run `bench_bitcoin` (`-evals=1 -scaling=0`: <1 second running time) as part `make check` to allow for quick identification of assertion/sanitizer failures or crashes in benchmarking code. This is already tested in Travis but it is nice to have it locally too. The cost is near zero. Tree-SHA512: 1f51b86b34bf97f75785f2694891d80f1bfb3e050211e6f6c35d8d9bc80c75bdebaa5ebfa51855ac0cf76d8773c3026bc576f60d0227afb0e646d728b83abde7
…" to allow for quick identification of assertion/sanitizer failures in benchmarking code dfef0df tests: Dry run bench_bitcoin (-evals=1 -scaling=0: <1 second running time) as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code (practicalswift) 00c6306 Remove RUN_BENCH logic (practicalswift) Pull request description: Dry run `bench_bitcoin` (`-evals=1 -scaling=0`: <1 second running time) as part `make check` to allow for quick identification of assertion/sanitizer failures or crashes in benchmarking code. This is already tested in Travis but it is nice to have it locally too. The cost is near zero. Tree-SHA512: 1f51b86b34bf97f75785f2694891d80f1bfb3e050211e6f6c35d8d9bc80c75bdebaa5ebfa51855ac0cf76d8773c3026bc576f60d0227afb0e646d728b83abde7
…" to allow for quick identification of assertion/sanitizer failures in benchmarking code dfef0df tests: Dry run bench_bitcoin (-evals=1 -scaling=0: <1 second running time) as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code (practicalswift) 00c6306 Remove RUN_BENCH logic (practicalswift) Pull request description: Dry run `bench_bitcoin` (`-evals=1 -scaling=0`: <1 second running time) as part `make check` to allow for quick identification of assertion/sanitizer failures or crashes in benchmarking code. This is already tested in Travis but it is nice to have it locally too. The cost is near zero. Tree-SHA512: 1f51b86b34bf97f75785f2694891d80f1bfb3e050211e6f6c35d8d9bc80c75bdebaa5ebfa51855ac0cf76d8773c3026bc576f60d0227afb0e646d728b83abde7
…" to allow for quick identification of assertion/sanitizer failures in benchmarking code dfef0df tests: Dry run bench_bitcoin (-evals=1 -scaling=0: <1 second running time) as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code (practicalswift) 00c6306 Remove RUN_BENCH logic (practicalswift) Pull request description: Dry run `bench_bitcoin` (`-evals=1 -scaling=0`: <1 second running time) as part `make check` to allow for quick identification of assertion/sanitizer failures or crashes in benchmarking code. This is already tested in Travis but it is nice to have it locally too. The cost is near zero. Tree-SHA512: 1f51b86b34bf97f75785f2694891d80f1bfb3e050211e6f6c35d8d9bc80c75bdebaa5ebfa51855ac0cf76d8773c3026bc576f60d0227afb0e646d728b83abde7
…" to allow for quick identification of assertion/sanitizer failures in benchmarking code dfef0df tests: Dry run bench_bitcoin (-evals=1 -scaling=0: <1 second running time) as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code (practicalswift) 00c6306 Remove RUN_BENCH logic (practicalswift) Pull request description: Dry run `bench_bitcoin` (`-evals=1 -scaling=0`: <1 second running time) as part `make check` to allow for quick identification of assertion/sanitizer failures or crashes in benchmarking code. This is already tested in Travis but it is nice to have it locally too. The cost is near zero. Tree-SHA512: 1f51b86b34bf97f75785f2694891d80f1bfb3e050211e6f6c35d8d9bc80c75bdebaa5ebfa51855ac0cf76d8773c3026bc576f60d0227afb0e646d728b83abde7
…" to allow for quick identification of assertion/sanitizer failures in benchmarking code dfef0df tests: Dry run bench_bitcoin (-evals=1 -scaling=0: <1 second running time) as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code (practicalswift) 00c6306 Remove RUN_BENCH logic (practicalswift) Pull request description: Dry run `bench_bitcoin` (`-evals=1 -scaling=0`: <1 second running time) as part `make check` to allow for quick identification of assertion/sanitizer failures or crashes in benchmarking code. This is already tested in Travis but it is nice to have it locally too. The cost is near zero. Tree-SHA512: 1f51b86b34bf97f75785f2694891d80f1bfb3e050211e6f6c35d8d9bc80c75bdebaa5ebfa51855ac0cf76d8773c3026bc576f60d0227afb0e646d728b83abde7
…" to allow for quick identification of assertion/sanitizer failures in benchmarking code dfef0df tests: Dry run bench_bitcoin (-evals=1 -scaling=0: <1 second running time) as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code (practicalswift) 00c6306 Remove RUN_BENCH logic (practicalswift) Pull request description: Dry run `bench_bitcoin` (`-evals=1 -scaling=0`: <1 second running time) as part `make check` to allow for quick identification of assertion/sanitizer failures or crashes in benchmarking code. This is already tested in Travis but it is nice to have it locally too. The cost is near zero. Tree-SHA512: 1f51b86b34bf97f75785f2694891d80f1bfb3e050211e6f6c35d8d9bc80c75bdebaa5ebfa51855ac0cf76d8773c3026bc576f60d0227afb0e646d728b83abde7
…" to allow for quick identification of assertion/sanitizer failures in benchmarking code dfef0df tests: Dry run bench_bitcoin (-evals=1 -scaling=0: <1 second running time) as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code (practicalswift) 00c6306 Remove RUN_BENCH logic (practicalswift) Pull request description: Dry run `bench_bitcoin` (`-evals=1 -scaling=0`: <1 second running time) as part `make check` to allow for quick identification of assertion/sanitizer failures or crashes in benchmarking code. This is already tested in Travis but it is nice to have it locally too. The cost is near zero. Tree-SHA512: 1f51b86b34bf97f75785f2694891d80f1bfb3e050211e6f6c35d8d9bc80c75bdebaa5ebfa51855ac0cf76d8773c3026bc576f60d0227afb0e646d728b83abde7
…" to allow for quick identification of assertion/sanitizer failures in benchmarking code dfef0df tests: Dry run bench_bitcoin (-evals=1 -scaling=0: <1 second running time) as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code (practicalswift) 00c6306 Remove RUN_BENCH logic (practicalswift) Pull request description: Dry run `bench_bitcoin` (`-evals=1 -scaling=0`: <1 second running time) as part `make check` to allow for quick identification of assertion/sanitizer failures or crashes in benchmarking code. This is already tested in Travis but it is nice to have it locally too. The cost is near zero. Tree-SHA512: 1f51b86b34bf97f75785f2694891d80f1bfb3e050211e6f6c35d8d9bc80c75bdebaa5ebfa51855ac0cf76d8773c3026bc576f60d0227afb0e646d728b83abde7
…" to allow for quick identification of assertion/sanitizer failures in benchmarking code dfef0df tests: Dry run bench_bitcoin (-evals=1 -scaling=0: <1 second running time) as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code (practicalswift) 00c6306 Remove RUN_BENCH logic (practicalswift) Pull request description: Dry run `bench_bitcoin` (`-evals=1 -scaling=0`: <1 second running time) as part `make check` to allow for quick identification of assertion/sanitizer failures or crashes in benchmarking code. This is already tested in Travis but it is nice to have it locally too. The cost is near zero. Tree-SHA512: 1f51b86b34bf97f75785f2694891d80f1bfb3e050211e6f6c35d8d9bc80c75bdebaa5ebfa51855ac0cf76d8773c3026bc576f60d0227afb0e646d728b83abde7
…" to allow for quick identification of assertion/sanitizer failures in benchmarking code dfef0df tests: Dry run bench_bitcoin (-evals=1 -scaling=0: <1 second running time) as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code (practicalswift) 00c6306 Remove RUN_BENCH logic (practicalswift) Pull request description: Dry run `bench_bitcoin` (`-evals=1 -scaling=0`: <1 second running time) as part `make check` to allow for quick identification of assertion/sanitizer failures or crashes in benchmarking code. This is already tested in Travis but it is nice to have it locally too. The cost is near zero. Tree-SHA512: 1f51b86b34bf97f75785f2694891d80f1bfb3e050211e6f6c35d8d9bc80c75bdebaa5ebfa51855ac0cf76d8773c3026bc576f60d0227afb0e646d728b83abde7
Dry run
bench_bitcoin(-evals=1 -scaling=0: <1 second running time) as partmake checkto allow for quick identification of assertion/sanitizer failures or crashes in benchmarking code.This is already tested in Travis but it is nice to have it locally too. The cost is near zero.