test(sync): stage test suite

[rkrasiuk]

Create a common test suite for stages.

The stage_test_suite macro contains the default stage tests impl:

• call Stage::execute with empty db
• call Stage::execute with input stage_progress equal to prev_stage_progress
• call Stage::execute with normal input and validate the database entries
• call Stage::unwind with empty db
• call Stage::execute and call Stage::unwind to the original stage_progress. validate the database state after each call

[rkrasiuk]

@onbjerg one more thing i've noticed is that some bodies tests are flaky. full_body_download or partial_body_download can fail once a while

[codecov]

Codecov Report

Merging #204 (14e1800) into main (6b336c6) will decrease coverage by 0.87%.
The diff coverage is 98.25%.

❗ Current head 14e1800 differs from pull request most recent head deb0807. Consider uploading reports for the commit deb0807 to get more accurate results

@@            Coverage Diff             @@
##             main     #204      +/-   ##
==========================================
- Coverage   67.74%   66.86%   -0.88%     
==========================================
  Files         213      217       +4     
  Lines       18142    18205      +63     
==========================================
- Hits        12290    12173     -117     
- Misses       5852     6032     +180     

Impacted Files	Coverage Δ
crates/stages/src/lib.rs	100.00% <ø> (ø)
crates/stages/src/util.rs	100.00% <ø> (+1.04%)	⬆️
crates/interfaces/src/test_utils/headers.rs	88.79% <91.89%> (+2.12%)	⬆️
crates/stages/src/test_utils/runner.rs	96.55% <96.55%> (ø)
crates/stages/src/stages/bodies.rs	95.39% <96.87%> (-1.09%)	⬇️
crates/db/src/kv/mod.rs	94.83% <100.00%> (ø)
crates/net/headers-downloaders/src/linear.rs	95.43% <100.00%> (ø)
crates/stages/src/stages/headers.rs	96.98% <100.00%> (-0.45%)	⬇️
crates/stages/src/stages/tx_index.rs	96.33% <100.00%> (+1.65%)	⬆️
crates/stages/src/test_utils/macros.rs	100.00% <100.00%> (ø)
... and 32 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[onbjerg]

@onbjerg one more thing i've noticed is that some bodies tests are flaky. full_body_download or partial_body_download can fail once a while

This is likely because the blocks are randomly generated and empty blocks are skipped, i.e. if a block has no ommers and no transactions we don't request it from the network. So the amount of blocks processed by the stage might deviate a bit

[onbjerg]

A few questions, but also unsure how to validate if the body tests do the same things as before/cover the same edge cases. Any ideas on how we could do that?

[rkrasiuk]

@onbjerg not sure other than the manual inspection. i removed the tests that are covered by the stage_test_suite, but left all the edge cases. dedupped code in the impl of ExecuteStageTestRunner and UnwindStageTestRunner

[onbjerg]

@rkrasiuk Seems like the same lines are covered in both stages, so should be good

[gakonst]

@rkrasiuk any idea why f860865 deadlocks? i reverted and merged but it seems weird to me

[rkrasiuk]

@gakonst yeah, for headers you need to update chain tip before await the value