Test Coverage Report

[ggwpez]

Add just commands:

• just coverage to generate coverage report of the runtimes/ folder
  
• just coverage update ditto
• just coverage show try to open it, will open it into the default sheets app on MacOs

Depends on polkadot-fellows/runtimes#650

We can use this to track coverage over time and ensure that critical things are tested before going live.
Currently only supports storage checks, functional checks will be added next.

Example

The script scans for string Assert storage "Pallet::StorageItem::when::what". The string can appear anywhere in a line, but not multiple in one line.

The following can be used to mark the requirement that PreimageFor is empty on the Asset Hub before the migration as fulfilled:

fn pre_check(_rc_pre_payload: Self::RcPrePayload) -> Self::AhPrePayload {
	// AH does not have a preimage pallet, therefore must be empty.
	assert!(
		alias::PreimageFor::<T>::iter_keys().next().is_none(),
		"Assert storage 'Preimage::PreimageFor::ah_pre::empty'"
	);
}

You can also put it into a comment:

// Assert storage "Preimage::PreimageFor::ah_post::consistent"
for (key, preimage) in alias::PreimageFor::<T>::iter() {
	assert!(preimage.len() > 0, "Preimage::PreimageFor is empty");
        ...

It is possible to link requirements multiple times. The script records the file and line number to produce a GitHub permalink that will be valid once you push the code.

Output currently looks like this:

Workflow

Run just coverage, add a new assertion and run it again to check that the % number goes up. Alternatively, you can also put a typo to ensure that it picked up correctly.

[rockbmb]

I'll take a loot at the functional checks feature when it's done; it'd nice to also be able to use it for PET tests, although it's not critical.