coverage: revert workarounds that are no longer neccessary

[ggreenway]

Description: Turn off 'Flaky' for coverage tests.
Risk Level: Low

Signed-off-by: Greg Greenway ggreenway@apple.com

[lizan]

@ggreenway coverage test is not always to fail, it depends how gtest shard tests into 50 runs, i.e. adding/removing test change the sharding. At some point it will break. It is always better to shuffle test order to detect flakiness / unintended dependencies between tests.

[ggreenway]

Hmm. What I was trying to fight is inconsistent coverage results between runs. Are we better off closing this so the tests don’t get stuck in a broken state?

[mattklein123]

Can we potentially keep the shuffle but remove the flaky tag so it runs once? We have fixed a bunch of things and also have stack traces now so it should be easier to understand what happens if it fails during the flake run?

[ggreenway]

Sounds reasonable to me. @lizan wdyt?

[lizan]

SGTM

[mattklein123]

The irony: coverage failed. Sigh. Maybe we shouldn't shuffle and just deal with whatever issues show up?

[ggreenway]

I'm confused. For that coverage run, there were no failures, but it says coverage was only 96.5%. The SHUFFLE shouldn't affect the overall coverage, just whether it passes/fails, right?

[mattklein123]

I'm confused. For that coverage run, there were no failures, but it says coverage was only 96.5%. The SHUFFLE shouldn't affect the overall coverage, just whether it passes/fails, right?

I'm actually not sure this is the case. I could see the order mattering in terms of the calculations, especially wrt to death tests and other strange things.

[ggreenway]

I'm fine with removing the SHUFFLE and seeing how it goes from there.