Use sharding for performance tests

[swissspidy]

Running dozens of Lighthouse tests is very slow. By using sharding we can run the 3 test files all in parallel, theoretically speeding up things overall.

LH is not really relevant for the tests in this repo, so this is more a proof of concept.

---

Performance Test Results

Performance test results for 1a280dd are in 🛎️!

Note: the numbers in parentheses show the difference to the previous (baseline) test run.

WordPress Admin

Locale	Scenario	Object Cache	wp-memory-usage	wp-total	TTFB
en_US	Default		15.90 MB	244.70 ms (-29.31 ms / -12%)	263.30 ms (-34.65 ms / -13%)
de_DE	Default		32.42 MB	339.46 ms (+13.83 ms / +4%)	364.10 ms (+16.85 ms / +5%)
de_DE	Ginger MO (MO)		20.49 MB	304.24 ms	328.15 ms
de_DE	Ginger MO (PHP)		17.50 MB	258.81 ms (+6.59 ms / +3%)	290.40 ms (+18.20 ms / +6%)
de_DE	Ginger MO (JSON)		20.49 MB	297.62 ms (+12.43 ms / +4%)	321.55 ms (+19.65 ms / +6%)
de_DE	Native Gettext		16.36 MB	265.54 ms	287.20 ms
de_DE	DynaMo		20.99 MB	288.97 ms	306.40 ms (-11.80 ms / -4%)
de_DE	Cache in APCu		58.81 MB	343.91 ms (+11.77 ms / +3%)	364.10 ms (+14.60 ms / +4%)
en_US	Default	✅	16.07 MB	190.36 ms	204.60 ms
de_DE	Default	✅	32.35 MB	280.37 ms	296.65 ms
de_DE	Ginger MO (MO)	✅	20.42 MB	247.62 ms (+15.13 ms / +6%)	263.45 ms (+10.45 ms / +4%)
de_DE	Ginger MO (PHP)	✅	17.43 MB	209.36 ms (+10.59 ms / +5%)	224.95 ms (+9.55 ms / +4%)
de_DE	Ginger MO (JSON)	✅	20.42 MB	234.24 ms (+10.35 ms / +4%)	249.70 ms (+10.40 ms / +4%)
de_DE	Native Gettext	✅	16.36 MB	208.05 ms	230.75 ms
de_DE	DynaMo	✅	20.17 MB	224.91 ms (+10.04 ms / +4%)	239.85 ms (+6.75 ms / +3%)
de_DE	Cache in APCu	✅	58.74 MB	280.52 ms (-11.21 ms / -4%)	296.95 ms (-14.25 ms / -5%)
de_DE	Cache in object cache	✅	32.38 MB	273.13 ms (+7.01 ms / +3%)	288.40 ms

Twenty Twenty-Three

Locale	Scenario	Object Cache	wp-memory-usage	wp-total	TTFB
en_US	Default		15.05 MB	172.70 ms	181.90 ms
de_DE	Default		28.45 MB	230.93 ms (-13.26 ms / -6%)	239.25 ms (-13.70 ms / -6%)
de_DE	Ginger MO (MO)		18.47 MB	201.94 ms (-9.94 ms / -5%)	209.60 ms (-11.25 ms / -5%)
de_DE	Ginger MO (PHP)		16.19 MB	173.72 ms (-9.97 ms / -6%)	181.45 ms (-11.35 ms / -6%)
de_DE	Ginger MO (JSON)		18.47 MB	188.76 ms (-7.14 ms / -4%)	196.60 ms (-7.60 ms / -4%)
de_DE	Native Gettext		15.20 MB	171.67 ms	179.60 ms
de_DE	DynaMo		18.79 MB	197.64 ms	205.90 ms
de_DE	Cache in APCu		49.85 MB	231.49 ms (-11.23 ms / -5%)	240.25 ms (-12.55 ms / -5%)
en_US	Default	✅	15.11 MB	149.24 ms (-6.93 ms / -5%)	156.35 ms (-7.30 ms / -5%)
de_DE	Default	✅	28.53 MB	216.04 ms (+6.09 ms / +3%)	224.30 ms (+6.20 ms / +3%)
de_DE	Ginger MO (MO)	✅	18.55 MB	193.58 ms	201.85 ms (+5.45 ms / +3%)
de_DE	Ginger MO (PHP)	✅	16.27 MB	162.99 ms (+4.12 ms / +3%)	170.55 ms (+4.35 ms / +3%)
de_DE	Ginger MO (JSON)	✅	18.55 MB	177.54 ms	184.80 ms
de_DE	Native Gettext	✅	15.35 MB	161.72 ms	169.15 ms
de_DE	DynaMo	✅	18.01 MB	165.38 ms	172.55 ms
de_DE	Cache in APCu	✅	49.93 MB	217.82 ms	225.90 ms
de_DE	Cache in object cache	✅	28.59 MB	212.47 ms	220.35 ms

Twenty Twenty-One

Locale	Scenario	Object Cache	wp-memory-usage	wp-total	TTFB
en_US	Default		14.66 MB	131.63 ms (+3.75 ms / +3%)	139.05 ms (+3.60 ms / +3%)
de_DE	Default		28.12 MB	184.09 ms (+6.94 ms / +4%)	192.10 ms (+7.25 ms / +4%)
de_DE	Ginger MO (MO)		18.09 MB	156.45 ms	163.65 ms
de_DE	Ginger MO (PHP)		15.80 MB	140.25 ms (+7.47 ms / +5%)	147.35 ms (+7.55 ms / +5%)
de_DE	Ginger MO (JSON)		18.09 MB	151.86 ms	160.70 ms
de_DE	Native Gettext		14.81 MB	140.16 ms (+5.23 ms / +4%)	147.15 ms (+4.75 ms / +3%)
de_DE	DynaMo		18.42 MB	153.56 ms (-5.37 ms / -3%)	160.95 ms (-6.35 ms / -4%)
de_DE	Cache in APCu		49.63 MB	191.75 ms	200.05 ms
en_US	Default	✅	14.64 MB	116.92 ms	123.95 ms
de_DE	Default	✅	28.12 MB	165.56 ms	172.95 ms
de_DE	Ginger MO (MO)	✅	18.09 MB	142.84 ms	149.70 ms
de_DE	Ginger MO (PHP)	✅	15.81 MB	124.84 ms (-3.68 ms / -3%)	131.40 ms (-4.25 ms / -3%)
de_DE	Ginger MO (JSON)	✅	18.09 MB	134.03 ms	140.90 ms
de_DE	Native Gettext	✅	14.88 MB	121.32 ms (-6.49 ms / -5%)	128.10 ms (-6.60 ms / -5%)
de_DE	DynaMo	✅	17.56 MB	129.94 ms (+3.44 ms / +3%)	136.60 ms
de_DE	Cache in APCu	✅	49.63 MB	185.47 ms	193.15 ms
de_DE	Cache in object cache	✅	28.18 MB	167.42 ms	174.70 ms

[swissspidy]

Playwright now has better support for merging sharded reports, see https://github.com/microsoft/playwright/releases/tag/v1.37.0