-
Notifications
You must be signed in to change notification settings - Fork 476
test(profiling): address flakiness in test_uwsgi.py
#15958
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
test(profiling): address flakiness in test_uwsgi.py
#15958
Conversation
Codeowners resolved as |
Performance SLOsComparing candidate kowalski/test-profiling-address-flakiness-in-test_uwsgi_threads_processes_no_primary_lazy_apps (acdf55b) with baseline main (0b7d961) 📈 Performance Regressions (3 suites)📈 iastaspects - 118/118✅ add_aspectTime: ✅ 104.130µs (SLO: <130.000µs 📉 -19.9%) vs baseline: +3.6% Memory: ✅ 42.644MB (SLO: <43.250MB 🟡 -1.4%) vs baseline: +4.9% ✅ add_inplace_aspectTime: ✅ 101.595µs (SLO: <130.000µs 📉 -21.8%) vs baseline: +0.7% Memory: ✅ 42.625MB (SLO: <43.250MB 🟡 -1.4%) vs baseline: +4.7% ✅ add_inplace_noaspectTime: ✅ 28.020µs (SLO: <40.000µs 📉 -30.0%) vs baseline: -1.6% Memory: ✅ 42.625MB (SLO: <43.500MB -2.0%) vs baseline: +5.0% ✅ add_noaspectTime: ✅ 48.969µs (SLO: <70.000µs 📉 -30.0%) vs baseline: +0.7% Memory: ✅ 42.566MB (SLO: <43.500MB -2.1%) vs baseline: +4.6% ✅ bytearray_aspectTime: ✅ 257.149µs (SLO: <400.000µs 📉 -35.7%) vs baseline: -0.7% Memory: ✅ 42.664MB (SLO: <43.500MB 🟡 -1.9%) vs baseline: +4.7% ✅ bytearray_extend_aspectTime: ✅ 650.356µs (SLO: <800.000µs 📉 -18.7%) vs baseline: -0.6% Memory: ✅ 42.585MB (SLO: <43.500MB -2.1%) vs baseline: +4.7% ✅ bytearray_extend_noaspectTime: ✅ 268.006µs (SLO: <400.000µs 📉 -33.0%) vs baseline: ~same Memory: ✅ 42.625MB (SLO: <43.500MB -2.0%) vs baseline: +4.9% ✅ bytearray_noaspectTime: ✅ 139.656µs (SLO: <300.000µs 📉 -53.4%) vs baseline: +0.9% Memory: ✅ 42.664MB (SLO: <43.500MB 🟡 -1.9%) vs baseline: +5.2% ✅ bytes_aspectTime: ✅ 223.037µs (SLO: <300.000µs 📉 -25.7%) vs baseline: -1.0% Memory: ✅ 42.605MB (SLO: <43.500MB -2.1%) vs baseline: +4.8% ✅ bytes_noaspectTime: ✅ 133.773µs (SLO: <200.000µs 📉 -33.1%) vs baseline: -0.3% Memory: ✅ 42.664MB (SLO: <43.500MB 🟡 -1.9%) vs baseline: +5.0% ✅ bytesio_aspectTime: ✅ 3.845ms (SLO: <5.000ms 📉 -23.1%) vs baseline: +0.6% Memory: ✅ 42.644MB (SLO: <43.500MB 🟡 -2.0%) vs baseline: +4.9% ✅ bytesio_noaspectTime: ✅ 320.795µs (SLO: <420.000µs 📉 -23.6%) vs baseline: ~same Memory: ✅ 42.684MB (SLO: <43.500MB 🟡 -1.9%) vs baseline: +4.8% ✅ capitalize_aspectTime: ✅ 89.964µs (SLO: <300.000µs 📉 -70.0%) vs baseline: -0.4% Memory: ✅ 42.684MB (SLO: <43.500MB 🟡 -1.9%) vs baseline: +5.1% ✅ capitalize_noaspectTime: ✅ 252.710µs (SLO: <300.000µs 📉 -15.8%) vs baseline: +1.0% Memory: ✅ 42.605MB (SLO: <43.500MB -2.1%) vs baseline: +4.6% ✅ casefold_aspectTime: ✅ 93.188µs (SLO: <500.000µs 📉 -81.4%) vs baseline: +3.2% Memory: ✅ 42.585MB (SLO: <43.500MB -2.1%) vs baseline: +4.7% ✅ casefold_noaspectTime: ✅ 308.588µs (SLO: <500.000µs 📉 -38.3%) vs baseline: -0.3% Memory: ✅ 42.723MB (SLO: <43.500MB 🟡 -1.8%) vs baseline: +4.8% ✅ decode_aspectTime: ✅ 87.199µs (SLO: <100.000µs 📉 -12.8%) vs baseline: -0.3% Memory: ✅ 42.625MB (SLO: <43.500MB -2.0%) vs baseline: +4.8% ✅ decode_noaspectTime: ✅ 153.514µs (SLO: <210.000µs 📉 -26.9%) vs baseline: +0.8% Memory: ✅ 42.625MB (SLO: <43.500MB -2.0%) vs baseline: +5.1% ✅ encode_aspectTime: ✅ 85.055µs (SLO: <200.000µs 📉 -57.5%) vs baseline: -0.1% Memory: ✅ 42.585MB (SLO: <43.500MB -2.1%) vs baseline: +4.6% ✅ encode_noaspectTime: ✅ 138.932µs (SLO: <200.000µs 📉 -30.5%) vs baseline: +0.5% Memory: ✅ 42.684MB (SLO: <43.500MB 🟡 -1.9%) vs baseline: +4.9% ✅ format_aspectTime: ✅ 14.785ms (SLO: <19.200ms 📉 -23.0%) vs baseline: +0.6% Memory: ✅ 42.802MB (SLO: <43.250MB 🟡 -1.0%) vs baseline: +4.7% ✅ format_map_aspectTime: ✅ 16.456ms (SLO: <21.500ms 📉 -23.5%) vs baseline: ~same Memory: ✅ 42.703MB (SLO: <43.500MB 🟡 -1.8%) vs baseline: +4.7% ✅ format_map_noaspectTime: ✅ 367.594µs (SLO: <500.000µs 📉 -26.5%) vs baseline: ~same Memory: ✅ 42.664MB (SLO: <43.250MB 🟡 -1.4%) vs baseline: +4.8% ✅ format_noaspectTime: ✅ 308.068µs (SLO: <500.000µs 📉 -38.4%) vs baseline: +0.6% Memory: ✅ 42.566MB (SLO: <43.250MB 🟡 -1.6%) vs baseline: +4.7% ✅ index_aspectTime: ✅ 123.150µs (SLO: <300.000µs 📉 -58.9%) vs baseline: -0.3% Memory: ✅ 42.644MB (SLO: <43.250MB 🟡 -1.4%) vs baseline: +5.0% ✅ index_noaspectTime: ✅ 40.906µs (SLO: <300.000µs 📉 -86.4%) vs baseline: +0.3% Memory: ✅ 42.644MB (SLO: <43.500MB 🟡 -2.0%) vs baseline: +5.0% ✅ join_aspectTime: ✅ 219.135µs (SLO: <300.000µs 📉 -27.0%) vs baseline: +0.2% Memory: ✅ 42.625MB (SLO: <43.500MB -2.0%) vs baseline: +4.7% ✅ join_noaspectTime: ✅ 150.849µs (SLO: <300.000µs 📉 -49.7%) vs baseline: +0.8% Memory: ✅ 42.664MB (SLO: <43.250MB 🟡 -1.4%) vs baseline: +4.9% ✅ ljust_aspectTime: ✅ 497.827µs (SLO: <700.000µs 📉 -28.9%) vs baseline: +0.3% Memory: ✅ 42.625MB (SLO: <43.250MB 🟡 -1.4%) vs baseline: +4.8% ✅ ljust_noaspectTime: ✅ 253.881µs (SLO: <300.000µs 📉 -15.4%) vs baseline: -0.2% Memory: ✅ 42.605MB (SLO: <43.250MB 🟡 -1.5%) vs baseline: +4.7% ✅ lower_aspectTime: ✅ 302.929µs (SLO: <500.000µs 📉 -39.4%) vs baseline: -0.5% Memory: ✅ 42.585MB (SLO: <43.500MB -2.1%) vs baseline: +4.7% ✅ lower_noaspectTime: ✅ 234.322µs (SLO: <300.000µs 📉 -21.9%) vs baseline: +0.3% Memory: ✅ 42.644MB (SLO: <43.250MB 🟡 -1.4%) vs baseline: +5.0% ✅ lstrip_aspectTime: ✅ 0.269ms (SLO: <3.000ms 📉 -91.0%) vs baseline: -0.5% Memory: ✅ 42.605MB (SLO: <43.250MB 🟡 -1.5%) vs baseline: +5.0% ✅ lstrip_noaspectTime: ✅ 0.201ms (SLO: <3.000ms 📉 -93.3%) vs baseline: 📈 +13.1% Memory: ✅ 42.664MB (SLO: <43.500MB 🟡 -1.9%) vs baseline: +4.9% ✅ modulo_aspectTime: ✅ 14.562ms (SLO: <18.750ms 📉 -22.3%) vs baseline: +1.3% Memory: ✅ 42.762MB (SLO: <43.500MB 🟡 -1.7%) vs baseline: +4.5% ✅ modulo_aspect_for_bytearray_bytearrayTime: ✅ 14.894ms (SLO: <19.350ms 📉 -23.0%) vs baseline: +0.2% Memory: ✅ 42.880MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +5.1% ✅ modulo_aspect_for_bytesTime: ✅ 14.533ms (SLO: <18.900ms 📉 -23.1%) vs baseline: +0.1% Memory: ✅ 42.861MB (SLO: <43.500MB 🟡 -1.5%) vs baseline: +5.0% ✅ modulo_aspect_for_bytes_bytearrayTime: ✅ 14.695ms (SLO: <19.150ms 📉 -23.3%) vs baseline: ~same Memory: ✅ 42.861MB (SLO: <43.500MB 🟡 -1.5%) vs baseline: +5.1% ✅ modulo_noaspectTime: ✅ 0.358ms (SLO: <3.000ms 📉 -88.1%) vs baseline: -0.5% Memory: ✅ 42.703MB (SLO: <43.500MB 🟡 -1.8%) vs baseline: +5.1% ✅ replace_aspectTime: ✅ 18.446ms (SLO: <24.000ms 📉 -23.1%) vs baseline: -0.2% Memory: ✅ 42.802MB (SLO: <44.000MB -2.7%) vs baseline: +4.8% ✅ replace_noaspectTime: ✅ 283.518µs (SLO: <300.000µs -5.5%) vs baseline: -1.3% Memory: ✅ 42.684MB (SLO: <43.500MB 🟡 -1.9%) vs baseline: +4.9% ✅ repr_aspectTime: ✅ 318.149µs (SLO: <420.000µs 📉 -24.3%) vs baseline: +0.8% Memory: ✅ 42.625MB (SLO: <43.500MB -2.0%) vs baseline: +4.9% ✅ repr_noaspectTime: ✅ 46.455µs (SLO: <90.000µs 📉 -48.4%) vs baseline: -1.1% Memory: ✅ 42.605MB (SLO: <43.500MB -2.1%) vs baseline: +4.9% ✅ rstrip_aspectTime: ✅ 378.031µs (SLO: <500.000µs 📉 -24.4%) vs baseline: -0.5% Memory: ✅ 42.684MB (SLO: <43.500MB 🟡 -1.9%) vs baseline: +5.0% ✅ rstrip_noaspectTime: ✅ 180.375µs (SLO: <300.000µs 📉 -39.9%) vs baseline: -1.0% Memory: ✅ 42.585MB (SLO: <43.500MB -2.1%) vs baseline: +4.7% ✅ slice_aspectTime: ✅ 183.213µs (SLO: <300.000µs 📉 -38.9%) vs baseline: +0.7% Memory: ✅ 42.723MB (SLO: <43.500MB 🟡 -1.8%) vs baseline: +5.0% ✅ slice_noaspectTime: ✅ 54.148µs (SLO: <90.000µs 📉 -39.8%) vs baseline: -0.3% Memory: ✅ 42.585MB (SLO: <43.500MB -2.1%) vs baseline: +5.0% ✅ stringio_aspectTime: ✅ 3.881ms (SLO: <5.000ms 📉 -22.4%) vs baseline: -0.5% Memory: ✅ 42.644MB (SLO: <43.500MB 🟡 -2.0%) vs baseline: +5.0% ✅ stringio_noaspectTime: ✅ 385.415µs (SLO: <500.000µs 📉 -22.9%) vs baseline: +8.4% Memory: ✅ 42.684MB (SLO: <43.500MB 🟡 -1.9%) vs baseline: +5.0% ✅ strip_aspectTime: ✅ 271.939µs (SLO: <350.000µs 📉 -22.3%) vs baseline: +0.5% Memory: ✅ 42.625MB (SLO: <43.500MB -2.0%) vs baseline: +4.7% ✅ strip_noaspectTime: ✅ 178.720µs (SLO: <240.000µs 📉 -25.5%) vs baseline: -0.4% Memory: ✅ 42.703MB (SLO: <43.500MB 🟡 -1.8%) vs baseline: +4.9% ✅ swapcase_aspectTime: ✅ 341.356µs (SLO: <500.000µs 📉 -31.7%) vs baseline: -0.5% Memory: ✅ 42.605MB (SLO: <43.500MB -2.1%) vs baseline: +4.9% ✅ swapcase_noaspectTime: ✅ 273.013µs (SLO: <400.000µs 📉 -31.7%) vs baseline: +0.8% Memory: ✅ 42.625MB (SLO: <43.500MB -2.0%) vs baseline: +5.0% ✅ title_aspectTime: ✅ 333.920µs (SLO: <500.000µs 📉 -33.2%) vs baseline: +1.0% Memory: ✅ 42.703MB (SLO: <43.000MB 🟡 -0.7%) vs baseline: +4.9% ✅ title_noaspectTime: ✅ 258.270µs (SLO: <400.000µs 📉 -35.4%) vs baseline: ~same Memory: ✅ 42.743MB (SLO: <43.500MB 🟡 -1.7%) vs baseline: +5.1% ✅ translate_aspectTime: ✅ 500.794µs (SLO: <700.000µs 📉 -28.5%) vs baseline: +0.7% Memory: ✅ 42.605MB (SLO: <43.500MB -2.1%) vs baseline: +4.7% ✅ translate_noaspectTime: ✅ 425.093µs (SLO: <500.000µs 📉 -15.0%) vs baseline: -1.2% Memory: ✅ 42.605MB (SLO: <43.500MB -2.1%) vs baseline: +4.8% ✅ upper_aspectTime: ✅ 310.255µs (SLO: <500.000µs 📉 -37.9%) vs baseline: +1.7% Memory: ✅ 42.684MB (SLO: <43.500MB 🟡 -1.9%) vs baseline: +5.0% ✅ upper_noaspectTime: ✅ 234.825µs (SLO: <400.000µs 📉 -41.3%) vs baseline: +0.2% Memory: ✅ 42.605MB (SLO: <43.500MB -2.1%) vs baseline: +4.7% 📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 505.938µs (SLO: <700.000µs 📉 -27.7%) vs baseline: 📈 +20.4% Memory: ✅ 42.428MB (SLO: <43.500MB -2.5%) vs baseline: +4.6% ✅ ospathbasename_noaspectTime: ✅ 430.242µs (SLO: <700.000µs 📉 -38.5%) vs baseline: +0.7% Memory: ✅ 42.585MB (SLO: <43.500MB -2.1%) vs baseline: +5.0% ✅ ospathjoin_aspectTime: ✅ 623.502µs (SLO: <700.000µs 📉 -10.9%) vs baseline: +0.3% Memory: ✅ 42.349MB (SLO: <43.500MB -2.6%) vs baseline: +4.7% ✅ ospathjoin_noaspectTime: ✅ 627.326µs (SLO: <700.000µs 📉 -10.4%) vs baseline: -0.3% Memory: ✅ 42.448MB (SLO: <43.500MB -2.4%) vs baseline: +4.9% ✅ ospathnormcase_aspectTime: ✅ 350.035µs (SLO: <700.000µs 📉 -50.0%) vs baseline: +0.8% Memory: ✅ 42.566MB (SLO: <43.500MB -2.1%) vs baseline: +5.3% ✅ ospathnormcase_noaspectTime: ✅ 359.123µs (SLO: <700.000µs 📉 -48.7%) vs baseline: +0.6% Memory: ✅ 42.605MB (SLO: <43.500MB -2.1%) vs baseline: +5.2% ✅ ospathsplit_aspectTime: ✅ 489.439µs (SLO: <700.000µs 📉 -30.1%) vs baseline: +0.5% Memory: ✅ 42.448MB (SLO: <43.500MB -2.4%) vs baseline: +4.6% ✅ ospathsplit_noaspectTime: ✅ 495.945µs (SLO: <700.000µs 📉 -29.2%) vs baseline: +0.4% Memory: ✅ 42.507MB (SLO: <43.500MB -2.3%) vs baseline: +5.2% ✅ ospathsplitdrive_aspectTime: ✅ 373.634µs (SLO: <700.000µs 📉 -46.6%) vs baseline: +0.4% Memory: ✅ 42.349MB (SLO: <43.500MB -2.6%) vs baseline: +4.5% ✅ ospathsplitdrive_noaspectTime: ✅ 72.773µs (SLO: <700.000µs 📉 -89.6%) vs baseline: -0.6% Memory: ✅ 42.389MB (SLO: <43.500MB -2.6%) vs baseline: +4.7% ✅ ospathsplitext_aspectTime: ✅ 462.458µs (SLO: <700.000µs 📉 -33.9%) vs baseline: +1.8% Memory: ✅ 42.448MB (SLO: <43.500MB -2.4%) vs baseline: +4.8% ✅ ospathsplitext_noaspectTime: ✅ 464.964µs (SLO: <700.000µs 📉 -33.6%) vs baseline: +1.0% Memory: ✅ 42.310MB (SLO: <43.500MB -2.7%) vs baseline: +4.3% 📈 telemetryaddmetric - 30/30✅ 1-count-metric-1-timesTime: ✅ 3.402µs (SLO: <20.000µs 📉 -83.0%) vs baseline: 📈 +14.2% Memory: ✅ 34.918MB (SLO: <35.500MB 🟡 -1.6%) vs baseline: +4.9% ✅ 1-count-metrics-100-timesTime: ✅ 197.527µs (SLO: <220.000µs 📉 -10.2%) vs baseline: -1.0% Memory: ✅ 34.937MB (SLO: <35.500MB 🟡 -1.6%) vs baseline: +4.9% ✅ 1-distribution-metric-1-timesTime: ✅ 3.331µs (SLO: <20.000µs 📉 -83.3%) vs baseline: -0.7% Memory: ✅ 34.859MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +4.4% ✅ 1-distribution-metrics-100-timesTime: ✅ 212.272µs (SLO: <230.000µs -7.7%) vs baseline: -0.9% Memory: ✅ 35.055MB (SLO: <35.500MB 🟡 -1.3%) vs baseline: +5.2% ✅ 1-gauge-metric-1-timesTime: ✅ 2.201µs (SLO: <20.000µs 📉 -89.0%) vs baseline: +0.4% Memory: ✅ 34.937MB (SLO: <35.500MB 🟡 -1.6%) vs baseline: +4.7% ✅ 1-gauge-metrics-100-timesTime: ✅ 136.434µs (SLO: <150.000µs -9.0%) vs baseline: ~same Memory: ✅ 34.918MB (SLO: <35.500MB 🟡 -1.6%) vs baseline: +5.0% ✅ 1-rate-metric-1-timesTime: ✅ 3.148µs (SLO: <20.000µs 📉 -84.3%) vs baseline: -0.3% Memory: ✅ 34.957MB (SLO: <35.500MB 🟡 -1.5%) vs baseline: +5.3% ✅ 1-rate-metrics-100-timesTime: ✅ 211.147µs (SLO: <250.000µs 📉 -15.5%) vs baseline: -0.7% Memory: ✅ 35.016MB (SLO: <35.500MB 🟡 -1.4%) vs baseline: +5.0% ✅ 100-count-metrics-100-timesTime: ✅ 20.019ms (SLO: <22.000ms -9.0%) vs baseline: +0.9% Memory: ✅ 34.898MB (SLO: <35.500MB 🟡 -1.7%) vs baseline: +4.9% ✅ 100-distribution-metrics-100-timesTime: ✅ 2.216ms (SLO: <2.550ms 📉 -13.1%) vs baseline: -0.5% Memory: ✅ 34.918MB (SLO: <35.500MB 🟡 -1.6%) vs baseline: +4.6% ✅ 100-gauge-metrics-100-timesTime: ✅ 1.401ms (SLO: <1.550ms -9.6%) vs baseline: -0.4% Memory: ✅ 34.878MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +4.4% ✅ 100-rate-metrics-100-timesTime: ✅ 2.169ms (SLO: <2.550ms 📉 -14.9%) vs baseline: +0.2% Memory: ✅ 34.957MB (SLO: <35.500MB 🟡 -1.5%) vs baseline: +4.9% ✅ flush-1-metricTime: ✅ 4.516µs (SLO: <20.000µs 📉 -77.4%) vs baseline: -1.2% Memory: ✅ 35.252MB (SLO: <35.500MB 🟡 -0.7%) vs baseline: +4.9% ✅ flush-100-metricsTime: ✅ 174.075µs (SLO: <250.000µs 📉 -30.4%) vs baseline: -0.7% Memory: ✅ 35.350MB (SLO: <35.500MB 🟡 -0.4%) vs baseline: +5.2% ✅ flush-1000-metricsTime: ✅ 2.179ms (SLO: <2.500ms 📉 -12.8%) vs baseline: -0.5% Memory: ✅ 36.117MB (SLO: <36.500MB 🟡 -1.0%) vs baseline: +4.9% 🟡 Near SLO Breach (15 suites)🟡 coreapiscenario - 10/10 (1 unstable)
|
Description
See test runs
Testing
Risks
Additional Notes