-
Notifications
You must be signed in to change notification settings - Fork 477
test(profiling): unflake test_gunicorn, again [backport 4.1]
#15942
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test(profiling): unflake test_gunicorn, again [backport 4.1]
#15942
Conversation
test_gunicorn, again
|
|
Bootstrap import analysisComparison of import times between this PR and base. SummaryThe average import time from this PR is: 247 ± 2 ms. The average import time from base is: 251 ± 2 ms. The import time difference between this PR and base is: -4.7 ± 0.1 ms. Import time breakdownThe following import paths have shrunk:
|
test_gunicorn, againtest_gunicorn, again [backport 4.2]
test_gunicorn, again [backport 4.2]test_gunicorn, again [backport 4.1]
Performance SLOsComparing candidate kowalski/test-profiling-unflake-test_gunicorn-again (7c5c7bf) with baseline 4.1 (b412b81) 📈 Performance Regressions (3 suites)📈 iastaspects - 118/118✅ add_aspectTime: ✅ 18.217µs (SLO: <20.000µs -8.9%) vs baseline: 📈 +20.8% Memory: ✅ 42.841MB (SLO: <43.250MB 🟡 -0.9%) vs baseline: +4.6% ✅ add_inplace_aspectTime: ✅ 15.102µs (SLO: <20.000µs 📉 -24.5%) vs baseline: ~same Memory: ✅ 42.939MB (SLO: <43.250MB 🟡 -0.7%) vs baseline: +4.8% ✅ add_inplace_noaspectTime: ✅ 0.338µs (SLO: <10.000µs 📉 -96.6%) vs baseline: -2.1% Memory: ✅ 42.664MB (SLO: <43.500MB 🟡 -1.9%) vs baseline: +4.2% ✅ add_noaspectTime: ✅ 0.548µs (SLO: <10.000µs 📉 -94.5%) vs baseline: -0.2% Memory: ✅ 42.939MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.8% ✅ bytearray_aspectTime: ✅ 18.019µs (SLO: <30.000µs 📉 -39.9%) vs baseline: ~same Memory: ✅ 42.880MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +4.7% ✅ bytearray_extend_aspectTime: ✅ 23.718µs (SLO: <30.000µs 📉 -20.9%) vs baseline: -0.4% Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.8% ✅ bytearray_extend_noaspectTime: ✅ 2.676µs (SLO: <10.000µs 📉 -73.2%) vs baseline: +0.7% Memory: ✅ 42.841MB (SLO: <43.500MB 🟡 -1.5%) vs baseline: +4.8% ✅ bytearray_noaspectTime: ✅ 1.461µs (SLO: <10.000µs 📉 -85.4%) vs baseline: -0.7% Memory: ✅ 42.684MB (SLO: <43.500MB 🟡 -1.9%) vs baseline: +4.2% ✅ bytes_aspectTime: ✅ 16.805µs (SLO: <20.000µs 📉 -16.0%) vs baseline: +0.2% Memory: ✅ 42.939MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.7% ✅ bytes_noaspectTime: ✅ 1.411µs (SLO: <10.000µs 📉 -85.9%) vs baseline: +0.8% Memory: ✅ 42.743MB (SLO: <43.500MB 🟡 -1.7%) vs baseline: +4.3% ✅ bytesio_aspectTime: ✅ 55.184µs (SLO: <70.000µs 📉 -21.2%) vs baseline: -0.7% Memory: ✅ 43.018MB (SLO: <43.500MB 🟡 -1.1%) vs baseline: +4.9% ✅ bytesio_noaspectTime: ✅ 3.209µs (SLO: <10.000µs 📉 -67.9%) vs baseline: -0.5% Memory: ✅ 42.703MB (SLO: <43.500MB 🟡 -1.8%) vs baseline: +4.0% ✅ capitalize_aspectTime: ✅ 14.799µs (SLO: <20.000µs 📉 -26.0%) vs baseline: -0.3% Memory: ✅ 42.979MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.6% ✅ capitalize_noaspectTime: ✅ 2.606µs (SLO: <10.000µs 📉 -73.9%) vs baseline: +2.8% Memory: ✅ 42.802MB (SLO: <43.500MB 🟡 -1.6%) vs baseline: +4.5% ✅ casefold_aspectTime: ✅ 14.661µs (SLO: <20.000µs 📉 -26.7%) vs baseline: -0.6% Memory: ✅ 42.979MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.9% ✅ casefold_noaspectTime: ✅ 3.084µs (SLO: <10.000µs 📉 -69.2%) vs baseline: ~same Memory: ✅ 42.585MB (SLO: <43.500MB -2.1%) vs baseline: +3.9% ✅ decode_aspectTime: ✅ 15.668µs (SLO: <30.000µs 📉 -47.8%) vs baseline: +0.5% Memory: ✅ 42.939MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.8% ✅ decode_noaspectTime: ✅ 1.598µs (SLO: <10.000µs 📉 -84.0%) vs baseline: -0.3% Memory: ✅ 42.821MB (SLO: <43.500MB 🟡 -1.6%) vs baseline: +4.5% ✅ encode_aspectTime: ✅ 18.403µs (SLO: <30.000µs 📉 -38.7%) vs baseline: 📈 +21.9% Memory: ✅ 42.979MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +5.0% ✅ encode_noaspectTime: ✅ 1.480µs (SLO: <10.000µs 📉 -85.2%) vs baseline: ~same Memory: ✅ 42.861MB (SLO: <43.500MB 🟡 -1.5%) vs baseline: +4.9% ✅ format_aspectTime: ✅ 171.089µs (SLO: <200.000µs 📉 -14.5%) vs baseline: -0.3% Memory: ✅ 43.096MB (SLO: <43.250MB 🟡 -0.4%) vs baseline: +4.8% ✅ format_map_aspectTime: ✅ 192.095µs (SLO: <200.000µs -4.0%) vs baseline: +0.3% Memory: ✅ 43.037MB (SLO: <43.500MB 🟡 -1.1%) vs baseline: +4.7% ✅ format_map_noaspectTime: ✅ 3.772µs (SLO: <10.000µs 📉 -62.3%) vs baseline: +0.6% Memory: ✅ 42.762MB (SLO: <43.250MB 🟡 -1.1%) vs baseline: +4.3% ✅ format_noaspectTime: ✅ 3.175µs (SLO: <10.000µs 📉 -68.3%) vs baseline: +1.0% Memory: ✅ 42.684MB (SLO: <43.250MB 🟡 -1.3%) vs baseline: +4.1% ✅ index_aspectTime: ✅ 15.242µs (SLO: <20.000µs 📉 -23.8%) vs baseline: -0.3% Memory: ✅ 43.018MB (SLO: <43.250MB 🟡 -0.5%) vs baseline: +4.8% ✅ index_noaspectTime: ✅ 0.465µs (SLO: <10.000µs 📉 -95.4%) vs baseline: -0.2% Memory: ✅ 42.743MB (SLO: <43.500MB 🟡 -1.7%) vs baseline: +4.3% ✅ join_aspectTime: ✅ 17.041µs (SLO: <20.000µs 📉 -14.8%) vs baseline: -0.3% Memory: ✅ 42.880MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +4.5% ✅ join_noaspectTime: ✅ 1.505µs (SLO: <10.000µs 📉 -85.0%) vs baseline: -0.1% Memory: ✅ 42.998MB (SLO: <43.250MB 🟡 -0.6%) vs baseline: +5.0% ✅ ljust_aspectTime: ✅ 20.827µs (SLO: <30.000µs 📉 -30.6%) vs baseline: -0.9% Memory: ✅ 42.998MB (SLO: <43.250MB 🟡 -0.6%) vs baseline: +4.9% ✅ ljust_noaspectTime: ✅ 2.689µs (SLO: <10.000µs 📉 -73.1%) vs baseline: +1.5% Memory: ✅ 42.802MB (SLO: <43.250MB 🟡 -1.0%) vs baseline: +4.7% ✅ lower_aspectTime: ✅ 17.975µs (SLO: <30.000µs 📉 -40.1%) vs baseline: -0.3% Memory: ✅ 42.979MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.9% ✅ lower_noaspectTime: ✅ 2.386µs (SLO: <10.000µs 📉 -76.1%) vs baseline: +0.2% Memory: ✅ 42.585MB (SLO: <43.250MB 🟡 -1.5%) vs baseline: +3.7% ✅ lstrip_aspectTime: ✅ 17.705µs (SLO: <30.000µs 📉 -41.0%) vs baseline: -0.3% Memory: ✅ 43.018MB (SLO: <43.250MB 🟡 -0.5%) vs baseline: +4.6% ✅ lstrip_noaspectTime: ✅ 1.836µs (SLO: <10.000µs 📉 -81.6%) vs baseline: -0.2% Memory: ✅ 42.664MB (SLO: <43.500MB 🟡 -1.9%) vs baseline: +4.0% ✅ modulo_aspectTime: ✅ 166.680µs (SLO: <200.000µs 📉 -16.7%) vs baseline: -0.2% Memory: ✅ 43.136MB (SLO: <43.500MB 🟡 -0.8%) vs baseline: +5.0% ✅ modulo_aspect_for_bytearray_bytearrayTime: ✅ 174.948µs (SLO: <200.000µs 📉 -12.5%) vs baseline: +0.2% Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.3% ✅ modulo_aspect_for_bytesTime: ✅ 168.521µs (SLO: <200.000µs 📉 -15.7%) vs baseline: +0.2% Memory: ✅ 42.998MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.6% ✅ modulo_aspect_for_bytes_bytearrayTime: ✅ 171.678µs (SLO: <200.000µs 📉 -14.2%) vs baseline: -0.3% Memory: ✅ 42.998MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.6% ✅ modulo_noaspectTime: ✅ 3.711µs (SLO: <10.000µs 📉 -62.9%) vs baseline: -0.8% Memory: ✅ 42.625MB (SLO: <43.500MB -2.0%) vs baseline: +3.9% ✅ replace_aspectTime: ✅ 214.163µs (SLO: <300.000µs 📉 -28.6%) vs baseline: -0.2% Memory: ✅ 42.959MB (SLO: <44.000MB -2.4%) vs baseline: +4.5% ✅ replace_noaspectTime: ✅ 5.219µs (SLO: <10.000µs 📉 -47.8%) vs baseline: ~same Memory: ✅ 42.605MB (SLO: <43.500MB -2.1%) vs baseline: +4.0% ✅ repr_aspectTime: ✅ 1.397µs (SLO: <10.000µs 📉 -86.0%) vs baseline: +0.7% Memory: ✅ 42.664MB (SLO: <43.500MB 🟡 -1.9%) vs baseline: +4.2% ✅ repr_noaspectTime: ✅ 0.527µs (SLO: <10.000µs 📉 -94.7%) vs baseline: +0.6% Memory: ✅ 42.880MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +4.6% ✅ rstrip_aspectTime: ✅ 23.169µs (SLO: <30.000µs 📉 -22.8%) vs baseline: 📈 +20.1% Memory: ✅ 43.057MB (SLO: <43.500MB 🟡 -1.0%) vs baseline: +5.1% ✅ rstrip_noaspectTime: ✅ 1.874µs (SLO: <10.000µs 📉 -81.3%) vs baseline: -0.2% Memory: ✅ 42.605MB (SLO: <43.500MB -2.1%) vs baseline: +4.0% ✅ slice_aspectTime: ✅ 16.016µs (SLO: <20.000µs 📉 -19.9%) vs baseline: -0.7% Memory: ✅ 42.900MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +4.7% ✅ slice_noaspectTime: ✅ 0.603µs (SLO: <10.000µs 📉 -94.0%) vs baseline: -0.3% Memory: ✅ 42.821MB (SLO: <43.500MB 🟡 -1.6%) vs baseline: +4.9% ✅ stringio_aspectTime: ✅ 54.559µs (SLO: <80.000µs 📉 -31.8%) vs baseline: +0.3% Memory: ✅ 42.939MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.6% ✅ stringio_noaspectTime: ✅ 3.550µs (SLO: <10.000µs 📉 -64.5%) vs baseline: -0.2% Memory: ✅ 42.743MB (SLO: <43.500MB 🟡 -1.7%) vs baseline: +4.3% ✅ strip_aspectTime: ✅ 17.764µs (SLO: <20.000µs 📉 -11.2%) vs baseline: +0.2% Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.8% ✅ strip_noaspectTime: ✅ 1.842µs (SLO: <10.000µs 📉 -81.6%) vs baseline: +0.7% Memory: ✅ 42.743MB (SLO: <43.500MB 🟡 -1.7%) vs baseline: +4.3% ✅ swapcase_aspectTime: ✅ 18.520µs (SLO: <30.000µs 📉 -38.3%) vs baseline: -0.1% Memory: ✅ 42.979MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +5.1% ✅ swapcase_noaspectTime: ✅ 2.786µs (SLO: <10.000µs 📉 -72.1%) vs baseline: +0.8% Memory: ✅ 42.743MB (SLO: <43.500MB 🟡 -1.7%) vs baseline: +4.4% ✅ title_aspectTime: ✅ 18.189µs (SLO: <30.000µs 📉 -39.4%) vs baseline: -0.6% Memory: ✅ 42.920MB (SLO: <43.000MB 🟡 -0.2%) vs baseline: +4.8% ✅ title_noaspectTime: ✅ 2.622µs (SLO: <10.000µs 📉 -73.8%) vs baseline: -1.4% Memory: ✅ 42.802MB (SLO: <43.500MB 🟡 -1.6%) vs baseline: +4.4% ✅ translate_aspectTime: ✅ 20.632µs (SLO: <30.000µs 📉 -31.2%) vs baseline: ~same Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.6% ✅ translate_noaspectTime: ✅ 4.238µs (SLO: <10.000µs 📉 -57.6%) vs baseline: -0.4% Memory: ✅ 42.625MB (SLO: <43.500MB -2.0%) vs baseline: +4.2% ✅ upper_aspectTime: ✅ 18.018µs (SLO: <30.000µs 📉 -39.9%) vs baseline: ~same Memory: ✅ 42.979MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.8% ✅ upper_noaspectTime: ✅ 2.385µs (SLO: <10.000µs 📉 -76.1%) vs baseline: +0.9% Memory: ✅ 42.743MB (SLO: <43.500MB 🟡 -1.7%) vs baseline: +4.3% 📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 5.157µs (SLO: <10.000µs 📉 -48.4%) vs baseline: 📈 +21.7% Memory: ✅ 41.366MB (SLO: <43.500MB -4.9%) vs baseline: +4.6% ✅ ospathbasename_noaspectTime: ✅ 4.371µs (SLO: <10.000µs 📉 -56.3%) vs baseline: +1.1% Memory: ✅ 41.406MB (SLO: <43.500MB -4.8%) vs baseline: +4.9% ✅ ospathjoin_aspectTime: ✅ 6.151µs (SLO: <10.000µs 📉 -38.5%) vs baseline: -0.4% Memory: ✅ 41.445MB (SLO: <43.500MB -4.7%) vs baseline: +5.2% ✅ ospathjoin_noaspectTime: ✅ 6.175µs (SLO: <10.000µs 📉 -38.3%) vs baseline: -0.4% Memory: ✅ 41.425MB (SLO: <43.500MB -4.8%) vs baseline: +4.8% ✅ ospathnormcase_aspectTime: ✅ 3.508µs (SLO: <10.000µs 📉 -64.9%) vs baseline: +0.6% Memory: ✅ 41.406MB (SLO: <43.500MB -4.8%) vs baseline: +4.8% ✅ ospathnormcase_noaspectTime: ✅ 3.591µs (SLO: <10.000µs 📉 -64.1%) vs baseline: ~same Memory: ✅ 41.406MB (SLO: <43.500MB -4.8%) vs baseline: +4.9% ✅ ospathsplit_aspectTime: ✅ 4.860µs (SLO: <10.000µs 📉 -51.4%) vs baseline: +0.5% Memory: ✅ 41.386MB (SLO: <43.500MB -4.9%) vs baseline: +4.7% ✅ ospathsplit_noaspectTime: ✅ 4.958µs (SLO: <10.000µs 📉 -50.4%) vs baseline: -0.2% Memory: ✅ 41.465MB (SLO: <43.500MB -4.7%) vs baseline: +5.1% ✅ ospathsplitdrive_aspectTime: ✅ 3.769µs (SLO: <10.000µs 📉 -62.3%) vs baseline: +0.8% Memory: ✅ 41.386MB (SLO: <43.500MB -4.9%) vs baseline: +4.6% ✅ ospathsplitdrive_noaspectTime: ✅ 0.756µs (SLO: <10.000µs 📉 -92.4%) vs baseline: -0.5% Memory: ✅ 41.406MB (SLO: <43.500MB -4.8%) vs baseline: +4.9% ✅ ospathsplitext_aspectTime: ✅ 4.643µs (SLO: <10.000µs 📉 -53.6%) vs baseline: +0.7% Memory: ✅ 41.386MB (SLO: <43.500MB -4.9%) vs baseline: +4.9% ✅ ospathsplitext_noaspectTime: ✅ 4.715µs (SLO: <10.000µs 📉 -52.9%) vs baseline: +1.0% Memory: ✅ 41.327MB (SLO: <43.500MB -5.0%) vs baseline: +4.7% 📈 telemetryaddmetric - 30/30✅ 1-count-metric-1-timesTime: ✅ 3.396µs (SLO: <20.000µs 📉 -83.0%) vs baseline: 📈 +10.9% Memory: ✅ 34.878MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +4.7% ✅ 1-count-metrics-100-timesTime: ✅ 200.644µs (SLO: <220.000µs -8.8%) vs baseline: ~same Memory: ✅ 34.819MB (SLO: <35.500MB 🟡 -1.9%) vs baseline: +4.8% ✅ 1-distribution-metric-1-timesTime: ✅ 3.389µs (SLO: <20.000µs 📉 -83.1%) vs baseline: -0.4% Memory: ✅ 34.839MB (SLO: <35.500MB 🟡 -1.9%) vs baseline: +4.6% ✅ 1-distribution-metrics-100-timesTime: ✅ 214.381µs (SLO: <230.000µs -6.8%) vs baseline: -0.4% Memory: ✅ 34.878MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +4.7% ✅ 1-gauge-metric-1-timesTime: ✅ 2.240µs (SLO: <20.000µs 📉 -88.8%) vs baseline: -0.8% Memory: ✅ 34.859MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +4.7% ✅ 1-gauge-metrics-100-timesTime: ✅ 136.308µs (SLO: <150.000µs -9.1%) vs baseline: -0.4% Memory: ✅ 34.878MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +4.8% ✅ 1-rate-metric-1-timesTime: ✅ 3.184µs (SLO: <20.000µs 📉 -84.1%) vs baseline: -0.6% Memory: ✅ 34.957MB (SLO: <35.500MB 🟡 -1.5%) vs baseline: +4.9% ✅ 1-rate-metrics-100-timesTime: ✅ 214.173µs (SLO: <250.000µs 📉 -14.3%) vs baseline: -0.5% Memory: ✅ 34.721MB (SLO: <35.500MB -2.2%) vs baseline: +4.3% ✅ 100-count-metrics-100-timesTime: ✅ 20.109ms (SLO: <22.000ms -8.6%) vs baseline: -0.6% Memory: ✅ 34.741MB (SLO: <35.500MB -2.1%) vs baseline: +4.5% ✅ 100-distribution-metrics-100-timesTime: ✅ 2.281ms (SLO: <2.550ms 📉 -10.6%) vs baseline: +0.1% Memory: ✅ 34.800MB (SLO: <35.500MB 🟡 -2.0%) vs baseline: +4.9% ✅ 100-gauge-metrics-100-timesTime: ✅ 1.406ms (SLO: <1.550ms -9.3%) vs baseline: +0.3% Memory: ✅ 34.918MB (SLO: <35.500MB 🟡 -1.6%) vs baseline: +5.3% ✅ 100-rate-metrics-100-timesTime: ✅ 2.207ms (SLO: <2.550ms 📉 -13.5%) vs baseline: ~same Memory: ✅ 34.819MB (SLO: <35.500MB 🟡 -1.9%) vs baseline: +4.9% ✅ flush-1-metricTime: ✅ 4.539µs (SLO: <20.000µs 📉 -77.3%) vs baseline: -0.5% Memory: ✅ 34.819MB (SLO: <35.500MB 🟡 -1.9%) vs baseline: +4.9% ✅ flush-100-metricsTime: ✅ 173.374µs (SLO: <250.000µs 📉 -30.7%) vs baseline: -0.4% Memory: ✅ 35.114MB (SLO: <35.500MB 🟡 -1.1%) vs baseline: +4.7% ✅ flush-1000-metricsTime: ✅ 2.187ms (SLO: <2.500ms 📉 -12.5%) vs baseline: +0.2% Memory: ✅ 36.058MB (SLO: <36.500MB 🟡 -1.2%) vs baseline: +5.0% 🟡 Near SLO Breach (15 suites)🟡 coreapiscenario - 10/10 (1 unstable)
|
288c130 to
17915b1
Compare
17915b1 to
7c5c7bf
Compare
Description
I think I may have been a little bit too quick with this one (#15864)... this should unflake again.
See test runs