-
Notifications
You must be signed in to change notification settings - Fork 477
ci: remove *-profile jobs #15970
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ci: remove *-profile jobs #15970
Conversation
Codeowners resolved as |
Performance SLOsComparing candidate emmett.butler/profile-jobs (096382d) with baseline main (76fae6c) 📈 Performance Regressions (3 suites)📈 iastaspects - 118/118✅ add_aspectTime: ✅ 105.338µs (SLO: <130.000µs 📉 -19.0%) vs baseline: +4.6% Memory: ✅ 42.625MB (SLO: <43.250MB 🟡 -1.4%) vs baseline: +4.7% ✅ add_inplace_aspectTime: ✅ 102.475µs (SLO: <130.000µs 📉 -21.2%) vs baseline: +1.6% Memory: ✅ 42.644MB (SLO: <43.250MB 🟡 -1.4%) vs baseline: +5.0% ✅ add_inplace_noaspectTime: ✅ 28.115µs (SLO: <40.000µs 📉 -29.7%) vs baseline: -0.9% Memory: ✅ 42.664MB (SLO: <43.500MB 🟡 -1.9%) vs baseline: +4.8% ✅ add_noaspectTime: ✅ 48.747µs (SLO: <70.000µs 📉 -30.4%) vs baseline: ~same Memory: ✅ 42.605MB (SLO: <43.500MB -2.1%) vs baseline: +4.8% ✅ bytearray_aspectTime: ✅ 257.396µs (SLO: <400.000µs 📉 -35.7%) vs baseline: +1.1% Memory: ✅ 42.644MB (SLO: <43.500MB 🟡 -2.0%) vs baseline: +4.9% ✅ bytearray_extend_aspectTime: ✅ 650.975µs (SLO: <800.000µs 📉 -18.6%) vs baseline: +0.4% Memory: ✅ 42.723MB (SLO: <43.500MB 🟡 -1.8%) vs baseline: +4.9% ✅ bytearray_extend_noaspectTime: ✅ 266.927µs (SLO: <400.000µs 📉 -33.3%) vs baseline: -0.3% Memory: ✅ 42.585MB (SLO: <43.500MB -2.1%) vs baseline: +4.8% ✅ bytearray_noaspectTime: ✅ 139.555µs (SLO: <300.000µs 📉 -53.5%) vs baseline: -0.1% Memory: ✅ 42.644MB (SLO: <43.500MB 🟡 -2.0%) vs baseline: +4.9% ✅ bytes_aspectTime: ✅ 224.547µs (SLO: <300.000µs 📉 -25.2%) vs baseline: +1.5% Memory: ✅ 42.762MB (SLO: <43.500MB 🟡 -1.7%) vs baseline: +5.0% ✅ bytes_noaspectTime: ✅ 134.868µs (SLO: <200.000µs 📉 -32.6%) vs baseline: +1.1% Memory: ✅ 42.782MB (SLO: <43.500MB 🟡 -1.7%) vs baseline: +5.3% ✅ bytesio_aspectTime: ✅ 3.836ms (SLO: <5.000ms 📉 -23.3%) vs baseline: -0.8% Memory: ✅ 42.605MB (SLO: <43.500MB -2.1%) vs baseline: +4.9% ✅ bytesio_noaspectTime: ✅ 319.562µs (SLO: <420.000µs 📉 -23.9%) vs baseline: -0.7% Memory: ✅ 42.546MB (SLO: <43.500MB -2.2%) vs baseline: +4.7% ✅ capitalize_aspectTime: ✅ 90.089µs (SLO: <300.000µs 📉 -70.0%) vs baseline: -0.3% Memory: ✅ 42.664MB (SLO: <43.500MB 🟡 -1.9%) vs baseline: +4.8% ✅ capitalize_noaspectTime: ✅ 253.214µs (SLO: <300.000µs 📉 -15.6%) vs baseline: -0.8% Memory: ✅ 42.605MB (SLO: <43.500MB -2.1%) vs baseline: +4.8% ✅ casefold_aspectTime: ✅ 92.913µs (SLO: <500.000µs 📉 -81.4%) vs baseline: +3.0% Memory: ✅ 42.625MB (SLO: <43.500MB -2.0%) vs baseline: +4.9% ✅ casefold_noaspectTime: ✅ 309.275µs (SLO: <500.000µs 📉 -38.1%) vs baseline: -0.5% Memory: ✅ 42.684MB (SLO: <43.500MB 🟡 -1.9%) vs baseline: +4.9% ✅ decode_aspectTime: ✅ 87.282µs (SLO: <100.000µs 📉 -12.7%) vs baseline: +0.2% Memory: ✅ 42.644MB (SLO: <43.500MB 🟡 -2.0%) vs baseline: +5.0% ✅ decode_noaspectTime: ✅ 152.371µs (SLO: <210.000µs 📉 -27.4%) vs baseline: -1.1% Memory: ✅ 42.605MB (SLO: <43.500MB -2.1%) vs baseline: +4.7% ✅ encode_aspectTime: ✅ 84.761µs (SLO: <200.000µs 📉 -57.6%) vs baseline: -0.3% Memory: ✅ 42.546MB (SLO: <43.500MB -2.2%) vs baseline: +4.7% ✅ encode_noaspectTime: ✅ 137.233µs (SLO: <200.000µs 📉 -31.4%) vs baseline: -0.4% Memory: ✅ 42.644MB (SLO: <43.500MB 🟡 -2.0%) vs baseline: +5.0% ✅ format_aspectTime: ✅ 14.705ms (SLO: <19.200ms 📉 -23.4%) vs baseline: -0.2% Memory: ✅ 42.821MB (SLO: <43.250MB 🟡 -1.0%) vs baseline: +4.9% ✅ format_map_aspectTime: ✅ 16.479ms (SLO: <21.500ms 📉 -23.4%) vs baseline: ~same Memory: ✅ 42.821MB (SLO: <43.500MB 🟡 -1.6%) vs baseline: +4.8% ✅ format_map_noaspectTime: ✅ 370.893µs (SLO: <500.000µs 📉 -25.8%) vs baseline: +0.1% Memory: ✅ 42.585MB (SLO: <43.250MB 🟡 -1.5%) vs baseline: +4.3% ✅ format_noaspectTime: ✅ 305.875µs (SLO: <500.000µs 📉 -38.8%) vs baseline: ~same Memory: ✅ 42.605MB (SLO: <43.250MB 🟡 -1.5%) vs baseline: +4.9% ✅ index_aspectTime: ✅ 123.865µs (SLO: <300.000µs 📉 -58.7%) vs baseline: +0.4% Memory: ✅ 42.605MB (SLO: <43.250MB 🟡 -1.5%) vs baseline: +4.7% ✅ index_noaspectTime: ✅ 40.645µs (SLO: <300.000µs 📉 -86.5%) vs baseline: ~same Memory: ✅ 42.605MB (SLO: <43.500MB -2.1%) vs baseline: +4.8% ✅ join_aspectTime: ✅ 219.274µs (SLO: <300.000µs 📉 -26.9%) vs baseline: +1.1% Memory: ✅ 42.644MB (SLO: <43.500MB 🟡 -2.0%) vs baseline: +4.8% ✅ join_noaspectTime: ✅ 148.983µs (SLO: <300.000µs 📉 -50.3%) vs baseline: +0.5% Memory: ✅ 42.625MB (SLO: <43.250MB 🟡 -1.4%) vs baseline: +4.8% ✅ ljust_aspectTime: ✅ 499.278µs (SLO: <700.000µs 📉 -28.7%) vs baseline: ~same Memory: ✅ 42.585MB (SLO: <43.250MB 🟡 -1.5%) vs baseline: +4.7% ✅ ljust_noaspectTime: ✅ 256.069µs (SLO: <300.000µs 📉 -14.6%) vs baseline: ~same Memory: ✅ 42.625MB (SLO: <43.250MB 🟡 -1.4%) vs baseline: +4.9% ✅ lower_aspectTime: ✅ 306.515µs (SLO: <500.000µs 📉 -38.7%) vs baseline: -0.3% Memory: ✅ 42.605MB (SLO: <43.500MB -2.1%) vs baseline: +4.9% ✅ lower_noaspectTime: ✅ 232.522µs (SLO: <300.000µs 📉 -22.5%) vs baseline: -1.7% Memory: ✅ 42.644MB (SLO: <43.250MB 🟡 -1.4%) vs baseline: +4.7% ✅ lstrip_aspectTime: ✅ 0.271ms (SLO: <3.000ms 📉 -91.0%) vs baseline: -0.4% Memory: ✅ 42.664MB (SLO: <43.250MB 🟡 -1.4%) vs baseline: +5.0% ✅ lstrip_noaspectTime: ✅ 0.199ms (SLO: <3.000ms 📉 -93.4%) vs baseline: 📈 +12.4% Memory: ✅ 42.644MB (SLO: <43.500MB 🟡 -2.0%) vs baseline: +4.9% ✅ modulo_aspectTime: ✅ 14.379ms (SLO: <18.750ms 📉 -23.3%) vs baseline: ~same Memory: ✅ 42.703MB (SLO: <43.500MB 🟡 -1.8%) vs baseline: +4.5% ✅ modulo_aspect_for_bytearray_bytearrayTime: ✅ 14.863ms (SLO: <19.350ms 📉 -23.2%) vs baseline: +0.1% Memory: ✅ 42.821MB (SLO: <43.500MB 🟡 -1.6%) vs baseline: +4.7% ✅ modulo_aspect_for_bytesTime: ✅ 14.506ms (SLO: <18.900ms 📉 -23.2%) vs baseline: +0.4% Memory: ✅ 42.782MB (SLO: <43.500MB 🟡 -1.7%) vs baseline: +4.6% ✅ modulo_aspect_for_bytes_bytearrayTime: ✅ 14.668ms (SLO: <19.150ms 📉 -23.4%) vs baseline: -0.4% Memory: ✅ 42.861MB (SLO: <43.500MB 🟡 -1.5%) vs baseline: +4.9% ✅ modulo_noaspectTime: ✅ 0.357ms (SLO: <3.000ms 📉 -88.1%) vs baseline: -0.5% Memory: ✅ 42.723MB (SLO: <43.500MB 🟡 -1.8%) vs baseline: +5.2% ✅ replace_aspectTime: ✅ 18.460ms (SLO: <24.000ms 📉 -23.1%) vs baseline: +0.1% Memory: ✅ 42.821MB (SLO: <44.000MB -2.7%) vs baseline: +4.9% ✅ replace_noaspectTime: ✅ 283.993µs (SLO: <300.000µs -5.3%) vs baseline: -0.4% Memory: ✅ 42.585MB (SLO: <43.500MB -2.1%) vs baseline: +4.7% ✅ repr_aspectTime: ✅ 316.280µs (SLO: <420.000µs 📉 -24.7%) vs baseline: -0.5% Memory: ✅ 42.644MB (SLO: <43.500MB 🟡 -2.0%) vs baseline: +4.8% ✅ repr_noaspectTime: ✅ 46.701µs (SLO: <90.000µs 📉 -48.1%) vs baseline: +0.3% Memory: ✅ 42.605MB (SLO: <43.500MB -2.1%) vs baseline: +4.9% ✅ rstrip_aspectTime: ✅ 379.808µs (SLO: <500.000µs 📉 -24.0%) vs baseline: +0.1% Memory: ✅ 42.605MB (SLO: <43.500MB -2.1%) vs baseline: +4.7% ✅ rstrip_noaspectTime: ✅ 181.318µs (SLO: <300.000µs 📉 -39.6%) vs baseline: -0.4% Memory: ✅ 42.644MB (SLO: <43.500MB 🟡 -2.0%) vs baseline: +5.0% ✅ slice_aspectTime: ✅ 189.590µs (SLO: <300.000µs 📉 -36.8%) vs baseline: +3.7% Memory: ✅ 42.605MB (SLO: <43.500MB -2.1%) vs baseline: +4.9% ✅ slice_noaspectTime: ✅ 53.958µs (SLO: <90.000µs 📉 -40.0%) vs baseline: -0.3% Memory: ✅ 42.546MB (SLO: <43.500MB -2.2%) vs baseline: +4.6% ✅ stringio_aspectTime: ✅ 3.897ms (SLO: <5.000ms 📉 -22.1%) vs baseline: -0.1% Memory: ✅ 42.605MB (SLO: <43.500MB -2.1%) vs baseline: +4.6% ✅ stringio_noaspectTime: ✅ 354.517µs (SLO: <500.000µs 📉 -29.1%) vs baseline: -0.4% Memory: ✅ 42.566MB (SLO: <43.500MB -2.1%) vs baseline: +4.5% ✅ strip_aspectTime: ✅ 269.611µs (SLO: <350.000µs 📉 -23.0%) vs baseline: -0.3% Memory: ✅ 42.605MB (SLO: <43.500MB -2.1%) vs baseline: +4.7% ✅ strip_noaspectTime: ✅ 178.639µs (SLO: <240.000µs 📉 -25.6%) vs baseline: +0.6% Memory: ✅ 42.703MB (SLO: <43.500MB 🟡 -1.8%) vs baseline: +5.0% ✅ swapcase_aspectTime: ✅ 345.832µs (SLO: <500.000µs 📉 -30.8%) vs baseline: +1.3% Memory: ✅ 42.625MB (SLO: <43.500MB -2.0%) vs baseline: +4.8% ✅ swapcase_noaspectTime: ✅ 271.940µs (SLO: <400.000µs 📉 -32.0%) vs baseline: +0.4% Memory: ✅ 42.605MB (SLO: <43.500MB -2.1%) vs baseline: +4.9% ✅ title_aspectTime: ✅ 333.084µs (SLO: <500.000µs 📉 -33.4%) vs baseline: ~same Memory: ✅ 42.625MB (SLO: <43.000MB 🟡 -0.9%) vs baseline: +4.8% ✅ title_noaspectTime: ✅ 260.599µs (SLO: <400.000µs 📉 -34.9%) vs baseline: +0.7% Memory: ✅ 42.625MB (SLO: <43.500MB -2.0%) vs baseline: +4.7% ✅ translate_aspectTime: ✅ 501.946µs (SLO: <700.000µs 📉 -28.3%) vs baseline: +1.2% Memory: ✅ 42.644MB (SLO: <43.500MB 🟡 -2.0%) vs baseline: +4.8% ✅ translate_noaspectTime: ✅ 423.460µs (SLO: <500.000µs 📉 -15.3%) vs baseline: -2.3% Memory: ✅ 42.703MB (SLO: <43.500MB 🟡 -1.8%) vs baseline: +5.1% ✅ upper_aspectTime: ✅ 307.520µs (SLO: <500.000µs 📉 -38.5%) vs baseline: +0.5% Memory: ✅ 42.664MB (SLO: <43.500MB 🟡 -1.9%) vs baseline: +4.8% ✅ upper_noaspectTime: ✅ 234.745µs (SLO: <400.000µs 📉 -41.3%) vs baseline: +0.1% Memory: ✅ 42.664MB (SLO: <43.500MB 🟡 -1.9%) vs baseline: +4.9% 📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 507.308µs (SLO: <700.000µs 📉 -27.5%) vs baseline: 📈 +20.2% Memory: ✅ 42.290MB (SLO: <43.500MB -2.8%) vs baseline: +4.8% ✅ ospathbasename_noaspectTime: ✅ 425.587µs (SLO: <700.000µs 📉 -39.2%) vs baseline: -0.2% Memory: ✅ 42.467MB (SLO: <43.500MB -2.4%) vs baseline: +5.1% ✅ ospathjoin_aspectTime: ✅ 617.360µs (SLO: <700.000µs 📉 -11.8%) vs baseline: -1.3% Memory: ✅ 42.310MB (SLO: <43.500MB -2.7%) vs baseline: +5.1% ✅ ospathjoin_noaspectTime: ✅ 631.811µs (SLO: <700.000µs -9.7%) vs baseline: ~same Memory: ✅ 42.467MB (SLO: <43.500MB -2.4%) vs baseline: +5.1% ✅ ospathnormcase_aspectTime: ✅ 352.790µs (SLO: <700.000µs 📉 -49.6%) vs baseline: +0.2% Memory: ✅ 42.389MB (SLO: <43.500MB -2.6%) vs baseline: +4.8% ✅ ospathnormcase_noaspectTime: ✅ 355.636µs (SLO: <700.000µs 📉 -49.2%) vs baseline: -0.5% Memory: ✅ 42.330MB (SLO: <43.500MB -2.7%) vs baseline: +4.6% ✅ ospathsplit_aspectTime: ✅ 486.392µs (SLO: <700.000µs 📉 -30.5%) vs baseline: -0.6% Memory: ✅ 42.467MB (SLO: <43.500MB -2.4%) vs baseline: +5.0% ✅ ospathsplit_noaspectTime: ✅ 497.531µs (SLO: <700.000µs 📉 -28.9%) vs baseline: +0.1% Memory: ✅ 42.487MB (SLO: <43.500MB -2.3%) vs baseline: +5.4% ✅ ospathsplitdrive_aspectTime: ✅ 371.562µs (SLO: <700.000µs 📉 -46.9%) vs baseline: ~same Memory: ✅ 42.408MB (SLO: <43.500MB -2.5%) vs baseline: +4.9% ✅ ospathsplitdrive_noaspectTime: ✅ 73.597µs (SLO: <700.000µs 📉 -89.5%) vs baseline: +1.3% Memory: ✅ 42.389MB (SLO: <43.500MB -2.6%) vs baseline: +4.8% ✅ ospathsplitext_aspectTime: ✅ 455.940µs (SLO: <700.000µs 📉 -34.9%) vs baseline: -0.3% Memory: ✅ 42.330MB (SLO: <43.500MB -2.7%) vs baseline: +4.8% ✅ ospathsplitext_noaspectTime: ✅ 460.252µs (SLO: <700.000µs 📉 -34.2%) vs baseline: +0.2% Memory: ✅ 42.290MB (SLO: <43.500MB -2.8%) vs baseline: +4.8% 📈 telemetryaddmetric - 30/30✅ 1-count-metric-1-timesTime: ✅ 3.406µs (SLO: <20.000µs 📉 -83.0%) vs baseline: 📈 +12.8% Memory: ✅ 34.918MB (SLO: <35.500MB 🟡 -1.6%) vs baseline: +4.8% ✅ 1-count-metrics-100-timesTime: ✅ 199.886µs (SLO: <220.000µs -9.1%) vs baseline: -0.2% Memory: ✅ 34.937MB (SLO: <35.500MB 🟡 -1.6%) vs baseline: +4.8% ✅ 1-distribution-metric-1-timesTime: ✅ 3.361µs (SLO: <20.000µs 📉 -83.2%) vs baseline: -0.6% Memory: ✅ 34.918MB (SLO: <35.500MB 🟡 -1.6%) vs baseline: +5.0% ✅ 1-distribution-metrics-100-timesTime: ✅ 214.609µs (SLO: <230.000µs -6.7%) vs baseline: +0.2% Memory: ✅ 34.918MB (SLO: <35.500MB 🟡 -1.6%) vs baseline: +4.7% ✅ 1-gauge-metric-1-timesTime: ✅ 2.196µs (SLO: <20.000µs 📉 -89.0%) vs baseline: -0.8% Memory: ✅ 34.859MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +4.7% ✅ 1-gauge-metrics-100-timesTime: ✅ 137.166µs (SLO: <150.000µs -8.6%) vs baseline: +0.3% Memory: ✅ 34.977MB (SLO: <35.500MB 🟡 -1.5%) vs baseline: +5.2% ✅ 1-rate-metric-1-timesTime: ✅ 3.173µs (SLO: <20.000µs 📉 -84.1%) vs baseline: ~same Memory: ✅ 34.859MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +4.9% ✅ 1-rate-metrics-100-timesTime: ✅ 214.854µs (SLO: <250.000µs 📉 -14.1%) vs baseline: +0.2% Memory: ✅ 35.036MB (SLO: <35.500MB 🟡 -1.3%) vs baseline: +5.7% ✅ 100-count-metrics-100-timesTime: ✅ 19.958ms (SLO: <22.000ms -9.3%) vs baseline: +0.4% Memory: ✅ 34.800MB (SLO: <35.500MB 🟡 -2.0%) vs baseline: +4.6% ✅ 100-distribution-metrics-100-timesTime: ✅ 2.222ms (SLO: <2.550ms 📉 -12.9%) vs baseline: -0.6% Memory: ✅ 34.898MB (SLO: <35.500MB 🟡 -1.7%) vs baseline: +4.5% ✅ 100-gauge-metrics-100-timesTime: ✅ 1.411ms (SLO: <1.550ms -9.0%) vs baseline: +1.0% Memory: ✅ 34.898MB (SLO: <35.500MB 🟡 -1.7%) vs baseline: +4.9% ✅ 100-rate-metrics-100-timesTime: ✅ 2.189ms (SLO: <2.550ms 📉 -14.1%) vs baseline: +0.7% Memory: ✅ 34.878MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +4.7% ✅ flush-1-metricTime: ✅ 4.590µs (SLO: <20.000µs 📉 -77.0%) vs baseline: -0.1% Memory: ✅ 35.232MB (SLO: <35.500MB 🟡 -0.8%) vs baseline: +5.0% ✅ flush-100-metricsTime: ✅ 173.495µs (SLO: <250.000µs 📉 -30.6%) vs baseline: -0.7% Memory: ✅ 35.271MB (SLO: <35.500MB 🟡 -0.6%) vs baseline: +5.1% ✅ flush-1000-metricsTime: ✅ 2.185ms (SLO: <2.500ms 📉 -12.6%) vs baseline: +0.2% Memory: ✅ 36.137MB (SLO: <36.500MB 🟡 -1.0%) vs baseline: +4.9% 🟡 Near SLO Breach (14 suites)🟡 coreapiscenario - 10/10 (1 unstable)
|
|
@P403n1x87 do you have any idea what's going on here with the "insufficient permissions" error? https://gitlab.ddbuild.io/DataDog/apm-reliability/dd-trace-py/-/jobs/1340383332 |
This change removes the
*-profilejobs from the github actions config because they're not actionable. In the future it would make sense to add benchmarks that exercise the same code and provide pass/fail statuses per commit based on configurable thresholds.