-
Notifications
You must be signed in to change notification settings - Fork 480
perf(debugger): reduce code origin startup time #15272
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
|
d2f1ae1 to
533e5b8
Compare
Bootstrap import analysisComparison of import times between this PR and base. SummaryThe average import time from this PR is: 246 ± 2 ms. The average import time from base is: 250 ± 4 ms. The import time difference between this PR and base is: -4.3 ± 0.1 ms. Import time breakdownThe following import paths have appeared:
|
Performance SLOsComparing candidate tyler.finethy/benchmark-co-instrumentation (102a46c) with baseline main (3c8055f) 📈 Performance Regressions (2 suites)📈 iastaspects - 118/118✅ add_aspectTime: ✅ 18.219µs (SLO: <20.000µs -8.9%) vs baseline: 📈 +22.2% Memory: ✅ 42.664MB (SLO: <43.250MB 🟡 -1.4%) vs baseline: +4.7% ✅ add_inplace_aspectTime: ✅ 14.917µs (SLO: <20.000µs 📉 -25.4%) vs baseline: -0.5% Memory: ✅ 42.743MB (SLO: <43.250MB 🟡 -1.2%) vs baseline: +5.1% ✅ add_inplace_noaspectTime: ✅ 0.340µs (SLO: <10.000µs 📉 -96.6%) vs baseline: +0.3% Memory: ✅ 42.625MB (SLO: <43.000MB 🟡 -0.9%) vs baseline: +4.9% ✅ add_noaspectTime: ✅ 0.544µs (SLO: <10.000µs 📉 -94.6%) vs baseline: +0.3% Memory: ✅ 42.625MB (SLO: <43.000MB 🟡 -0.9%) vs baseline: +4.9% ✅ bytearray_aspectTime: ✅ 17.882µs (SLO: <30.000µs 📉 -40.4%) vs baseline: -1.5% Memory: ✅ 42.782MB (SLO: <43.500MB 🟡 -1.7%) vs baseline: +4.7% ✅ bytearray_extend_aspectTime: ✅ 23.828µs (SLO: <30.000µs 📉 -20.6%) vs baseline: +0.7% Memory: ✅ 42.605MB (SLO: <43.500MB -2.1%) vs baseline: +4.0% ✅ bytearray_extend_noaspectTime: ✅ 2.636µs (SLO: <10.000µs 📉 -73.6%) vs baseline: -0.4% Memory: ✅ 42.743MB (SLO: <43.500MB 🟡 -1.7%) vs baseline: +5.0% ✅ bytearray_noaspectTime: ✅ 1.442µs (SLO: <10.000µs 📉 -85.6%) vs baseline: -0.5% Memory: ✅ 42.644MB (SLO: <43.500MB 🟡 -2.0%) vs baseline: +4.5% ✅ bytes_aspectTime: ✅ 16.774µs (SLO: <20.000µs 📉 -16.1%) vs baseline: +0.6% Memory: ✅ 42.762MB (SLO: <43.000MB 🟡 -0.6%) vs baseline: +4.6% ✅ bytes_noaspectTime: ✅ 1.401µs (SLO: <10.000µs 📉 -86.0%) vs baseline: +0.6% Memory: ✅ 42.644MB (SLO: <43.000MB 🟡 -0.8%) vs baseline: +4.5% ✅ bytesio_aspectTime: ✅ 55.021µs (SLO: <70.000µs 📉 -21.4%) vs baseline: ~same Memory: ✅ 42.585MB (SLO: <43.500MB -2.1%) vs baseline: +4.3% ✅ bytesio_noaspectTime: ✅ 3.211µs (SLO: <10.000µs 📉 -67.9%) vs baseline: -0.2% Memory: ✅ 42.625MB (SLO: <43.500MB -2.0%) vs baseline: +4.4% ✅ capitalize_aspectTime: ✅ 14.634µs (SLO: <20.000µs 📉 -26.8%) vs baseline: -0.3% Memory: ✅ 42.625MB (SLO: <43.000MB 🟡 -0.9%) vs baseline: +4.3% ✅ capitalize_noaspectTime: ✅ 2.557µs (SLO: <10.000µs 📉 -74.4%) vs baseline: -0.3% Memory: ✅ 42.487MB (SLO: <43.000MB 🟡 -1.2%) vs baseline: +4.2% ✅ casefold_aspectTime: ✅ 14.668µs (SLO: <20.000µs 📉 -26.7%) vs baseline: +0.2% Memory: ✅ 42.625MB (SLO: <43.000MB 🟡 -0.9%) vs baseline: +4.7% ✅ casefold_noaspectTime: ✅ 3.094µs (SLO: <10.000µs 📉 -69.1%) vs baseline: +0.5% Memory: ✅ 42.703MB (SLO: <43.000MB 🟡 -0.7%) vs baseline: +4.9% ✅ decode_aspectTime: ✅ 15.516µs (SLO: <30.000µs 📉 -48.3%) vs baseline: -0.3% Memory: ✅ 42.782MB (SLO: <43.500MB 🟡 -1.7%) vs baseline: +4.6% ✅ decode_noaspectTime: ✅ 1.585µs (SLO: <10.000µs 📉 -84.2%) vs baseline: ~same Memory: ✅ 42.743MB (SLO: <43.500MB 🟡 -1.7%) vs baseline: +4.9% ✅ encode_aspectTime: ✅ 18.151µs (SLO: <30.000µs 📉 -39.5%) vs baseline: 📈 +21.8% Memory: ✅ 42.585MB (SLO: <43.500MB -2.1%) vs baseline: +4.2% ✅ encode_noaspectTime: ✅ 1.468µs (SLO: <10.000µs 📉 -85.3%) vs baseline: +0.6% Memory: ✅ 42.723MB (SLO: <43.000MB 🟡 -0.6%) vs baseline: +5.0% ✅ format_aspectTime: ✅ 170.968µs (SLO: <200.000µs 📉 -14.5%) vs baseline: ~same Memory: ✅ 42.723MB (SLO: <43.250MB 🟡 -1.2%) vs baseline: +4.1% ✅ format_map_aspectTime: ✅ 191.529µs (SLO: <200.000µs -4.2%) vs baseline: -0.1% Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.7% ✅ format_map_noaspectTime: ✅ 3.757µs (SLO: <10.000µs 📉 -62.4%) vs baseline: +1.0% Memory: ✅ 42.664MB (SLO: <43.250MB 🟡 -1.4%) vs baseline: +4.8% ✅ format_noaspectTime: ✅ 3.159µs (SLO: <10.000µs 📉 -68.4%) vs baseline: +0.8% Memory: ✅ 42.703MB (SLO: <43.250MB 🟡 -1.3%) vs baseline: +4.5% ✅ index_aspectTime: ✅ 15.180µs (SLO: <20.000µs 📉 -24.1%) vs baseline: -0.4% Memory: ✅ 42.762MB (SLO: <43.250MB 🟡 -1.1%) vs baseline: +4.7% ✅ index_noaspectTime: ✅ 0.467µs (SLO: <10.000µs 📉 -95.3%) vs baseline: +0.7% Memory: ✅ 42.625MB (SLO: <43.000MB 🟡 -0.9%) vs baseline: +4.7% ✅ join_aspectTime: ✅ 16.852µs (SLO: <20.000µs 📉 -15.7%) vs baseline: -0.6% Memory: ✅ 42.625MB (SLO: <43.500MB -2.0%) vs baseline: +4.6% ✅ join_noaspectTime: ✅ 1.477µs (SLO: <10.000µs 📉 -85.2%) vs baseline: -0.2% Memory: ✅ 42.703MB (SLO: <43.250MB 🟡 -1.3%) vs baseline: +5.1% ✅ ljust_aspectTime: ✅ 20.762µs (SLO: <30.000µs 📉 -30.8%) vs baseline: ~same Memory: ✅ 42.566MB (SLO: <43.250MB 🟡 -1.6%) vs baseline: +4.1% ✅ ljust_noaspectTime: ✅ 2.659µs (SLO: <10.000µs 📉 -73.4%) vs baseline: -0.1% Memory: ✅ 42.723MB (SLO: <43.250MB 🟡 -1.2%) vs baseline: +4.7% ✅ lower_aspectTime: ✅ 17.804µs (SLO: <30.000µs 📉 -40.7%) vs baseline: -0.4% Memory: ✅ 42.684MB (SLO: <43.000MB 🟡 -0.7%) vs baseline: +4.6% ✅ lower_noaspectTime: ✅ 2.380µs (SLO: <10.000µs 📉 -76.2%) vs baseline: +0.3% Memory: ✅ 42.507MB (SLO: <43.250MB 🟡 -1.7%) vs baseline: +4.2% ✅ lstrip_aspectTime: ✅ 17.577µs (SLO: <20.000µs 📉 -12.1%) vs baseline: -0.4% Memory: ✅ 42.880MB (SLO: <43.250MB 🟡 -0.9%) vs baseline: +4.7% ✅ lstrip_noaspectTime: ✅ 1.829µs (SLO: <10.000µs 📉 -81.7%) vs baseline: -0.5% Memory: ✅ 42.861MB (SLO: <43.000MB 🟡 -0.3%) vs baseline: +5.2% ✅ modulo_aspectTime: ✅ 166.334µs (SLO: <200.000µs 📉 -16.8%) vs baseline: -0.3% Memory: ✅ 42.762MB (SLO: <43.500MB 🟡 -1.7%) vs baseline: +3.9% ✅ modulo_aspect_for_bytearray_bytearrayTime: ✅ 179.753µs (SLO: <200.000µs 📉 -10.1%) vs baseline: +2.8% Memory: ✅ 42.782MB (SLO: <43.500MB 🟡 -1.7%) vs baseline: +4.7% ✅ modulo_aspect_for_bytesTime: ✅ 168.773µs (SLO: <200.000µs 📉 -15.6%) vs baseline: +0.5% Memory: ✅ 42.861MB (SLO: <43.500MB 🟡 -1.5%) vs baseline: +5.2% ✅ modulo_aspect_for_bytes_bytearrayTime: ✅ 171.651µs (SLO: <200.000µs 📉 -14.2%) vs baseline: +0.2% Memory: ✅ 42.900MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +4.8% ✅ modulo_noaspectTime: ✅ 3.710µs (SLO: <10.000µs 📉 -62.9%) vs baseline: +0.8% Memory: ✅ 42.802MB (SLO: <43.000MB 🟡 -0.5%) vs baseline: +5.0% ✅ replace_aspectTime: ✅ 214.392µs (SLO: <300.000µs 📉 -28.5%) vs baseline: +0.1% Memory: ✅ 42.920MB (SLO: <44.000MB -2.5%) vs baseline: +5.3% ✅ replace_noaspectTime: ✅ 5.171µs (SLO: <10.000µs 📉 -48.3%) vs baseline: +0.5% Memory: ✅ 42.605MB (SLO: <43.000MB 🟡 -0.9%) vs baseline: +4.5% ✅ repr_aspectTime: ✅ 1.380µs (SLO: <10.000µs 📉 -86.2%) vs baseline: +0.3% Memory: ✅ 42.644MB (SLO: <43.500MB 🟡 -2.0%) vs baseline: +4.6% ✅ repr_noaspectTime: ✅ 0.525µs (SLO: <10.000µs 📉 -94.8%) vs baseline: +0.1% Memory: ✅ 42.644MB (SLO: <43.000MB 🟡 -0.8%) vs baseline: +5.1% ✅ rstrip_aspectTime: ✅ 19.112µs (SLO: <30.000µs 📉 -36.3%) vs baseline: +0.6% Memory: ✅ 42.664MB (SLO: <43.000MB 🟡 -0.8%) vs baseline: +4.4% ✅ rstrip_noaspectTime: ✅ 1.900µs (SLO: <10.000µs 📉 -81.0%) vs baseline: +1.5% Memory: ✅ 42.664MB (SLO: <43.000MB 🟡 -0.8%) vs baseline: +4.9% ✅ slice_aspectTime: ✅ 15.927µs (SLO: <20.000µs 📉 -20.4%) vs baseline: -0.2% Memory: ✅ 42.644MB (SLO: <43.000MB 🟡 -0.8%) vs baseline: +4.8% ✅ slice_noaspectTime: ✅ 0.596µs (SLO: <10.000µs 📉 -94.0%) vs baseline: -0.5% Memory: ✅ 42.605MB (SLO: <43.000MB 🟡 -0.9%) vs baseline: +4.7% ✅ stringio_aspectTime: ✅ 53.787µs (SLO: <80.000µs 📉 -32.8%) vs baseline: +0.3% Memory: ✅ 42.723MB (SLO: <43.500MB 🟡 -1.8%) vs baseline: +4.7% ✅ stringio_noaspectTime: ✅ 3.578µs (SLO: <10.000µs 📉 -64.2%) vs baseline: +2.9% Memory: ✅ 42.703MB (SLO: <43.500MB 🟡 -1.8%) vs baseline: +5.1% ✅ strip_aspectTime: ✅ 17.592µs (SLO: <20.000µs 📉 -12.0%) vs baseline: -0.2% Memory: ✅ 42.625MB (SLO: <43.000MB 🟡 -0.9%) vs baseline: +4.6% ✅ strip_noaspectTime: ✅ 1.833µs (SLO: <10.000µs 📉 -81.7%) vs baseline: -0.5% Memory: ✅ 42.762MB (SLO: <43.000MB 🟡 -0.6%) vs baseline: +5.2% ✅ swapcase_aspectTime: ✅ 18.430µs (SLO: <30.000µs 📉 -38.6%) vs baseline: +0.8% Memory: ✅ 42.625MB (SLO: <43.000MB 🟡 -0.9%) vs baseline: +4.6% ✅ swapcase_noaspectTime: ✅ 2.769µs (SLO: <10.000µs 📉 -72.3%) vs baseline: +1.1% Memory: ✅ 42.762MB (SLO: <43.000MB 🟡 -0.6%) vs baseline: +4.9% ✅ title_aspectTime: ✅ 18.130µs (SLO: <20.000µs -9.3%) vs baseline: -0.4% Memory: ✅ 42.605MB (SLO: <43.000MB 🟡 -0.9%) vs baseline: +4.7% ✅ title_noaspectTime: ✅ 2.678µs (SLO: <10.000µs 📉 -73.2%) vs baseline: +1.2% Memory: ✅ 42.644MB (SLO: <43.000MB 🟡 -0.8%) vs baseline: +4.8% ✅ translate_aspectTime: ✅ 23.917µs (SLO: <30.000µs 📉 -20.3%) vs baseline: 📈 +17.4% Memory: ✅ 42.821MB (SLO: <43.000MB 🟡 -0.4%) vs baseline: +5.1% ✅ translate_noaspectTime: ✅ 4.267µs (SLO: <10.000µs 📉 -57.3%) vs baseline: +0.7% Memory: ✅ 42.625MB (SLO: <43.000MB 🟡 -0.9%) vs baseline: +4.9% ✅ upper_aspectTime: ✅ 17.881µs (SLO: <30.000µs 📉 -40.4%) vs baseline: ~same Memory: ✅ 42.644MB (SLO: <43.000MB 🟡 -0.8%) vs baseline: +4.3% ✅ upper_noaspectTime: ✅ 2.401µs (SLO: <10.000µs 📉 -76.0%) vs baseline: +1.5% Memory: ✅ 42.723MB (SLO: <43.000MB 🟡 -0.6%) vs baseline: +4.8% 📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 5.253µs (SLO: <10.000µs 📉 -47.5%) vs baseline: 📈 +22.5% Memory: ✅ 41.445MB (SLO: <43.000MB -3.6%) vs baseline: +5.1% ✅ ospathbasename_noaspectTime: ✅ 4.319µs (SLO: <10.000µs 📉 -56.8%) vs baseline: -0.5% Memory: ✅ 41.504MB (SLO: <43.000MB -3.5%) vs baseline: +5.1% ✅ ospathjoin_aspectTime: ✅ 6.155µs (SLO: <10.000µs 📉 -38.4%) vs baseline: +0.2% Memory: ✅ 41.406MB (SLO: <43.000MB -3.7%) vs baseline: +4.6% ✅ ospathjoin_noaspectTime: ✅ 6.195µs (SLO: <10.000µs 📉 -38.1%) vs baseline: -0.7% Memory: ✅ 41.445MB (SLO: <43.000MB -3.6%) vs baseline: +5.1% ✅ ospathnormcase_aspectTime: ✅ 3.568µs (SLO: <10.000µs 📉 -64.3%) vs baseline: +0.5% Memory: ✅ 41.386MB (SLO: <43.000MB -3.8%) vs baseline: +4.7% ✅ ospathnormcase_noaspectTime: ✅ 3.565µs (SLO: <10.000µs 📉 -64.4%) vs baseline: -1.2% Memory: ✅ 41.425MB (SLO: <43.000MB -3.7%) vs baseline: +4.9% ✅ ospathsplit_aspectTime: ✅ 4.847µs (SLO: <10.000µs 📉 -51.5%) vs baseline: -0.5% Memory: ✅ 41.504MB (SLO: <43.000MB -3.5%) vs baseline: +5.1% ✅ ospathsplit_noaspectTime: ✅ 5.010µs (SLO: <10.000µs 📉 -49.9%) vs baseline: +0.5% Memory: ✅ 41.465MB (SLO: <43.000MB -3.6%) vs baseline: +5.1% ✅ ospathsplitdrive_aspectTime: ✅ 3.786µs (SLO: <10.000µs 📉 -62.1%) vs baseline: +0.1% Memory: ✅ 41.425MB (SLO: <43.000MB -3.7%) vs baseline: +5.1% ✅ ospathsplitdrive_noaspectTime: ✅ 0.745µs (SLO: <10.000µs 📉 -92.6%) vs baseline: +0.6% Memory: ✅ 41.406MB (SLO: <43.000MB -3.7%) vs baseline: +4.7% ✅ ospathsplitext_aspectTime: ✅ 4.646µs (SLO: <10.000µs 📉 -53.5%) vs baseline: +0.9% Memory: ✅ 41.465MB (SLO: <43.000MB -3.6%) vs baseline: +5.0% ✅ ospathsplitext_noaspectTime: ✅ 4.725µs (SLO: <10.000µs 📉 -52.7%) vs baseline: +1.1% Memory: ✅ 41.425MB (SLO: <43.000MB -3.7%) vs baseline: +4.9% 🟡 Near SLO Breach (16 suites)🟡 coreapiscenario - 10/10 (1 unstable)
|
Start with a benchmark to measure opportunities for improvements refs: DEBUG-4605
3102d43 to
fc88001
Compare
f92a00e to
1467c02
Compare
|
@tylfin is this PR still needed? there is a merge conflict |
Yes, we still need to figure out what's making the system tests fail. I'll resume working on it next week. |
…rk-co-instrumentation
…rk-co-instrumentation
da7f944 to
b387e54
Compare
b387e54 to
2d4fc69
Compare
|
I can't approve since I opened the PR but LGTM |
…rk-co-instrumentation
184bbb0 to
102a46c
Compare
…" (DataDog#15776) This reverts commit DataDog#15272 We detected errors " Context already registered" that we believe related to PR DataDog#15272. So we undo this change and backport the fix.
We re-introduce the improvement of #15272 with the bugfix that allows the same view function to be registered multiple times
We re-introduce the improvement of DataDog#15272 with the bugfix that allows the same view function to be registered multiple times
Description
We adopt a lazy wrapping approach in Code Origin to reduce the startup cost when instrumenting view functions. With the lazy approach, the heavy instrumentation is performed when the view function is invoked for the first time, preventing a potentially large delay on boot.
refs: DEBUG-4605
Testing
Added benchmark to validate the performance improvements
Risks
N/A
Additional Notes