test(fuzzing): add internal fuzzing infra support - CHAOSPLT-1355#15685
test(fuzzing): add internal fuzzing infra support - CHAOSPLT-1355#15685taegyunkim merged 42 commits intomainfrom
Conversation
|
|
2b7ad83 to
97cd508
Compare
Codeowners resolved as |
Performance SLOsComparing candidate edouard/add-base-fuzzing-setup (5c7739e) with baseline main (b9c7d4a) 📈 Performance Regressions (2 suites)📈 iastaspects - 117/117✅ add_aspectTime: ✅ 103.451µs (SLO: <130.000µs 📉 -20.4%) vs baseline: +1.8% Memory: ✅ 42.920MB (SLO: <43.250MB 🟡 -0.8%) vs baseline: +4.8% ✅ add_inplace_aspectTime: ✅ 102.313µs (SLO: <130.000µs 📉 -21.3%) vs baseline: -0.4% Memory: ✅ 42.920MB (SLO: <43.250MB 🟡 -0.8%) vs baseline: +4.7% ✅ add_inplace_noaspectTime: ✅ 28.127µs (SLO: <40.000µs 📉 -29.7%) vs baseline: -1.1% Memory: ✅ 42.841MB (SLO: <43.500MB 🟡 -1.5%) vs baseline: +4.7% ✅ add_noaspectTime: ✅ 48.920µs (SLO: <70.000µs 📉 -30.1%) vs baseline: +0.3% Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.6% ✅ bytearray_aspectTime: ✅ 249.237µs (SLO: <400.000µs 📉 -37.7%) vs baseline: -0.4% Memory: ✅ 42.861MB (SLO: <43.500MB 🟡 -1.5%) vs baseline: +4.5% ✅ bytearray_extend_aspectTime: ✅ 649.576µs (SLO: <800.000µs 📉 -18.8%) vs baseline: +0.9% Memory: ✅ 42.861MB (SLO: <43.500MB 🟡 -1.5%) vs baseline: +4.5% ✅ bytearray_extend_noaspectTime: ✅ 262.782µs (SLO: <400.000µs 📉 -34.3%) vs baseline: ~same Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.8% ✅ bytearray_noaspectTime: ✅ 137.657µs (SLO: <300.000µs 📉 -54.1%) vs baseline: +1.1% Memory: ✅ 42.880MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +4.6% ✅ bytes_aspectTime: ✅ 219.722µs (SLO: <300.000µs 📉 -26.8%) vs baseline: +1.7% Memory: ✅ 42.880MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +4.5% ✅ bytes_noaspectTime: ✅ 133.748µs (SLO: <200.000µs 📉 -33.1%) vs baseline: -1.2% Memory: ✅ 42.959MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.8% ✅ bytesio_aspectTime: ✅ 3.853ms (SLO: <5.000ms 📉 -22.9%) vs baseline: +0.6% Memory: ✅ 42.900MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +4.7% ✅ bytesio_noaspectTime: ✅ 313.100µs (SLO: <420.000µs 📉 -25.5%) vs baseline: -0.7% Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +5.0% ✅ capitalize_aspectTime: ✅ 88.892µs (SLO: <300.000µs 📉 -70.4%) vs baseline: -0.5% Memory: ✅ 42.959MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.9% ✅ capitalize_noaspectTime: ✅ 249.346µs (SLO: <300.000µs 📉 -16.9%) vs baseline: +1.6% Memory: ✅ 42.880MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +4.5% ✅ casefold_aspectTime: ✅ 89.327µs (SLO: <500.000µs 📉 -82.1%) vs baseline: +0.4% Memory: ✅ 42.998MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +5.0% ✅ casefold_noaspectTime: ✅ 303.640µs (SLO: <500.000µs 📉 -39.3%) vs baseline: +1.2% Memory: ✅ 42.959MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.6% ✅ decode_aspectTime: ✅ 87.377µs (SLO: <100.000µs 📉 -12.6%) vs baseline: +0.1% Memory: ✅ 42.880MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +4.5% ✅ decode_noaspectTime: ✅ 153.383µs (SLO: <210.000µs 📉 -27.0%) vs baseline: -1.4% Memory: ✅ 42.861MB (SLO: <43.500MB 🟡 -1.5%) vs baseline: +4.5% ✅ encode_aspectTime: ✅ 85.853µs (SLO: <200.000µs 📉 -57.1%) vs baseline: +0.9% Memory: ✅ 42.959MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.7% ✅ encode_noaspectTime: ✅ 141.828µs (SLO: <200.000µs 📉 -29.1%) vs baseline: -0.8% Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.5% ✅ format_aspectTime: ✅ 14.672ms (SLO: <19.200ms 📉 -23.6%) vs baseline: -0.4% Memory: ✅ 43.037MB (SLO: <43.250MB 🟡 -0.5%) vs baseline: +4.6% ✅ format_map_aspectTime: ✅ 16.465ms (SLO: <21.500ms 📉 -23.4%) vs baseline: ~same Memory: ✅ 43.136MB (SLO: <43.500MB 🟡 -0.8%) vs baseline: +5.0% ✅ format_map_noaspectTime: ✅ 376.445µs (SLO: <500.000µs 📉 -24.7%) vs baseline: ~same Memory: ✅ 42.920MB (SLO: <43.250MB 🟡 -0.8%) vs baseline: +4.9% ✅ format_noaspectTime: ✅ 302.045µs (SLO: <500.000µs 📉 -39.6%) vs baseline: -1.0% Memory: ✅ 42.861MB (SLO: <43.250MB 🟡 -0.9%) vs baseline: +4.8% ✅ index_aspectTime: ✅ 130.841µs (SLO: <300.000µs 📉 -56.4%) vs baseline: +6.6% Memory: ✅ 42.939MB (SLO: <43.250MB 🟡 -0.7%) vs baseline: +4.7% ✅ index_noaspectTime: ✅ 40.265µs (SLO: <300.000µs 📉 -86.6%) vs baseline: -0.2% Memory: ✅ 42.979MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.8% ✅ join_aspectTime: ✅ 214.400µs (SLO: <300.000µs 📉 -28.5%) vs baseline: -0.7% Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.7% ✅ join_noaspectTime: ✅ 143.520µs (SLO: <300.000µs 📉 -52.2%) vs baseline: +0.6% Memory: ✅ 42.841MB (SLO: <43.250MB 🟡 -0.9%) vs baseline: +4.5% ✅ ljust_aspectTime: ✅ 598.260µs (SLO: <700.000µs 📉 -14.5%) vs baseline: 📈 +17.8% Memory: ✅ 42.841MB (SLO: <43.250MB 🟡 -0.9%) vs baseline: +4.4% ✅ ljust_noaspectTime: ✅ 255.343µs (SLO: <300.000µs 📉 -14.9%) vs baseline: -1.1% Memory: ✅ 42.900MB (SLO: <43.250MB 🟡 -0.8%) vs baseline: +4.5% ✅ lower_aspectTime: ✅ 292.438µs (SLO: <500.000µs 📉 -41.5%) vs baseline: +0.5% Memory: ✅ 42.959MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.6% ✅ lower_noaspectTime: ✅ 230.750µs (SLO: <300.000µs 📉 -23.1%) vs baseline: +1.4% Memory: ✅ 42.900MB (SLO: <43.250MB 🟡 -0.8%) vs baseline: +4.6% ✅ lstrip_aspectTime: ✅ 0.275ms (SLO: <3.000ms 📉 -90.8%) vs baseline: -0.4% Memory: ✅ 42.920MB (SLO: <43.250MB 🟡 -0.8%) vs baseline: +4.5% ✅ lstrip_noaspectTime: ✅ 0.176ms (SLO: <3.000ms 📉 -94.1%) vs baseline: ~same Memory: ✅ 42.880MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +4.6% ✅ modulo_aspectTime: ✅ 14.380ms (SLO: <18.750ms 📉 -23.3%) vs baseline: ~same Memory: ✅ 43.037MB (SLO: <43.500MB 🟡 -1.1%) vs baseline: +4.7% ✅ modulo_aspect_for_bytearray_bytearrayTime: ✅ 14.839ms (SLO: <19.350ms 📉 -23.3%) vs baseline: ~same Memory: ✅ 43.096MB (SLO: <43.500MB 🟡 -0.9%) vs baseline: +4.9% ✅ modulo_aspect_for_bytesTime: ✅ 14.464ms (SLO: <18.900ms 📉 -23.5%) vs baseline: ~same Memory: ✅ 43.136MB (SLO: <43.500MB 🟡 -0.8%) vs baseline: +4.9% ✅ modulo_aspect_for_bytes_bytearrayTime: ✅ 14.598ms (SLO: <19.150ms 📉 -23.8%) vs baseline: -0.7% Memory: ✅ 43.096MB (SLO: <43.500MB 🟡 -0.9%) vs baseline: +5.1% ✅ modulo_noaspectTime: ✅ 0.357ms (SLO: <3.000ms 📉 -88.1%) vs baseline: +0.5% Memory: ✅ 42.821MB (SLO: <43.500MB 🟡 -1.6%) vs baseline: +4.3% ✅ replace_aspectTime: ✅ 18.381ms (SLO: <24.000ms 📉 -23.4%) vs baseline: -0.3% Memory: ✅ 43.018MB (SLO: <44.000MB -2.2%) vs baseline: +4.7% ✅ replace_noaspectTime: ✅ 282.593µs (SLO: <300.000µs -5.8%) vs baseline: +0.5% Memory: ✅ 42.900MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +4.8% ✅ repr_aspectTime: ✅ 318.843µs (SLO: <420.000µs 📉 -24.1%) vs baseline: +0.8% Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.5% ✅ repr_noaspectTime: ✅ 46.564µs (SLO: <90.000µs 📉 -48.3%) vs baseline: -0.4% Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.6% ✅ rstrip_aspectTime: ✅ 390.246µs (SLO: <500.000µs 📉 -22.0%) vs baseline: -0.5% Memory: ✅ 42.979MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.6% ✅ rstrip_noaspectTime: ✅ 182.604µs (SLO: <300.000µs 📉 -39.1%) vs baseline: +0.8% Memory: ✅ 42.880MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +4.4% ✅ slice_aspectTime: ✅ 182.894µs (SLO: <300.000µs 📉 -39.0%) vs baseline: +1.0% Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.7% ✅ slice_noaspectTime: ✅ 53.866µs (SLO: <90.000µs 📉 -40.1%) vs baseline: +0.3% Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.8% ✅ stringio_aspectTime: ✅ 4.481ms (SLO: <5.000ms 📉 -10.4%) vs baseline: 📈 +14.8% Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.5% ✅ stringio_noaspectTime: ✅ 345.841µs (SLO: <500.000µs 📉 -30.8%) vs baseline: +0.2% ✅ strip_aspectTime: ✅ 272.504µs (SLO: <350.000µs 📉 -22.1%) vs baseline: -1.5% Memory: ✅ 42.900MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +4.5% ✅ strip_noaspectTime: ✅ 178.075µs (SLO: <240.000µs 📉 -25.8%) vs baseline: +0.2% Memory: ✅ 42.880MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +4.9% ✅ swapcase_aspectTime: ✅ 326.552µs (SLO: <500.000µs 📉 -34.7%) vs baseline: -0.8% Memory: ✅ 42.900MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +4.6% ✅ swapcase_noaspectTime: ✅ 266.143µs (SLO: <400.000µs 📉 -33.5%) vs baseline: +1.9% Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.9% ✅ title_aspectTime: ✅ 316.803µs (SLO: <500.000µs 📉 -36.6%) vs baseline: ~same Memory: ✅ 42.900MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +4.5% ✅ title_noaspectTime: ✅ 254.705µs (SLO: <400.000µs 📉 -36.3%) vs baseline: +1.7% Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.9% ✅ translate_aspectTime: ✅ 488.474µs (SLO: <700.000µs 📉 -30.2%) vs baseline: +0.9% Memory: ✅ 42.880MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +4.5% ✅ translate_noaspectTime: ✅ 423.256µs (SLO: <500.000µs 📉 -15.3%) vs baseline: -0.6% Memory: ✅ 42.959MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.9% ✅ upper_aspectTime: ✅ 290.243µs (SLO: <500.000µs 📉 -42.0%) vs baseline: -0.1% Memory: ✅ 42.900MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +4.7% ✅ upper_noaspectTime: ✅ 226.599µs (SLO: <400.000µs 📉 -43.4%) vs baseline: -0.1% Memory: ✅ 42.861MB (SLO: <43.500MB 🟡 -1.5%) vs baseline: +4.6% 📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 516.764µs (SLO: <700.000µs 📉 -26.2%) vs baseline: 📈 +23.3% Memory: ✅ 42.861MB (SLO: <43.500MB 🟡 -1.5%) vs baseline: +5.1% ✅ ospathbasename_noaspectTime: ✅ 423.353µs (SLO: <700.000µs 📉 -39.5%) vs baseline: -0.7% Memory: ✅ 42.861MB (SLO: <43.500MB 🟡 -1.5%) vs baseline: +5.3% ✅ ospathjoin_aspectTime: ✅ 610.445µs (SLO: <700.000µs 📉 -12.8%) vs baseline: -0.6% Memory: ✅ 42.939MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +5.8% ✅ ospathjoin_noaspectTime: ✅ 616.782µs (SLO: <700.000µs 📉 -11.9%) vs baseline: -0.6% Memory: ✅ 42.880MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +5.2% ✅ ospathnormcase_aspectTime: ✅ 345.254µs (SLO: <700.000µs 📉 -50.7%) vs baseline: -0.8% Memory: ✅ 42.841MB (SLO: <43.500MB 🟡 -1.5%) vs baseline: +5.4% ✅ ospathnormcase_noaspectTime: ✅ 350.375µs (SLO: <700.000µs 📉 -49.9%) vs baseline: +0.2% Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +5.7% ✅ ospathsplit_aspectTime: ✅ 479.428µs (SLO: <700.000µs 📉 -31.5%) vs baseline: +1.1% Memory: ✅ 42.841MB (SLO: <43.500MB 🟡 -1.5%) vs baseline: +5.3% ✅ ospathsplit_noaspectTime: ✅ 488.730µs (SLO: <700.000µs 📉 -30.2%) vs baseline: +0.4% Memory: ✅ 42.959MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +5.4% ✅ ospathsplitdrive_aspectTime: ✅ 373.362µs (SLO: <700.000µs 📉 -46.7%) vs baseline: +1.0% Memory: ✅ 42.880MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +5.1% ✅ ospathsplitdrive_noaspectTime: ✅ 72.768µs (SLO: <700.000µs 📉 -89.6%) vs baseline: -0.3% Memory: ✅ 42.861MB (SLO: <43.500MB 🟡 -1.5%) vs baseline: +5.5% ✅ ospathsplitext_aspectTime: ✅ 456.508µs (SLO: <700.000µs 📉 -34.8%) vs baseline: ~same Memory: ✅ 42.939MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +5.3% ✅ ospathsplitext_noaspectTime: ✅ 464.727µs (SLO: <700.000µs 📉 -33.6%) vs baseline: +0.8% Memory: ✅ 42.841MB (SLO: <43.500MB 🟡 -1.5%) vs baseline: +5.3% 🟡 Near SLO Breach (14 suites)🟡 djangosimple - 30/30✅ appsecTime: ✅ 19.651ms (SLO: <22.300ms 📉 -11.9%) vs baseline: -0.2% Memory: ✅ 68.773MB (SLO: <70.500MB -2.4%) vs baseline: +5.0% ✅ exception-replay-enabledTime: ✅ 1.387ms (SLO: <1.450ms -4.3%) vs baseline: +0.4% Memory: ✅ 66.831MB (SLO: <67.500MB 🟡 -1.0%) vs baseline: +4.7% ✅ iastTime: ✅ 19.644ms (SLO: <22.250ms 📉 -11.7%) vs baseline: -0.2% Memory: ✅ 68.783MB (SLO: <71.000MB -3.1%) vs baseline: +5.1% ✅ profilerTime: ✅ 15.280ms (SLO: <16.550ms -7.7%) vs baseline: -0.1% Memory: ✅ 56.957MB (SLO: <57.500MB 🟡 -0.9%) vs baseline: +4.8% ✅ resource-renamingTime: ✅ 19.614ms (SLO: <21.750ms -9.8%) vs baseline: +0.1% Memory: ✅ 68.793MB (SLO: <70.500MB -2.4%) vs baseline: +4.8% ✅ span-code-originTime: ✅ 20.106ms (SLO: <28.200ms 📉 -28.7%) vs baseline: +1.4% Memory: ✅ 68.643MB (SLO: <71.000MB -3.3%) vs baseline: +4.9% ✅ tracerTime: ✅ 19.677ms (SLO: <21.750ms -9.5%) vs baseline: -0.2% Memory: ✅ 68.773MB (SLO: <71.000MB -3.1%) vs baseline: +5.2% ✅ tracer-and-profilerTime: ✅ 21.749ms (SLO: <23.500ms -7.4%) vs baseline: +0.2% Memory: ✅ 70.228MB (SLO: <71.000MB 🟡 -1.1%) vs baseline: +4.7% ✅ tracer-dont-create-db-spansTime: ✅ 19.706ms (SLO: <21.500ms -8.3%) vs baseline: ~same Memory: ✅ 68.734MB (SLO: <71.000MB -3.2%) vs baseline: +4.8% ✅ tracer-minimalTime: ✅ 16.832ms (SLO: <17.500ms -3.8%) vs baseline: +0.6% Memory: ✅ 68.699MB (SLO: <71.000MB -3.2%) vs baseline: +4.9% ✅ tracer-nativeTime: ✅ 19.570ms (SLO: <21.750ms 📉 -10.0%) vs baseline: ~same Memory: ✅ 68.754MB (SLO: <72.500MB -5.2%) vs baseline: +4.9% ✅ tracer-no-cachesTime: ✅ 17.659ms (SLO: <19.650ms 📉 -10.1%) vs baseline: ~same Memory: ✅ 68.636MB (SLO: <71.000MB -3.3%) vs baseline: +4.9% ✅ tracer-no-databasesTime: ✅ 19.165ms (SLO: <20.100ms -4.7%) vs baseline: ~same Memory: ✅ 68.734MB (SLO: <71.000MB -3.2%) vs baseline: +5.1% ✅ tracer-no-middlewareTime: ✅ 19.404ms (SLO: <21.500ms -9.7%) vs baseline: +0.1% Memory: ✅ 68.714MB (SLO: <71.000MB -3.2%) vs baseline: +4.8% ✅ tracer-no-templatesTime: ✅ 19.618ms (SLO: <22.000ms 📉 -10.8%) vs baseline: +0.5% Memory: ✅ 68.751MB (SLO: <70.500MB -2.5%) vs baseline: +4.9% 🟡 errortrackingdjangosimple - 6/6✅ errortracking-enabled-allTime: ✅ 16.309ms (SLO: <19.850ms 📉 -17.8%) vs baseline: -0.3% Memory: ✅ 70.227MB (SLO: <71.000MB 🟡 -1.1%) vs baseline: +4.9% ✅ errortracking-enabled-userTime: ✅ 16.337ms (SLO: <19.400ms 📉 -15.8%) vs baseline: -0.4% Memory: ✅ 70.366MB (SLO: <71.000MB 🟡 -0.9%) vs baseline: +5.2% ✅ tracer-enabledTime: ✅ 16.315ms (SLO: <19.450ms 📉 -16.1%) vs baseline: -0.3% Memory: ✅ 70.401MB (SLO: <71.000MB 🟡 -0.8%) vs baseline: +5.1% 🟡 errortrackingflasksqli - 6/6✅ errortracking-enabled-allTime: ✅ 2.089ms (SLO: <2.300ms -9.2%) vs baseline: ~same Memory: ✅ 56.230MB (SLO: <56.500MB 🟡 -0.5%) vs baseline: +5.2% ✅ errortracking-enabled-userTime: ✅ 2.095ms (SLO: <2.250ms -6.9%) vs baseline: ~same Memory: ✅ 56.269MB (SLO: <56.500MB 🟡 -0.4%) vs baseline: +5.1% ✅ tracer-enabledTime: ✅ 2.084ms (SLO: <2.300ms -9.4%) vs baseline: -0.1% Memory: ✅ 56.289MB (SLO: <56.500MB 🟡 -0.4%) vs baseline: +5.3% 🟡 flasksimple - 18/18✅ appsec-getTime: ✅ 3.399ms (SLO: <4.750ms 📉 -28.4%) vs baseline: -0.2% Memory: ✅ 56.343MB (SLO: <66.500MB 📉 -15.3%) vs baseline: +5.3% ✅ appsec-postTime: ✅ 2.885ms (SLO: <6.750ms 📉 -57.3%) vs baseline: +0.2% Memory: ✅ 56.364MB (SLO: <66.500MB 📉 -15.2%) vs baseline: +5.5% ✅ appsec-telemetryTime: ✅ 3.437ms (SLO: <4.750ms 📉 -27.6%) vs baseline: +0.7% Memory: ✅ 56.325MB (SLO: <66.500MB 📉 -15.3%) vs baseline: +5.3% ✅ debuggerTime: ✅ 1.870ms (SLO: <2.000ms -6.5%) vs baseline: ~same Memory: ✅ 48.223MB (SLO: <49.500MB -2.6%) vs baseline: +5.2% ✅ iast-getTime: ✅ 1.860ms (SLO: <2.000ms -7.0%) vs baseline: ~same Memory: ✅ 45.256MB (SLO: <49.000MB -7.6%) vs baseline: +5.3% ✅ profilerTime: ✅ 1.907ms (SLO: <2.100ms -9.2%) vs baseline: -0.3% Memory: ✅ 49.206MB (SLO: <50.000MB 🟡 -1.6%) vs baseline: +4.8% ✅ resource-renamingTime: ✅ 3.385ms (SLO: <3.650ms -7.3%) vs baseline: ~same Memory: ✅ 56.386MB (SLO: <57.000MB 🟡 -1.1%) vs baseline: +5.4% ✅ tracerTime: ✅ 3.395ms (SLO: <3.650ms -7.0%) vs baseline: -0.3% Memory: ✅ 56.387MB (SLO: <56.500MB 🟡 -0.2%) vs baseline: +5.4% ✅ tracer-nativeTime: ✅ 3.392ms (SLO: <3.650ms -7.1%) vs baseline: ~same Memory: ✅ 56.344MB (SLO: <60.000MB -6.1%) vs baseline: +5.2% 🟡 flasksqli - 6/6✅ appsec-enabledTime: ✅ 2.080ms (SLO: <4.200ms 📉 -50.5%) vs baseline: -0.1% Memory: ✅ 56.309MB (SLO: <66.000MB 📉 -14.7%) vs baseline: +5.2% ✅ iast-enabledTime: ✅ 2.092ms (SLO: <2.800ms 📉 -25.3%) vs baseline: +0.4% Memory: ✅ 56.328MB (SLO: <62.500MB -9.9%) vs baseline: +5.4% ✅ tracer-enabledTime: ✅ 2.080ms (SLO: <2.250ms -7.6%) vs baseline: +0.1% Memory: ✅ 56.250MB (SLO: <56.500MB 🟡 -0.4%) vs baseline: +5.2% 🟡 httppropagationextract - 60/60✅ all_styles_all_headersTime: ✅ 80.433µs (SLO: <100.000µs 📉 -19.6%) vs baseline: -0.4% Memory: ✅ 35.330MB (SLO: <36.000MB 🟡 -1.9%) vs baseline: +5.3% ✅ b3_headersTime: ✅ 14.127µs (SLO: <20.000µs 📉 -29.4%) vs baseline: -0.2% Memory: ✅ 35.291MB (SLO: <36.000MB 🟡 -2.0%) vs baseline: +5.2% ✅ b3_single_headersTime: ✅ 13.248µs (SLO: <20.000µs 📉 -33.8%) vs baseline: -0.4% Memory: ✅ 35.252MB (SLO: <36.000MB -2.1%) vs baseline: +4.9% ✅ datadog_tracecontext_tracestate_not_propagated_on_trace_id_no_matchTime: ✅ 63.904µs (SLO: <80.000µs 📉 -20.1%) vs baseline: ~same Memory: ✅ 35.409MB (SLO: <36.000MB 🟡 -1.6%) vs baseline: +5.2% ✅ datadog_tracecontext_tracestate_propagated_on_trace_id_matchTime: ✅ 68.995µs (SLO: <80.000µs 📉 -13.8%) vs baseline: +4.8% Memory: ✅ 35.134MB (SLO: <36.000MB -2.4%) vs baseline: +4.6% ✅ empty_headersTime: ✅ 1.613µs (SLO: <10.000µs 📉 -83.9%) vs baseline: +1.0% Memory: ✅ 35.291MB (SLO: <36.000MB 🟡 -2.0%) vs baseline: +5.3% ✅ full_t_id_datadog_headersTime: ✅ 22.420µs (SLO: <30.000µs 📉 -25.3%) vs baseline: +0.4% Memory: ✅ 35.311MB (SLO: <36.000MB 🟡 -1.9%) vs baseline: +5.2% ✅ invalid_priority_headerTime: ✅ 6.530µs (SLO: <10.000µs 📉 -34.7%) vs baseline: ~same Memory: ✅ 35.291MB (SLO: <36.000MB 🟡 -2.0%) vs baseline: +4.9% ✅ invalid_span_id_headerTime: ✅ 6.536µs (SLO: <10.000µs 📉 -34.6%) vs baseline: -0.2% Memory: ✅ 35.252MB (SLO: <36.000MB -2.1%) vs baseline: +5.0% ✅ invalid_tags_headerTime: ✅ 6.522µs (SLO: <10.000µs 📉 -34.8%) vs baseline: -0.6% Memory: ✅ 35.330MB (SLO: <36.000MB 🟡 -1.9%) vs baseline: +4.9% ✅ invalid_trace_id_headerTime: ✅ 6.505µs (SLO: <10.000µs 📉 -34.9%) vs baseline: -0.8% Memory: ✅ 35.232MB (SLO: <36.000MB -2.1%) vs baseline: +4.7% ✅ large_header_no_matchesTime: ✅ 27.799µs (SLO: <30.000µs -7.3%) vs baseline: -0.4% Memory: ✅ 35.232MB (SLO: <36.000MB -2.1%) vs baseline: +4.9% ✅ large_valid_headers_allTime: ✅ 28.756µs (SLO: <40.000µs 📉 -28.1%) vs baseline: -0.9% Memory: ✅ 35.252MB (SLO: <36.000MB -2.1%) vs baseline: +4.7% ✅ medium_header_no_matchesTime: ✅ 9.780µs (SLO: <20.000µs 📉 -51.1%) vs baseline: -0.2% Memory: ✅ 35.212MB (SLO: <36.000MB -2.2%) vs baseline: +4.8% ✅ medium_valid_headers_allTime: ✅ 11.308µs (SLO: <20.000µs 📉 -43.5%) vs baseline: +0.5% Memory: ✅ 35.212MB (SLO: <36.000MB -2.2%) vs baseline: +5.0% ✅ none_propagation_styleTime: ✅ 1.700µs (SLO: <10.000µs 📉 -83.0%) vs baseline: ~same Memory: ✅ 35.271MB (SLO: <36.000MB -2.0%) vs baseline: +5.2% ✅ tracecontext_headersTime: ✅ 34.526µs (SLO: <40.000µs 📉 -13.7%) vs baseline: -0.6% Memory: ✅ 35.252MB (SLO: <36.000MB -2.1%) vs baseline: +4.5% ✅ valid_headers_allTime: ✅ 6.522µs (SLO: <10.000µs 📉 -34.8%) vs baseline: ~same Memory: ✅ 35.173MB (SLO: <36.000MB -2.3%) vs baseline: +4.7% ✅ valid_headers_basicTime: ✅ 6.101µs (SLO: <10.000µs 📉 -39.0%) vs baseline: -0.1% Memory: ✅ 35.271MB (SLO: <36.000MB -2.0%) vs baseline: +5.1% ✅ wsgi_empty_headersTime: ✅ 1.603µs (SLO: <10.000µs 📉 -84.0%) vs baseline: ~same Memory: ✅ 35.350MB (SLO: <36.000MB 🟡 -1.8%) vs baseline: +5.3% ✅ wsgi_invalid_priority_headerTime: ✅ 6.557µs (SLO: <10.000µs 📉 -34.4%) vs baseline: -0.8% Memory: ✅ 35.232MB (SLO: <36.000MB -2.1%) vs baseline: +4.9% ✅ wsgi_invalid_span_id_headerTime: ✅ 1.594µs (SLO: <10.000µs 📉 -84.1%) vs baseline: -0.1% Memory: ✅ 35.252MB (SLO: <36.000MB -2.1%) vs baseline: +5.0% ✅ wsgi_invalid_tags_headerTime: ✅ 6.558µs (SLO: <10.000µs 📉 -34.4%) vs baseline: +0.2% Memory: ✅ 35.212MB (SLO: <36.000MB -2.2%) vs baseline: +4.7% ✅ wsgi_invalid_trace_id_headerTime: ✅ 6.642µs (SLO: <10.000µs 📉 -33.6%) vs baseline: +1.1% Memory: ✅ 35.173MB (SLO: <36.000MB -2.3%) vs baseline: +4.4% ✅ wsgi_large_header_no_matchesTime: ✅ 28.931µs (SLO: <40.000µs 📉 -27.7%) vs baseline: ~same Memory: ✅ 35.606MB (SLO: <36.000MB 🟡 -1.1%) vs baseline: +6.0% ✅ wsgi_large_valid_headers_allTime: ✅ 29.968µs (SLO: <40.000µs 📉 -25.1%) vs baseline: ~same Memory: ✅ 35.547MB (SLO: <36.000MB 🟡 -1.3%) vs baseline: +5.9% ✅ wsgi_medium_header_no_matchesTime: ✅ 10.103µs (SLO: <20.000µs 📉 -49.5%) vs baseline: -0.2% Memory: ✅ 35.193MB (SLO: <36.000MB -2.2%) vs baseline: +4.6% ✅ wsgi_medium_valid_headers_allTime: ✅ 11.638µs (SLO: <20.000µs 📉 -41.8%) vs baseline: +0.1% Memory: ✅ 35.271MB (SLO: <36.000MB -2.0%) vs baseline: +4.6% ✅ wsgi_valid_headers_allTime: ✅ 6.557µs (SLO: <10.000µs 📉 -34.4%) vs baseline: -0.3% Memory: ✅ 35.173MB (SLO: <36.000MB -2.3%) vs baseline: +4.6% ✅ wsgi_valid_headers_basicTime: ✅ 6.139µs (SLO: <10.000µs 📉 -38.6%) vs baseline: +0.2% Memory: ✅ 35.193MB (SLO: <36.000MB -2.2%) vs baseline: +4.8% 🟡 httppropagationinject - 16/16✅ ids_onlyTime: ✅ 22.084µs (SLO: <30.000µs 📉 -26.4%) vs baseline: +6.0% Memory: ✅ 35.271MB (SLO: <36.000MB -2.0%) vs baseline: +4.9% ✅ with_allTime: ✅ 27.575µs (SLO: <40.000µs 📉 -31.1%) vs baseline: -0.6% Memory: ✅ 35.291MB (SLO: <36.000MB 🟡 -2.0%) vs baseline: +5.0% ✅ with_dd_originTime: ✅ 24.547µs (SLO: <30.000µs 📉 -18.2%) vs baseline: -0.6% Memory: ✅ 35.212MB (SLO: <36.000MB -2.2%) vs baseline: +4.7% ✅ with_priority_and_originTime: ✅ 23.864µs (SLO: <40.000µs 📉 -40.3%) vs baseline: -0.6% Memory: ✅ 35.291MB (SLO: <36.000MB 🟡 -2.0%) vs baseline: +5.0% ✅ with_sampling_priorityTime: ✅ 20.654µs (SLO: <30.000µs 📉 -31.2%) vs baseline: -1.1% Memory: ✅ 35.389MB (SLO: <36.000MB 🟡 -1.7%) vs baseline: +5.0% ✅ with_tagsTime: ✅ 25.580µs (SLO: <40.000µs 📉 -36.1%) vs baseline: -1.4% Memory: ✅ 35.311MB (SLO: <36.000MB 🟡 -1.9%) vs baseline: +4.8% ✅ with_tags_invalidTime: ✅ 27.190µs (SLO: <40.000µs 📉 -32.0%) vs baseline: -0.8% Memory: ✅ 35.350MB (SLO: <36.000MB 🟡 -1.8%) vs baseline: +5.0% ✅ with_tags_max_sizeTime: ✅ 26.255µs (SLO: <40.000µs 📉 -34.4%) vs baseline: -0.2% Memory: ✅ 35.291MB (SLO: <36.000MB 🟡 -2.0%) vs baseline: +4.9% 🟡 iast_aspects - 40/40✅ re_expand_aspectTime: ✅ 3.736ms (SLO: <4.880ms 📉 -23.5%) vs baseline: +6.3% Memory: ✅ 42.880MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +5.3% ✅ re_expand_noaspectTime: ✅ 3.508ms (SLO: <4.620ms 📉 -24.1%) vs baseline: ~same Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +5.3% ✅ re_findall_aspectTime: ✅ 340.445µs (SLO: <450.000µs 📉 -24.3%) vs baseline: +0.9% Memory: ✅ 42.861MB (SLO: <43.500MB 🟡 -1.5%) vs baseline: +5.3% ✅ re_findall_noaspectTime: ✅ 323.860µs (SLO: <430.000µs 📉 -24.7%) vs baseline: +0.5% Memory: ✅ 42.861MB (SLO: <43.500MB 🟡 -1.5%) vs baseline: +5.4% ✅ re_finditer_aspectTime: ✅ 454.875µs (SLO: <590.000µs 📉 -22.9%) vs baseline: +0.3% Memory: ✅ 42.880MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +5.1% ✅ re_finditer_noaspectTime: ✅ 326.254µs (SLO: <440.000µs 📉 -25.9%) vs baseline: +0.2% Memory: ✅ 42.979MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +5.6% ✅ re_fullmatch_aspectTime: ✅ 278.849µs (SLO: <360.000µs 📉 -22.5%) vs baseline: -0.6% Memory: ✅ 42.939MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +5.1% ✅ re_fullmatch_noaspectTime: ✅ 307.445µs (SLO: <400.000µs 📉 -23.1%) vs baseline: +0.2% Memory: ✅ 42.959MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +5.8% ✅ re_group_aspectTime: ✅ 483.590µs (SLO: <630.000µs 📉 -23.2%) vs baseline: +0.3% Memory: ✅ 42.821MB (SLO: <43.500MB 🟡 -1.6%) vs baseline: +5.3% ✅ re_group_noaspectTime: ✅ 489.342µs (SLO: <640.000µs 📉 -23.5%) vs baseline: +0.9% Memory: ✅ 42.979MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +5.7% ✅ re_groups_aspectTime: ✅ 494.444µs (SLO: <650.000µs 📉 -23.9%) vs baseline: ~same Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +5.1% ✅ re_groups_noaspectTime: ✅ 501.017µs (SLO: <650.000µs 📉 -22.9%) vs baseline: +0.3% Memory: ✅ 42.959MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +5.7% ✅ re_match_aspectTime: ✅ 284.326µs (SLO: <370.000µs 📉 -23.2%) vs baseline: -0.4% Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +5.0% ✅ re_match_noaspectTime: ✅ 311.739µs (SLO: <410.000µs 📉 -24.0%) vs baseline: +0.6% Memory: ✅ 42.939MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +5.4% ✅ re_search_aspectTime: ✅ 276.474µs (SLO: <350.000µs 📉 -21.0%) vs baseline: +3.8% Memory: ✅ 42.939MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +5.6% ✅ re_search_noaspectTime: ✅ 288.768µs (SLO: <380.000µs 📉 -24.0%) vs baseline: +0.3% Memory: ✅ 42.900MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +5.6% ✅ re_sub_aspectTime: ✅ 354.291µs (SLO: <460.000µs 📉 -23.0%) vs baseline: ~same Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +5.3% ✅ re_sub_noaspectTime: ✅ 391.827µs (SLO: <520.000µs 📉 -24.6%) vs baseline: +0.2% Memory: ✅ 42.900MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +5.5% ✅ re_subn_aspectTime: ✅ 372.674µs (SLO: <490.000µs 📉 -23.9%) vs baseline: ~same Memory: ✅ 42.880MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +5.4% ✅ re_subn_noaspectTime: ✅ 403.366µs (SLO: <530.000µs 📉 -23.9%) vs baseline: ~same Memory: ✅ 42.841MB (SLO: <43.500MB 🟡 -1.5%) vs baseline: +5.1% 🟡 ratelimiter - 12/12✅ defaultsTime: ✅ 2.401µs (SLO: <10.000µs 📉 -76.0%) vs baseline: +0.6% Memory: ✅ 35.429MB (SLO: <36.000MB 🟡 -1.6%) vs baseline: +4.3% ✅ high_rate_limitTime: ✅ 2.447µs (SLO: <10.000µs 📉 -75.5%) vs baseline: +1.0% Memory: ✅ 35.409MB (SLO: <36.000MB 🟡 -1.6%) vs baseline: +4.5% ✅ long_windowTime: ✅ 2.406µs (SLO: <10.000µs 📉 -75.9%) vs baseline: +0.9% Memory: ✅ 35.448MB (SLO: <36.000MB 🟡 -1.5%) vs baseline: +4.6% ✅ low_rate_limitTime: ✅ 2.401µs (SLO: <10.000µs 📉 -76.0%) vs baseline: +0.2% Memory: ✅ 35.429MB (SLO: <36.000MB 🟡 -1.6%) vs baseline: +4.5% ✅ no_rate_limitTime: ✅ 0.830µs (SLO: <10.000µs 📉 -91.7%) vs baseline: -0.1% Memory: ✅ 35.448MB (SLO: <36.000MB 🟡 -1.5%) vs baseline: +4.6% ✅ short_windowTime: ✅ 2.520µs (SLO: <10.000µs 📉 -74.8%) vs baseline: ~same Memory: ✅ 35.429MB (SLO: <36.000MB 🟡 -1.6%) vs baseline: +4.6% 🟡 recursivecomputation - 8/8✅ deepTime: ✅ 310.257ms (SLO: <320.950ms -3.3%) vs baseline: +0.4% Memory: ✅ 36.313MB (SLO: <36.500MB 🟡 -0.5%) vs baseline: +4.9% ✅ deep-profiledTime: ✅ 325.207ms (SLO: <359.150ms -9.5%) vs baseline: -0.3% Memory: ✅ 40.344MB (SLO: <40.500MB 🟡 -0.4%) vs baseline: +5.2% ✅ mediumTime: ✅ 7.067ms (SLO: <7.400ms -4.5%) vs baseline: ~same Memory: ✅ 35.212MB (SLO: <36.000MB -2.2%) vs baseline: +5.0% ✅ shallowTime: ✅ 0.968ms (SLO: <1.050ms -7.9%) vs baseline: +1.6% Memory: ✅ 35.173MB (SLO: <36.000MB -2.3%) vs baseline: +4.7% 🟡 sethttpmeta - 32/32✅ all-disabledTime: ✅ 10.496µs (SLO: <20.000µs 📉 -47.5%) vs baseline: -1.6% Memory: ✅ 36.078MB (SLO: <36.500MB 🟡 -1.2%) vs baseline: +5.0% ✅ all-enabledTime: ✅ 40.777µs (SLO: <50.000µs 📉 -18.4%) vs baseline: +1.7% Memory: ✅ 36.019MB (SLO: <36.500MB 🟡 -1.3%) vs baseline: +4.7% ✅ collectipvariant_existsTime: ✅ 40.756µs (SLO: <50.000µs 📉 -18.5%) vs baseline: -0.7% Memory: ✅ 35.999MB (SLO: <36.500MB 🟡 -1.4%) vs baseline: +5.1% ✅ no-collectipvariantTime: ✅ 40.051µs (SLO: <50.000µs 📉 -19.9%) vs baseline: -0.2% Memory: ✅ 36.117MB (SLO: <36.500MB 🟡 -1.0%) vs baseline: +5.2% ✅ no-useragentvariantTime: ✅ 38.700µs (SLO: <50.000µs 📉 -22.6%) vs baseline: -0.5% Memory: ✅ 36.019MB (SLO: <36.500MB 🟡 -1.3%) vs baseline: +5.0% ✅ obfuscation-no-queryTime: ✅ 40.477µs (SLO: <50.000µs 📉 -19.0%) vs baseline: -0.4% Memory: ✅ 35.999MB (SLO: <36.500MB 🟡 -1.4%) vs baseline: +4.9% ✅ obfuscation-regular-case-explicit-queryTime: ✅ 75.837µs (SLO: <90.000µs 📉 -15.7%) vs baseline: +0.2% Memory: ✅ 36.078MB (SLO: <36.500MB 🟡 -1.2%) vs baseline: +5.0% ✅ obfuscation-regular-case-implicit-queryTime: ✅ 76.407µs (SLO: <90.000µs 📉 -15.1%) vs baseline: ~same Memory: ✅ 35.979MB (SLO: <36.500MB 🟡 -1.4%) vs baseline: +4.7% ✅ obfuscation-send-querystring-disabledTime: ✅ 153.852µs (SLO: <170.000µs -9.5%) vs baseline: -0.2% Memory: ✅ 36.137MB (SLO: <36.500MB 🟡 -1.0%) vs baseline: +5.3% ✅ obfuscation-worst-case-explicit-queryTime: ✅ 148.721µs (SLO: <160.000µs -7.0%) vs baseline: ~same Memory: ✅ 36.038MB (SLO: <36.500MB 🟡 -1.3%) vs baseline: +4.9% ✅ obfuscation-worst-case-implicit-queryTime: ✅ 155.057µs (SLO: <170.000µs -8.8%) vs baseline: +0.3% Memory: ✅ 35.960MB (SLO: <36.500MB 🟡 -1.5%) vs baseline: +4.4% ✅ useragentvariant_exists_1Time: ✅ 39.826µs (SLO: <50.000µs 📉 -20.3%) vs baseline: +0.4% Memory: ✅ 36.038MB (SLO: <36.500MB 🟡 -1.3%) vs baseline: +4.9% ✅ useragentvariant_exists_2Time: ✅ 40.580µs (SLO: <50.000µs 📉 -18.8%) vs baseline: -0.5% Memory: ✅ 35.960MB (SLO: <36.500MB 🟡 -1.5%) vs baseline: +4.7% ✅ useragentvariant_exists_3Time: ✅ 40.080µs (SLO: <50.000µs 📉 -19.8%) vs baseline: -0.2% Memory: ✅ 35.999MB (SLO: <36.500MB 🟡 -1.4%) vs baseline: +4.7% ✅ useragentvariant_not_exists_1Time: ✅ 39.564µs (SLO: <50.000µs 📉 -20.9%) vs baseline: +0.2% Memory: ✅ 36.058MB (SLO: <36.500MB 🟡 -1.2%) vs baseline: +5.1% ✅ useragentvariant_not_exists_2Time: ✅ 39.676µs (SLO: <50.000µs 📉 -20.6%) vs baseline: ~same Memory: ✅ 36.038MB (SLO: <36.000MB +0.1%) vs baseline: +5.0% 🟡 span - 26/26✅ add-eventTime: ✅ 19.757ms (SLO: <22.500ms 📉 -12.2%) vs baseline: -0.3% Memory: ✅ 37.320MB (SLO: <53.000MB 📉 -29.6%) vs baseline: +4.9% ✅ add-metricsTime: ✅ 90.334ms (SLO: <93.500ms -3.4%) vs baseline: +0.4% Memory: ✅ 41.532MB (SLO: <53.000MB 📉 -21.6%) vs baseline: +4.4% ✅ add-tagsTime: ✅ 142.931ms (SLO: <155.000ms -7.8%) vs baseline: -1.0% Memory: ✅ 41.451MB (SLO: <53.000MB 📉 -21.8%) vs baseline: +4.8% ✅ get-contextTime: ✅ 17.757ms (SLO: <20.500ms 📉 -13.4%) vs baseline: -0.4% Memory: ✅ 37.080MB (SLO: <53.000MB 📉 -30.0%) vs baseline: +4.6% ✅ is-recordingTime: ✅ 18.100ms (SLO: <20.500ms 📉 -11.7%) vs baseline: ~same Memory: ✅ 37.159MB (SLO: <53.000MB 📉 -29.9%) vs baseline: +4.3% ✅ record-exceptionTime: ✅ 39.111ms (SLO: <41.000ms -4.6%) vs baseline: -0.3% Memory: ✅ 37.734MB (SLO: <53.000MB 📉 -28.8%) vs baseline: +5.0% ✅ set-statusTime: ✅ 19.565ms (SLO: <22.000ms 📉 -11.1%) vs baseline: ~same Memory: ✅ 37.061MB (SLO: <53.000MB 📉 -30.1%) vs baseline: +4.4% ✅ startTime: ✅ 18.627ms (SLO: <20.500ms -9.1%) vs baseline: +4.4% Memory: ✅ 37.139MB (SLO: <53.000MB 📉 -29.9%) vs baseline: +4.8% ✅ start-finishTime: ✅ 53.178ms (SLO: <54.000ms 🟡 -1.5%) vs baseline: +0.2% Memory: ✅ 35.134MB (SLO: <36.000MB -2.4%) vs baseline: +4.6% ✅ start-finish-telemetryTime: ✅ 54.424ms (SLO: <56.000ms -2.8%) vs baseline: ~same Memory: ✅ 35.134MB (SLO: <36.000MB -2.4%) vs baseline: +4.6% ✅ start-finish-traceid128Time: ✅ 56.587ms (SLO: <57.000ms 🟡 -0.7%) vs baseline: -0.5% Memory: ✅ 35.154MB (SLO: <36.000MB -2.4%) vs baseline: +4.6% ✅ start-traceid128Time: ✅ 18.120ms (SLO: <22.500ms 📉 -19.5%) vs baseline: ~same Memory: ✅ 37.100MB (SLO: <53.000MB 📉 -30.0%) vs baseline: +4.5% ✅ update-nameTime: ✅ 18.207ms (SLO: <22.000ms 📉 -17.2%) vs baseline: ~same Memory: ✅ 37.104MB (SLO: <53.000MB 📉 -30.0%) vs baseline: +4.2% 🟡 telemetryaddmetric - 30/30✅ 1-count-metric-1-timesTime: ✅ 3.288µs (SLO: <20.000µs 📉 -83.6%) vs baseline: +8.4% Memory: ✅ 35.095MB (SLO: <36.000MB -2.5%) vs baseline: +4.9% ✅ 1-count-metrics-100-timesTime: ✅ 200.004µs (SLO: <220.000µs -9.1%) vs baseline: -1.1% Memory: ✅ 35.154MB (SLO: <36.000MB -2.4%) vs baseline: +4.6% ✅ 1-distribution-metric-1-timesTime: ✅ 3.377µs (SLO: <20.000µs 📉 -83.1%) vs baseline: -0.2% Memory: ✅ 35.095MB (SLO: <36.000MB -2.5%) vs baseline: +4.6% ✅ 1-distribution-metrics-100-timesTime: ✅ 213.238µs (SLO: <230.000µs -7.3%) vs baseline: -1.2% Memory: ✅ 35.114MB (SLO: <36.000MB -2.5%) vs baseline: +4.7% ✅ 1-gauge-metric-1-timesTime: ✅ 2.218µs (SLO: <20.000µs 📉 -88.9%) vs baseline: -0.5% Memory: ✅ 35.095MB (SLO: <36.000MB -2.5%) vs baseline: +4.9% ✅ 1-gauge-metrics-100-timesTime: ✅ 137.033µs (SLO: <150.000µs -8.6%) vs baseline: +0.3% Memory: ✅ 35.134MB (SLO: <36.000MB -2.4%) vs baseline: +4.7% ✅ 1-rate-metric-1-timesTime: ✅ 3.193µs (SLO: <20.000µs 📉 -84.0%) vs baseline: -0.3% Memory: ✅ 35.134MB (SLO: <36.000MB -2.4%) vs baseline: +4.6% ✅ 1-rate-metrics-100-timesTime: ✅ 214.796µs (SLO: <250.000µs 📉 -14.1%) vs baseline: +0.5% Memory: ✅ 35.036MB (SLO: <36.000MB -2.7%) vs baseline: +4.4% ✅ 100-count-metrics-100-timesTime: ✅ 20.280ms (SLO: <22.000ms -7.8%) vs baseline: +1.2% Memory: ✅ 35.154MB (SLO: <36.000MB -2.4%) vs baseline: +4.7% ✅ 100-distribution-metrics-100-timesTime: ✅ 2.260ms (SLO: <2.550ms 📉 -11.4%) vs baseline: -0.3% Memory: ✅ 35.468MB (SLO: <36.000MB 🟡 -1.5%) vs baseline: +5.7% ✅ 100-gauge-metrics-100-timesTime: ✅ 1.406ms (SLO: <1.550ms -9.3%) vs baseline: +0.5% Memory: ✅ 35.075MB (SLO: <36.000MB -2.6%) vs baseline: +4.4% ✅ 100-rate-metrics-100-timesTime: ✅ 2.199ms (SLO: <2.550ms 📉 -13.8%) vs baseline: ~same Memory: ✅ 35.134MB (SLO: <36.000MB -2.4%) vs baseline: +4.8% ✅ flush-1-metricTime: ✅ 4.638µs (SLO: <20.000µs 📉 -76.8%) vs baseline: -1.9% Memory: ✅ 35.488MB (SLO: <36.000MB 🟡 -1.4%) vs baseline: +4.8% ✅ flush-100-metricsTime: ✅ 174.950µs (SLO: <250.000µs 📉 -30.0%) vs baseline: -0.6% Memory: ✅ 35.448MB (SLO: <36.000MB 🟡 -1.5%) vs baseline: +4.4% ✅ flush-1000-metricsTime: ✅ 2.187ms (SLO: <2.500ms 📉 -12.5%) vs baseline: -0.7% Memory: ✅ 36.333MB (SLO: <36.500MB 🟡 -0.5%) vs baseline: +5.0% 🟡 tracer - 6/6✅ largeTime: ✅ 30.330ms (SLO: <32.950ms -8.0%) vs baseline: +0.3% Memory: ✅ 36.372MB (SLO: <36.500MB 🟡 -0.3%) vs baseline: +5.0% ✅ mediumTime: ✅ 2.982ms (SLO: <3.200ms -6.8%) vs baseline: -0.2% Memory: ✅ 35.134MB (SLO: <36.000MB -2.4%) vs baseline: +4.6% ✅ smallTime: ✅ 346.984µs (SLO: <370.000µs -6.2%) vs baseline: +2.2% Memory: ✅ 35.154MB (SLO: <36.000MB -2.4%) vs baseline: +4.7%
|
4df9391 to
cd06286
Compare
cd06286 to
d669a17
Compare
This comment has been minimized.
This comment has been minimized.
brettlangdon
left a comment
There was a problem hiding this comment.
biggest blocker is having the fuzz stage being the first one, everything else is just random nits/thoughts, nothing really blocking.
…d-trace-py into edouard/add-base-fuzzing-setup
brettlangdon
left a comment
There was a problem hiding this comment.
changes to .gitlab-ci.yml and the dependency structure of fuzz.yml lgtm
|
/merge -m squash |
|
View all feedbacks in Devflow UI.
This pull request is not mergeable according to GitHub. Common reasons include pending required checks, missing approvals, or merge conflicts — but it could also be blocked by other repository rules or settings.
The expected merge time in
PR can't be merged according to github policy |
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
This pull request was merged directly. |
Description
This is a V1 (especially the
fuzz_infra.py) because we are lacking some capabilities (see the further improvement section)This PR adds support for the building and sending fuzzing binaries to our internal fuzzing infra on a schedule from main.
It aims to be run on a scheduled pipeline every day on main.
The goal is to fuzz important part of dd-trace-py on a daily schedule. Other important library like libddwaf and libdatadog are already onboarded!
Reporting of bugs will be done via a slack message (and datadog error tracking!) with context enrichment and bug-fix proposal!
Findings
shift exponent 36 is too large for 32-bit type 'int'
(old, not sure if it's still valid as of January 14th)
This code
causes:
Stack overflow
After a few seconds of run, it looks like there's an infinite loop causing a stack overflow?
The crashing input can be seen as the following:
How do I get these findings?
Download the crash:
Download the binary that was running the fuzz test (or recompile it locally)
Run the fuzz test only on the crashing input
You can then use GDB to debug the issues.
Testing the PR
Testing locally
docker build -f docker/Dockerfile.fuzz -t ddtrace-py-stack-fuzz . docker run -it ddtrace-py-stack-fuzzInfra test procedure:
Further improvements
The current process, for C codebase is quite finicky in part because of the toolchain (CMake) flexibility.
There are improvements planned in our infrastructure that would allow for end user to use Dockerfile directly with the same discovery of harness, but it's not ready yet.
Risks
This is tests only, no customer facing risk. It may find interesting bugs to be fixed.
Additional Notes
We have more extensive doc in confluence here for our fuzzing infra.