Skip to content

Nikita tkachenko/test env 19d8832a9d074b4e378be52f31eb1627522341a6 #8495

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 2 commits into from
Closed

Nikita tkachenko/test env 19d8832a9d074b4e378be52f31eb1627522341a6 #8495

wants to merge 2 commits into from

Conversation

@nikita-tkachenko-datadog
Copy link
Contributor

@nikita-tkachenko-datadog nikita-tkachenko-datadog commented Mar 4, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

randomanderson and others added 2 commits February 28, 2025 09:44
@randomanderson
Migrate agent integration tests to gitlab (#8448)
93bbda1 
* Add checks, spotless, and test_published_artifact

* small fixes

* WIP

* syntax errors

* add gradle daemon logs everywhere

* fix c/p error

* fix system tests dir

* add some ls

* always save artifacts

* fix workspace path for system tests

* move more jobs to the tests stage

* split muzzle into 6 jobs

* remove system-tests debug output

* fix path and index for muzzle

* add signing args

* 8 parallism

* no needs for spotless

* java 7 for test published artifacts

* use the image for the test jvm

* upload ci_app results

* when always -> on_success

* Cleanup TraceAgentTests to always run

* try agent integration test

* add the actual target

* clean up branch

* set keys and use explicit agent value in test

* remove DD_* variables from environment before tests

* save/restore must be toplevel

* some debug logic for keys

* yaml

* set api keys in after_script

* fix DDApiIntegrationTest

* remove agent integration tests from circleci

* cleanup merge issues

* Revert "remove agent integration tests from circleci"

This reverts commit 90602a2.

* assign CI_AGENT_HOST to localhost

* don't skip tests when populating cache

* codenarc
@daniel-mohedano
Use separate TestEventHandlers per framework in CI Vis instrumentatio…
19d8832 
…ns (#8451)
@nikita-tkachenko-datadog nikita-tkachenko-datadog added the tag: do not merge Do not merge changes label Mar 4, 2025
@pr-commenter
Copy link

pr-commenter bot commented Mar 4, 2025

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/test-env-19d8832a9d074b4e378be52f31eb1627522341a6
git_commit_date 1741082792 1740758111
git_commit_sha 8f945b0 19d8832
release_version 1.47.0-SNAPSHOT~8f945b02a7 1.47.0-SNAPSHOT~19d8832a9d
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1741091740 1741091740
ci_job_id 831518151 831518151
ci_pipeline_id 57626491 57626491
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner--t2f38de-project-304-concurrent-7-5dhvbaz4 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner--t2f38de-project-304-concurrent-7-5dhvbaz4 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 21 performance improvements and 5 performance regressions! Performance is the same for 27 metrics, 10 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:insecure-bank:iast:Agent better
[-691.701ms; -493.437ms] or [-7.006%; -4.998%]		 9.280s 9.873s
scenario:startup:insecure-bank:iast:Agent.start better
[-92.711ms; -80.126ms] or [-7.368%; -6.368%]		 1.172s 1.258s
scenario:startup:insecure-bank:iast:Remote Config worse
[+90.882µs; +112.410µs] or [+17.735%; +21.936%]		 614.082µs 512.436µs
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Agent better
[-796.981ms; -583.821ms] or [-8.066%; -5.909%]		 9.190s 9.881s
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Agent.start better
[-110.577ms; -69.180ms] or [-8.719%; -5.455%]		 1.178s 1.268s
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Agent better
[-711.059ms; -619.398ms] or [-7.200%; -6.272%]		 9.211s 9.876s
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Agent.start better
[-91.899ms; -60.858ms] or [-7.352%; -4.869%]		 1.174s 1.250s
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Remote Config worse
[+91.409µs; +119.490µs] or [+18.078%; +23.631%]		 611.096µs 505.647µs
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Telemetry better
[-3.436ms; -3.230ms] or [-28.772%; -27.045%]		 8.610ms 11.943ms
scenario:startup:insecure-bank:tracing:Agent better
[-681.956ms; -580.134ms] or [-7.349%; -6.252%]		 8.649s 9.280s
scenario:startup:insecure-bank:tracing:Agent.start better
[-111.782ms; -88.975ms] or [-9.821%; -7.817%]		 1.038s 1.138s
scenario:startup:petclinic:appsec:Agent better
[-902.823ms; -791.896ms] or [-7.801%; -6.843%]		 10.726s 11.573s
scenario:startup:petclinic:appsec:Agent.start better
[-104.176ms; -83.728ms] or [-8.180%; -6.574%]		 1.180s 1.274s
scenario:startup:petclinic:appsec:Remote Config worse
[+17.140µs; +68.178µs] or [+2.717%; +10.809%]		 673.400µs 630.741µs
scenario:startup:petclinic:appsec:Telemetry better
[-4.230ms; -3.840ms] or [-34.304%; -31.136%]		 8.297ms 12.332ms
scenario:startup:petclinic:iast:Agent better
[-1091.235ms; -673.057ms] or [-9.120%; -5.625%]		 11.083s 11.965s
scenario:startup:petclinic:iast:Agent.start better
[-99.770ms; -82.097ms] or [-7.899%; -6.500%]		 1.172s 1.263s
scenario:startup:petclinic:iast:Remote Config worse
[+67.226µs; +116.903µs] or [+12.930%; +22.485%]		 611.987µs 519.922µs
scenario:startup:petclinic:profiling:Agent better
[-987.017ms; -832.638ms] or [-8.401%; -7.087%]		 10.839s 11.749s
scenario:startup:petclinic:profiling:Agent.start better
[-138.802ms; -91.312ms] or [-10.004%; -6.581%]		 1.272s 1.387s
scenario:startup:petclinic:profiling:ProfilingAgent better
[-29.519ms; -25.140ms] or [-23.821%; -20.288%]		 96.589ms 123.919ms
scenario:startup:petclinic:profiling:Telemetry better
[-4.028ms; -3.465ms] or [-31.418%; -27.024%]		 9.075ms 12.822ms
scenario:startup:petclinic:profiling:Profiling better
[-29.516ms; -25.137ms] or [-23.815%; -20.281%]		 96.613ms 123.940ms
scenario:startup:petclinic:tracing:Agent better
[-1.202s; -1.023s] or [-10.486%; -8.928%]		 10.347s 11.459s
scenario:startup:petclinic:tracing:Agent.start better
[-99.406ms; -89.659ms] or [-8.767%; -7.908%]		 1.039s 1.134s
scenario:startup:petclinic:tracing:Remote Config worse
[+91.328µs; +139.681µs] or [+15.651%; +23.938%]		 699.018µs 583.513µs
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.47.0-SNAPSHOT~19d8832a9d, baseline=1.47.0-SNAPSHOT~8f945b02a7

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.138 s) : 0, 1138216
Total [baseline] (9.28 s) : 0, 9279883
Agent [candidate] (1.038 s) : 0, 1037838
Total [candidate] (8.649 s) : 0, 8648838
section iast
Agent [baseline] (1.258 s) : 0, 1258352
Total [baseline] (9.873 s) : 0, 9872689
Agent [candidate] (1.172 s) : 0, 1171934
Total [candidate] (9.28 s) : 0, 9280120
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.268 s) : 0, 1268303
Total [baseline] (9.881 s) : 0, 9880769
Agent [candidate] (1.178 s) : 0, 1178424
Total [candidate] (9.19 s) : 0, 9190368
section iast_TELEMETRY_OFF
Agent [baseline] (1.25 s) : 0, 1249976
Total [baseline] (9.876 s) : 0, 9875844
Agent [candidate] (1.174 s) : 0, 1173598
Total [candidate] (9.211 s) : 0, 9210616
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.138 s -
Agent iast 1.258 s 120.136 ms (10.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.268 s 130.086 ms (11.4%)
Agent iast_TELEMETRY_OFF 1.25 s 111.76 ms (9.8%)
Total tracing 9.28 s -
Total iast 9.873 s 592.806 ms (6.4%)
Total iast_HARDCODED_SECRET_DISABLED 9.881 s 600.886 ms (6.5%)
Total iast_TELEMETRY_OFF 9.876 s 595.961 ms (6.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.038 s -
Agent iast 1.172 s 134.096 ms (12.9%)
Agent iast_HARDCODED_SECRET_DISABLED 1.178 s 140.586 ms (13.5%)
Agent iast_TELEMETRY_OFF 1.174 s 135.759 ms (13.1%)
Total tracing 8.649 s -
Total iast 9.28 s 631.282 ms (7.3%)
Total iast_HARDCODED_SECRET_DISABLED 9.19 s 541.53 ms (6.3%)
Total iast_TELEMETRY_OFF 9.211 s 561.778 ms (6.5%) 
gantt
    title insecure-bank - break down per module: candidate=1.47.0-SNAPSHOT~19d8832a9d, baseline=1.47.0-SNAPSHOT~8f945b02a7

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (720.665 ms) : 0, 720665
BytebuddyAgent [candidate] (716.354 ms) : 0, 716354
GlobalTracer [baseline] (239.521 ms) : 0, 239521
GlobalTracer [candidate] (239.005 ms) : 0, 239005
AppSec [baseline] (54.892 ms) : 0, 54892
AppSec [candidate] (55.538 ms) : 0, 55538
Remote Config [baseline] (593.546 µs) : 0, 594
Remote Config [candidate] (708.667 µs) : 0, 709
Telemetry [baseline] (12.327 ms) : 0, 12327
Telemetry [candidate] (11.418 ms) : 0, 11418
Debugger [baseline] (95.105 ms) : 0, 95105
section iast
BytebuddyAgent [baseline] (835.61 ms) : 0, 835610
BytebuddyAgent [candidate] (836.417 ms) : 0, 836417
GlobalTracer [baseline] (230.396 ms) : 0, 230396
GlobalTracer [candidate] (230.601 ms) : 0, 230601
IAST [baseline] (22.859 ms) : 0, 22859
IAST [candidate] (23.185 ms) : 0, 23185
AppSec [baseline] (56.954 ms) : 0, 56954
AppSec [candidate] (57.5 ms) : 0, 57500
Remote Config [baseline] (512.436 µs) : 0, 512
Remote Config [candidate] (614.082 µs) : 0, 614
Telemetry [baseline] (13.589 ms) : 0, 13589
Telemetry [candidate] (8.698 ms) : 0, 8698
Debugger [baseline] (83.499 ms) : 0, 83499
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (842.129 ms) : 0, 842129
BytebuddyAgent [candidate] (843.142 ms) : 0, 843142
GlobalTracer [baseline] (231.979 ms) : 0, 231979
GlobalTracer [candidate] (230.234 ms) : 0, 230234
IAST [baseline] (23.162 ms) : 0, 23162
IAST [candidate] (22.885 ms) : 0, 22885
AppSec [baseline] (57.547 ms) : 0, 57547
AppSec [candidate] (57.647 ms) : 0, 57647
Remote Config [baseline] (521.511 µs) : 0, 522
Remote Config [candidate] (615.098 µs) : 0, 615
Telemetry [baseline] (13.732 ms) : 0, 13732
Telemetry [candidate] (8.797 ms) : 0, 8797
Debugger [baseline] (84.173 ms) : 0, 84173
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (830.993 ms) : 0, 830993
BytebuddyAgent [candidate] (839.398 ms) : 0, 839398
GlobalTracer [baseline] (229.242 ms) : 0, 229242
GlobalTracer [candidate] (230.751 ms) : 0, 230751
IAST [baseline] (22.267 ms) : 0, 22267
IAST [candidate] (25.504 ms) : 0, 25504
AppSec [baseline] (56.733 ms) : 0, 56733
AppSec [candidate] (53.722 ms) : 0, 53722
Remote Config [baseline] (505.647 µs) : 0, 506
Remote Config [candidate] (611.096 µs) : 0, 611
Telemetry [baseline] (11.943 ms) : 0, 11943
Telemetry [candidate] (8.61 ms) : 0, 8610
Debugger [baseline] (83.338 ms) : 0, 83338
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.47.0-SNAPSHOT~19d8832a9d, baseline=1.47.0-SNAPSHOT~8f945b02a7

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.134 s) : 0, 1133850
Total [baseline] (11.459 s) : 0, 11459279
Agent [candidate] (1.039 s) : 0, 1039318
Total [candidate] (10.347 s) : 0, 10346927
section appsec
Agent [baseline] (1.274 s) : 0, 1273597
Total [baseline] (11.573 s) : 0, 11573066
Agent [candidate] (1.18 s) : 0, 1179645
Total [candidate] (10.726 s) : 0, 10725706
section iast
Agent [baseline] (1.263 s) : 0, 1263064
Total [baseline] (11.965 s) : 0, 11964960
Agent [candidate] (1.172 s) : 0, 1172130
Total [candidate] (11.083 s) : 0, 11082814
section profiling
Agent [baseline] (1.387 s) : 0, 1387484
Total [baseline] (11.749 s) : 0, 11748788
Agent [candidate] (1.272 s) : 0, 1272427
Total [candidate] (10.839 s) : 0, 10838961
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.134 s -
Agent appsec 1.274 s 139.747 ms (12.3%)
Agent iast 1.263 s 129.214 ms (11.4%)
Agent profiling 1.387 s 253.634 ms (22.4%)
Total tracing 11.459 s -
Total appsec 11.573 s 113.787 ms (1.0%)
Total iast 11.965 s 505.681 ms (4.4%)
Total profiling 11.749 s 289.509 ms (2.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.039 s -
Agent appsec 1.18 s 140.328 ms (13.5%)
Agent iast 1.172 s 132.812 ms (12.8%)
Agent profiling 1.272 s 233.11 ms (22.4%)
Total tracing 10.347 s -
Total appsec 10.726 s 378.779 ms (3.7%)
Total iast 11.083 s 735.887 ms (7.1%)
Total profiling 10.839 s 492.034 ms (4.8%) 
gantt
    title petclinic - break down per module: candidate=1.47.0-SNAPSHOT~19d8832a9d, baseline=1.47.0-SNAPSHOT~8f945b02a7

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (715.69 ms) : 0, 715690
BytebuddyAgent [candidate] (717.27 ms) : 0, 717270
GlobalTracer [baseline] (239.429 ms) : 0, 239429
GlobalTracer [candidate] (239.223 ms) : 0, 239223
AppSec [baseline] (55.518 ms) : 0, 55518
AppSec [candidate] (55.721 ms) : 0, 55721
Remote Config [baseline] (583.513 µs) : 0, 584
Remote Config [candidate] (699.018 µs) : 0, 699
Telemetry [baseline] (12.34 ms) : 0, 12340
Telemetry [candidate] (11.489 ms) : 0, 11489
Debugger [baseline] (95.284 ms) : 0, 95284
section appsec
BytebuddyAgent [baseline] (736.116 ms) : 0, 736116
BytebuddyAgent [candidate] (732.736 ms) : 0, 732736
GlobalTracer [baseline] (236.921 ms) : 0, 236921
GlobalTracer [candidate] (235.291 ms) : 0, 235291
AppSec [baseline] (177.493 ms) : 0, 177493
AppSec [candidate] (176.972 ms) : 0, 176972
Remote Config [baseline] (630.741 µs) : 0, 631
Remote Config [candidate] (673.4 µs) : 0, 673
Telemetry [baseline] (12.332 ms) : 0, 12332
Telemetry [candidate] (8.297 ms) : 0, 8297
Debugger [baseline] (84.469 ms) : 0, 84469
IAST [baseline] (21.275 ms) : 0, 21275
IAST [candidate] (21.561 ms) : 0, 21561
section iast
BytebuddyAgent [baseline] (836.073 ms) : 0, 836073
BytebuddyAgent [candidate] (836.354 ms) : 0, 836354
GlobalTracer [baseline] (230.524 ms) : 0, 230524
GlobalTracer [candidate] (230.33 ms) : 0, 230330
AppSec [baseline] (56.699 ms) : 0, 56699
AppSec [candidate] (57.746 ms) : 0, 57746
Remote Config [baseline] (519.922 µs) : 0, 520
Remote Config [candidate] (611.987 µs) : 0, 612
Telemetry [baseline] (16.707 ms) : 0, 16707
Telemetry [candidate] (8.913 ms) : 0, 8913
Debugger [baseline] (83.784 ms) : 0, 83784
IAST [baseline] (23.729 ms) : 0, 23729
IAST [candidate] (23.171 ms) : 0, 23171
section profiling
ProfilingAgent [baseline] (123.919 ms) : 0, 123919
ProfilingAgent [candidate] (96.589 ms) : 0, 96589
BytebuddyAgent [baseline] (712.933 ms) : 0, 712933
BytebuddyAgent [candidate] (719.052 ms) : 0, 719052
GlobalTracer [baseline] (352.333 ms) : 0, 352333
GlobalTracer [candidate] (351.428 ms) : 0, 351428
AppSec [baseline] (55.495 ms) : 0, 55495
AppSec [candidate] (54.644 ms) : 0, 54644
Remote Config [baseline] (624.06 µs) : 0, 624
Remote Config [candidate] (671.738 µs) : 0, 672
Telemetry [baseline] (12.822 ms) : 0, 12822
Telemetry [candidate] (9.075 ms) : 0, 9075
Profiling [baseline] (123.94 ms) : 0, 123940
Profiling [candidate] (96.613 ms) : 0, 96613
Debugger [baseline] (88.737 ms) : 0, 88737
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-03-04T12:04:57 2025-03-04T12:12:48
git_branch master nikita-tkachenko/test-env-19d8832a9d074b4e378be52f31eb1627522341a6
git_commit_date 1741082792 1740758111
git_commit_sha 8f945b0 19d8832
release_version 1.47.0-SNAPSHOT~8f945b02a7 1.47.0-SNAPSHOT~19d8832a9d
start_time 2025-03-04T12:04:43 2025-03-04T12:12:34
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1741090765 1741090765
ci_job_id 831518152 831518152
ci_pipeline_id 57626491 57626491
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-8ksbllmc-project-304-concurrent-5-eejg7wux 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-8ksbllmc-project-304-concurrent-5-eejg7wux 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 13 metrics, 17 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~19d8832a9d, baseline=1.47.0-SNAPSHOT~8f945b02a7
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.338 ms) : 1318, 1358
.   : milestone, 1338,
appsec (1.729 ms) : 1705, 1752
.   : milestone, 1729,
appsec_no_iast (1.725 ms) : 1701, 1749
.   : milestone, 1725,
code_origins (1.686 ms) : 1653, 1718
.   : milestone, 1686,
iast (1.504 ms) : 1479, 1529
.   : milestone, 1504,
profiling (1.507 ms) : 1482, 1531
.   : milestone, 1507,
tracing (1.482 ms) : 1456, 1509
.   : milestone, 1482,
section candidate
no_agent (1.343 ms) : 1323, 1363
.   : milestone, 1343,
appsec (1.718 ms) : 1693, 1742
.   : milestone, 1718,
appsec_no_iast (1.727 ms) : 1702, 1752
.   : milestone, 1727,
code_origins (1.684 ms) : 1650, 1718
.   : milestone, 1684,
iast (1.507 ms) : 1483, 1531
.   : milestone, 1507,
profiling (1.56 ms) : 1534, 1586
.   : milestone, 1560,
tracing (1.472 ms) : 1447, 1496
.   : milestone, 1472,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.338 ms [1.318 ms, 1.358 ms] -
appsec 1.729 ms [1.705 ms, 1.752 ms] 391.063 µs (29.2%)
appsec_no_iast 1.725 ms [1.701 ms, 1.749 ms] 387.643 µs (29.0%)
code_origins 1.686 ms [1.653 ms, 1.718 ms] 348.096 µs (26.0%)
iast 1.504 ms [1.479 ms, 1.529 ms] 166.344 µs (12.4%)
profiling 1.507 ms [1.482 ms, 1.531 ms] 169.04 µs (12.6%)
tracing 1.482 ms [1.456 ms, 1.509 ms] 144.649 µs (10.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.343 ms [1.323 ms, 1.363 ms] -
appsec 1.718 ms [1.693 ms, 1.742 ms] 374.591 µs (27.9%)
appsec_no_iast 1.727 ms [1.702 ms, 1.752 ms] 384.146 µs (28.6%)
code_origins 1.684 ms [1.65 ms, 1.718 ms] 340.884 µs (25.4%)
iast 1.507 ms [1.483 ms, 1.531 ms] 163.623 µs (12.2%)
profiling 1.56 ms [1.534 ms, 1.586 ms] 217.264 µs (16.2%)
tracing 1.472 ms [1.447 ms, 1.496 ms] 128.581 µs (9.6%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~19d8832a9d, baseline=1.47.0-SNAPSHOT~8f945b02a7
    dateFormat X
    axisFormat %s
section baseline
no_agent (375.865 µs) : 356, 396
.   : milestone, 376,
iast (507.78 µs) : 486, 530
.   : milestone, 508,
iast_FULL (719.903 µs) : 698, 742
.   : milestone, 720,
iast_GLOBAL (555.673 µs) : 533, 578
.   : milestone, 556,
iast_HARDCODED_SECRET_DISABLED (507.917 µs) : 486, 530
.   : milestone, 508,
iast_INACTIVE (458.795 µs) : 437, 480
.   : milestone, 459,
iast_TELEMETRY_OFF (501.207 µs) : 479, 524
.   : milestone, 501,
tracing (459.403 µs) : 437, 481
.   : milestone, 459,
section candidate
no_agent (371.728 µs) : 352, 391
.   : milestone, 372,
iast (510.085 µs) : 488, 532
.   : milestone, 510,
iast_FULL (718.68 µs) : 697, 741
.   : milestone, 719,
iast_GLOBAL (558.483 µs) : 536, 581
.   : milestone, 558,
iast_HARDCODED_SECRET_DISABLED (511.51 µs) : 489, 534
.   : milestone, 512,
iast_INACTIVE (455.336 µs) : 434, 476
.   : milestone, 455,
iast_TELEMETRY_OFF (494.289 µs) : 472, 517
.   : milestone, 494,
tracing (451.395 µs) : 430, 473
.   : milestone, 451,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 375.865 µs [355.712 µs, 396.019 µs] -
iast 507.78 µs [485.842 µs, 529.717 µs] 131.914 µs (35.1%)
iast_FULL 719.903 µs [697.754 µs, 742.053 µs] 344.038 µs (91.5%)
iast_GLOBAL 555.673 µs [533.47 µs, 577.876 µs] 179.808 µs (47.8%)
iast_HARDCODED_SECRET_DISABLED 507.917 µs [486.172 µs, 529.662 µs] 132.052 µs (35.1%)
iast_INACTIVE 458.795 µs [437.24 µs, 480.35 µs] 82.93 µs (22.1%)
iast_TELEMETRY_OFF 501.207 µs [478.634 µs, 523.781 µs] 125.342 µs (33.3%)
tracing 459.403 µs [437.361 µs, 481.444 µs] 83.537 µs (22.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 371.728 µs [352.154 µs, 391.301 µs] -
iast 510.085 µs [487.951 µs, 532.219 µs] 138.357 µs (37.2%)
iast_FULL 718.68 µs [696.705 µs, 740.656 µs] 346.953 µs (93.3%)
iast_GLOBAL 558.483 µs [536.024 µs, 580.942 µs] 186.755 µs (50.2%)
iast_HARDCODED_SECRET_DISABLED 511.51 µs [488.884 µs, 534.136 µs] 139.782 µs (37.6%)
iast_INACTIVE 455.336 µs [434.493 µs, 476.18 µs] 83.609 µs (22.5%)
iast_TELEMETRY_OFF 494.289 µs [471.669 µs, 516.91 µs] 122.561 µs (33.0%)
tracing 451.395 µs [430.212 µs, 472.579 µs] 79.667 µs (21.4%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/test-env-19d8832a9d074b4e378be52f31eb1627522341a6
git_commit_date 1741082792 1740758111
git_commit_sha 8f945b0 19d8832
release_version 1.47.0-SNAPSHOT~8f945b02a7 1.47.0-SNAPSHOT~19d8832a9d
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1741091279 1741091279
ci_job_id 831518153 831518153
ci_pipeline_id 57626491 57626491
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner--t2f38de-project-304-concurrent-8-kbmn207h 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner--t2f38de-project-304-concurrent-8-kbmn207h 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~19d8832a9d, baseline=1.47.0-SNAPSHOT~8f945b02a7
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.008 s) : 15008000, 15008000
.   : milestone, 15008000,
appsec (15.083 s) : 15083000, 15083000
.   : milestone, 15083000,
iast (19.208 s) : 19208000, 19208000
.   : milestone, 19208000,
iast_GLOBAL (18.172 s) : 18172000, 18172000
.   : milestone, 18172000,
profiling (15.041 s) : 15041000, 15041000
.   : milestone, 15041000,
tracing (15.015 s) : 15015000, 15015000
.   : milestone, 15015000,
section candidate
no_agent (15.426 s) : 15426000, 15426000
.   : milestone, 15426000,
appsec (15.002 s) : 15002000, 15002000
.   : milestone, 15002000,
iast (18.847 s) : 18847000, 18847000
.   : milestone, 18847000,
iast_GLOBAL (18.113 s) : 18113000, 18113000
.   : milestone, 18113000,
profiling (15.105 s) : 15105000, 15105000
.   : milestone, 15105000,
tracing (14.663 s) : 14663000, 14663000
.   : milestone, 14663000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.008 s [15.008 s, 15.008 s] -
appsec 15.083 s [15.083 s, 15.083 s] 75.0 ms (0.5%)
iast 19.208 s [19.208 s, 19.208 s] 4.2 s (28.0%)
iast_GLOBAL 18.172 s [18.172 s, 18.172 s] 3.164 s (21.1%)
profiling 15.041 s [15.041 s, 15.041 s] 33.0 ms (0.2%)
tracing 15.015 s [15.015 s, 15.015 s] 7.0 ms (0.0%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.426 s [15.426 s, 15.426 s] -
appsec 15.002 s [15.002 s, 15.002 s] -424.0 ms (-2.7%)
iast 18.847 s [18.847 s, 18.847 s] 3.421 s (22.2%)
iast_GLOBAL 18.113 s [18.113 s, 18.113 s] 2.687 s (17.4%)
profiling 15.105 s [15.105 s, 15.105 s] -321.0 ms (-2.1%)
tracing 14.663 s [14.663 s, 14.663 s] -763.0 ms (-4.9%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~19d8832a9d, baseline=1.47.0-SNAPSHOT~8f945b02a7
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.469 ms) : 1458, 1481
.   : milestone, 1469,
appsec (2.319 ms) : 2275, 2362
.   : milestone, 2319,
iast (2.112 ms) : 2056, 2167
.   : milestone, 2112,
iast_GLOBAL (2.148 ms) : 2092, 2204
.   : milestone, 2148,
profiling (1.961 ms) : 1917, 2004
.   : milestone, 1961,
tracing (1.946 ms) : 1904, 1989
.   : milestone, 1946,
section candidate
no_agent (1.469 ms) : 1458, 1481
.   : milestone, 1469,
appsec (2.325 ms) : 2282, 2369
.   : milestone, 2325,
iast (2.113 ms) : 2058, 2169
.   : milestone, 2113,
iast_GLOBAL (2.148 ms) : 2093, 2204
.   : milestone, 2148,
profiling (1.981 ms) : 1936, 2026
.   : milestone, 1981,
tracing (1.947 ms) : 1904, 1989
.   : milestone, 1947,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.469 ms [1.458 ms, 1.481 ms] -
appsec 2.319 ms [2.275 ms, 2.362 ms] 849.267 µs (57.8%)
iast 2.112 ms [2.056 ms, 2.167 ms] 642.355 µs (43.7%)
iast_GLOBAL 2.148 ms [2.092 ms, 2.204 ms] 678.579 µs (46.2%)
profiling 1.961 ms [1.917 ms, 2.004 ms] 491.333 µs (33.4%)
tracing 1.946 ms [1.904 ms, 1.989 ms] 476.915 µs (32.5%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.469 ms [1.458 ms, 1.481 ms] -
appsec 2.325 ms [2.282 ms, 2.369 ms] 856.145 µs (58.3%)
iast 2.113 ms [2.058 ms, 2.169 ms] 644.125 µs (43.8%)
iast_GLOBAL 2.148 ms [2.093 ms, 2.204 ms] 679.186 µs (46.2%)
profiling 1.981 ms [1.936 ms, 2.026 ms] 511.594 µs (34.8%)
tracing 1.947 ms [1.904 ms, 1.989 ms] 477.256 µs (32.5%)

@nikita-tkachenko-datadog nikita-tkachenko-datadog deleted the nikita-tkachenko/test-env-19d8832a9d074b4e378be52f31eb1627522341a6 branch March 4, 2025 14:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

tag: do not merge Do not merge changes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@nikita-tkachenko-datadog @randomanderson @daniel-mohedano