Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add new assert to help troubleshoot flaky test #7729

Merged
merged 2 commits into from
Oct 7, 2024
Merged

Add new assert to help troubleshoot flaky test #7729

merged 2 commits into from
Oct 7, 2024

Conversation

jpbempel
Copy link
Member

@jpbempel jpbempel commented Oct 4, 2024

What Does This Do

make run flaky tests for debugger module

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@jpbempel
Add new assert to help troubleshoot flaky test
0d1c60f 
make run flaky tests for debuger module
@jpbempel jpbempel added comp: testing Testing tag: no release notes Changes to exclude from release notes comp: debugger Dynamic Instrumentation labels Oct 4, 2024
@jpbempel jpbempel requested review from a team as code owners October 4, 2024 09:31
@jpbempel jpbempel requested review from cimi and dougqh and removed request for a team October 4, 2024 09:31
@pr-commenter
Copy link

pr-commenter bot commented Oct 4, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/flaky-smoke-tests
git_commit_date 1728032175 1728040311
git_commit_sha 9db260f 6d5c892
release_version 1.41.0-SNAPSHOT~9db260fd16 1.41.0-SNAPSHOT~6d5c892d40
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1728042646 1728042646
ci_job_id 661533495 661533495
ci_pipeline_id 45811547 45811547
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 53 metrics, 10 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.41.0-SNAPSHOT~6d5c892d40, baseline=1.41.0-SNAPSHOT~9db260fd16

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.071 s) : 0, 1070982
Total [baseline] (8.561 s) : 0, 8560576
Agent [candidate] (1.065 s) : 0, 1065232
Total [candidate] (8.529 s) : 0, 8528864
section iast
Agent [baseline] (1.201 s) : 0, 1201289
Total [baseline] (9.129 s) : 0, 9128711
Agent [candidate] (1.195 s) : 0, 1194740
Total [candidate] (9.053 s) : 0, 9053060
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.202 s) : 0, 1202445
Total [baseline] (9.097 s) : 0, 9096807
Agent [candidate] (1.194 s) : 0, 1193656
Total [candidate] (9.05 s) : 0, 9050485
section iast_TELEMETRY_OFF
Agent [baseline] (1.197 s) : 0, 1196606
Total [baseline] (9.072 s) : 0, 9071513
Agent [candidate] (1.19 s) : 0, 1189578
Total [candidate] (9.072 s) : 0, 9072223
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.071 s -
Agent iast 1.201 s 130.307 ms (12.2%)
Agent iast_HARDCODED_SECRET_DISABLED 1.202 s 131.463 ms (12.3%)
Agent iast_TELEMETRY_OFF 1.197 s 125.624 ms (11.7%)
Total tracing 8.561 s -
Total iast 9.129 s 568.134 ms (6.6%)
Total iast_HARDCODED_SECRET_DISABLED 9.097 s 536.231 ms (6.3%)
Total iast_TELEMETRY_OFF 9.072 s 510.937 ms (6.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.065 s -
Agent iast 1.195 s 129.508 ms (12.2%)
Agent iast_HARDCODED_SECRET_DISABLED 1.194 s 128.424 ms (12.1%)
Agent iast_TELEMETRY_OFF 1.19 s 124.346 ms (11.7%)
Total tracing 8.529 s -
Total iast 9.053 s 524.196 ms (6.1%)
Total iast_HARDCODED_SECRET_DISABLED 9.05 s 521.621 ms (6.1%)
Total iast_TELEMETRY_OFF 9.072 s 543.359 ms (6.4%) 
gantt
    title insecure-bank - break down per module: candidate=1.41.0-SNAPSHOT~6d5c892d40, baseline=1.41.0-SNAPSHOT~9db260fd16

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (684.312 ms) : 0, 684312
BytebuddyAgent [candidate] (680.807 ms) : 0, 680807
GlobalTracer [baseline] (311.092 ms) : 0, 311092
GlobalTracer [candidate] (309.34 ms) : 0, 309340
AppSec [baseline] (53.534 ms) : 0, 53534
AppSec [candidate] (53.244 ms) : 0, 53244
Remote Config [baseline] (661.481 µs) : 0, 661
Remote Config [candidate] (660.951 µs) : 0, 661
Telemetry [baseline] (7.677 ms) : 0, 7677
Telemetry [candidate] (7.573 ms) : 0, 7573
section iast
BytebuddyAgent [baseline] (800.314 ms) : 0, 800314
BytebuddyAgent [candidate] (796.198 ms) : 0, 796198
GlobalTracer [baseline] (300.578 ms) : 0, 300578
GlobalTracer [candidate] (299.533 ms) : 0, 299533
AppSec [baseline] (54.319 ms) : 0, 54319
AppSec [candidate] (55.753 ms) : 0, 55753
Remote Config [baseline] (615.695 µs) : 0, 616
Remote Config [candidate] (615.222 µs) : 0, 615
Telemetry [baseline] (7.101 ms) : 0, 7101
Telemetry [candidate] (7.084 ms) : 0, 7084
IAST [baseline] (24.637 ms) : 0, 24637
IAST [candidate] (21.9 ms) : 0, 21900
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (801.802 ms) : 0, 801802
BytebuddyAgent [candidate] (795.425 ms) : 0, 795425
GlobalTracer [baseline] (300.702 ms) : 0, 300702
GlobalTracer [candidate] (298.874 ms) : 0, 298874
AppSec [baseline] (56.234 ms) : 0, 56234
AppSec [candidate] (53.197 ms) : 0, 53197
Remote Config [baseline] (622.894 µs) : 0, 623
Remote Config [candidate] (608.747 µs) : 0, 609
Telemetry [baseline] (7.155 ms) : 0, 7155
Telemetry [candidate] (7.008 ms) : 0, 7008
IAST [baseline] (22.144 ms) : 0, 22144
IAST [candidate] (24.851 ms) : 0, 24851
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (796.86 ms) : 0, 796860
BytebuddyAgent [candidate] (791.548 ms) : 0, 791548
GlobalTracer [baseline] (300.207 ms) : 0, 300207
GlobalTracer [candidate] (298.474 ms) : 0, 298474
AppSec [baseline] (55.787 ms) : 0, 55787
AppSec [candidate] (54.14 ms) : 0, 54140
Remote Config [baseline] (593.687 µs) : 0, 594
Remote Config [candidate] (608.189 µs) : 0, 608
Telemetry [baseline] (6.908 ms) : 0, 6908
Telemetry [candidate] (6.964 ms) : 0, 6964
IAST [baseline] (22.498 ms) : 0, 22498
IAST [candidate] (24.145 ms) : 0, 24145
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.41.0-SNAPSHOT~6d5c892d40, baseline=1.41.0-SNAPSHOT~9db260fd16

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.07 s) : 0, 1070027
Total [baseline] (10.387 s) : 0, 10386947
Agent [candidate] (1.068 s) : 0, 1068144
Total [candidate] (10.371 s) : 0, 10371047
section appsec
Agent [baseline] (1.204 s) : 0, 1203811
Total [baseline] (10.538 s) : 0, 10537898
Agent [candidate] (1.2 s) : 0, 1200474
Total [candidate] (10.628 s) : 0, 10627578
section iast
Agent [baseline] (1.201 s) : 0, 1200744
Total [baseline] (10.855 s) : 0, 10855476
Agent [candidate] (1.194 s) : 0, 1193680
Total [candidate] (10.789 s) : 0, 10789405
section profiling
Agent [baseline] (1.286 s) : 0, 1285692
Total [baseline] (10.714 s) : 0, 10714485
Agent [candidate] (1.273 s) : 0, 1272767
Total [candidate] (10.637 s) : 0, 10637345
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.07 s -
Agent appsec 1.204 s 133.784 ms (12.5%)
Agent iast 1.201 s 130.717 ms (12.2%)
Agent profiling 1.286 s 215.665 ms (20.2%)
Total tracing 10.387 s -
Total appsec 10.538 s 150.952 ms (1.5%)
Total iast 10.855 s 468.529 ms (4.5%)
Total profiling 10.714 s 327.538 ms (3.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.068 s -
Agent appsec 1.2 s 132.33 ms (12.4%)
Agent iast 1.194 s 125.537 ms (11.8%)
Agent profiling 1.273 s 204.623 ms (19.2%)
Total tracing 10.371 s -
Total appsec 10.628 s 256.531 ms (2.5%)
Total iast 10.789 s 418.359 ms (4.0%)
Total profiling 10.637 s 266.299 ms (2.6%) 
gantt
    title petclinic - break down per module: candidate=1.41.0-SNAPSHOT~6d5c892d40, baseline=1.41.0-SNAPSHOT~9db260fd16

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (681.91 ms) : 0, 681910
BytebuddyAgent [candidate] (682.209 ms) : 0, 682209
GlobalTracer [baseline] (311.945 ms) : 0, 311945
GlobalTracer [candidate] (310.241 ms) : 0, 310241
AppSec [baseline] (54.176 ms) : 0, 54176
AppSec [candidate] (53.669 ms) : 0, 53669
Remote Config [baseline] (661.973 µs) : 0, 662
Remote Config [candidate] (665.056 µs) : 0, 665
Telemetry [baseline] (7.682 ms) : 0, 7682
Telemetry [candidate] (7.66 ms) : 0, 7660
section appsec
BytebuddyAgent [baseline] (699.776 ms) : 0, 699776
BytebuddyAgent [candidate] (697.556 ms) : 0, 697556
GlobalTracer [baseline] (307.826 ms) : 0, 307826
GlobalTracer [candidate] (306.757 ms) : 0, 306757
AppSec [baseline] (164.169 ms) : 0, 164169
AppSec [candidate] (162.665 ms) : 0, 162665
Remote Config [baseline] (639.125 µs) : 0, 639
Remote Config [candidate] (630.749 µs) : 0, 631
Telemetry [baseline] (7.445 ms) : 0, 7445
Telemetry [candidate] (8.527 ms) : 0, 8527
IAST [baseline] (20.638 ms) : 0, 20638
IAST [candidate] (21.431 ms) : 0, 21431
section iast
BytebuddyAgent [baseline] (800.759 ms) : 0, 800759
BytebuddyAgent [candidate] (795.515 ms) : 0, 795515
GlobalTracer [baseline] (300.221 ms) : 0, 300221
GlobalTracer [candidate] (299.12 ms) : 0, 299120
AppSec [baseline] (54.133 ms) : 0, 54133
AppSec [candidate] (53.416 ms) : 0, 53416
Remote Config [baseline] (595.453 µs) : 0, 595
Remote Config [candidate] (596.176 µs) : 0, 596
Telemetry [baseline] (7.069 ms) : 0, 7069
Telemetry [candidate] (7.02 ms) : 0, 7020
IAST [baseline] (24.267 ms) : 0, 24267
IAST [candidate] (24.391 ms) : 0, 24391
section profiling
BytebuddyAgent [baseline] (685.212 ms) : 0, 685212
BytebuddyAgent [candidate] (678.725 ms) : 0, 678725
GlobalTracer [baseline] (398.876 ms) : 0, 398876
GlobalTracer [candidate] (394.541 ms) : 0, 394541
AppSec [baseline] (55.449 ms) : 0, 55449
AppSec [candidate] (55.015 ms) : 0, 55015
Remote Config [baseline] (670.931 µs) : 0, 671
Remote Config [candidate] (658.039 µs) : 0, 658
Telemetry [baseline] (7.572 ms) : 0, 7572
Telemetry [candidate] (7.51 ms) : 0, 7510
ProfilingAgent [baseline] (98.741 ms) : 0, 98741
ProfilingAgent [candidate] (97.551 ms) : 0, 97551
Profiling [baseline] (98.765 ms) : 0, 98765
Profiling [candidate] (97.574 ms) : 0, 97574
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-10-04T11:21:20 2024-10-04T11:28:13
git_branch master jpbempel/flaky-smoke-tests
git_commit_date 1728032175 1728040311
git_commit_sha 9db260f 6d5c892
release_version 1.41.0-SNAPSHOT~9db260fd16 1.41.0-SNAPSHOT~6d5c892d40
start_time 2024-10-04T11:21:07 2024-10-04T11:27:59
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1728041638 1728041638
ci_job_id 661533497 661533497
ci_pipeline_id 45811547 45811547
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.41.0-SNAPSHOT~6d5c892d40, baseline=1.41.0-SNAPSHOT~9db260fd16
    dateFormat X
    axisFormat %s
section baseline
no_agent (377.014 µs) : 357, 397
.   : milestone, 377,
iast (487.767 µs) : 467, 509
.   : milestone, 488,
iast_FULL (558.242 µs) : 537, 580
.   : milestone, 558,
iast_GLOBAL (510.363 µs) : 488, 533
.   : milestone, 510,
iast_HARDCODED_SECRET_DISABLED (487.22 µs) : 466, 508
.   : milestone, 487,
iast_INACTIVE (447.559 µs) : 427, 468
.   : milestone, 448,
iast_TELEMETRY_OFF (475.448 µs) : 454, 497
.   : milestone, 475,
tracing (448.725 µs) : 428, 470
.   : milestone, 449,
section candidate
no_agent (374.999 µs) : 355, 395
.   : milestone, 375,
iast (498.116 µs) : 476, 520
.   : milestone, 498,
iast_FULL (555.43 µs) : 534, 577
.   : milestone, 555,
iast_GLOBAL (508.351 µs) : 487, 530
.   : milestone, 508,
iast_HARDCODED_SECRET_DISABLED (486.735 µs) : 465, 508
.   : milestone, 487,
iast_INACTIVE (450.987 µs) : 430, 472
.   : milestone, 451,
iast_TELEMETRY_OFF (483.623 µs) : 462, 505
.   : milestone, 484,
tracing (452.909 µs) : 432, 474
.   : milestone, 453,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 377.014 µs [357.136 µs, 396.893 µs] -
iast 487.767 µs [466.617 µs, 508.917 µs] 110.752 µs (29.4%)
iast_FULL 558.242 µs [536.843 µs, 579.642 µs] 181.228 µs (48.1%)
iast_GLOBAL 510.363 µs [488.061 µs, 532.664 µs] 133.348 µs (35.4%)
iast_HARDCODED_SECRET_DISABLED 487.22 µs [466.074 µs, 508.367 µs] 110.206 µs (29.2%)
iast_INACTIVE 447.559 µs [426.992 µs, 468.126 µs] 70.545 µs (18.7%)
iast_TELEMETRY_OFF 475.448 µs [454.057 µs, 496.839 µs] 98.433 µs (26.1%)
tracing 448.725 µs [427.581 µs, 469.868 µs] 71.71 µs (19.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 374.999 µs [354.943 µs, 395.055 µs] -
iast 498.116 µs [476.234 µs, 519.999 µs] 123.117 µs (32.8%)
iast_FULL 555.43 µs [534.336 µs, 576.524 µs] 180.431 µs (48.1%)
iast_GLOBAL 508.351 µs [487.199 µs, 529.502 µs] 133.352 µs (35.6%)
iast_HARDCODED_SECRET_DISABLED 486.735 µs [465.485 µs, 507.984 µs] 111.736 µs (29.8%)
iast_INACTIVE 450.987 µs [429.803 µs, 472.172 µs] 75.988 µs (20.3%)
iast_TELEMETRY_OFF 483.623 µs [461.88 µs, 505.366 µs] 108.624 µs (29.0%)
tracing 452.909 µs [431.589 µs, 474.23 µs] 77.91 µs (20.8%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.41.0-SNAPSHOT~6d5c892d40, baseline=1.41.0-SNAPSHOT~9db260fd16
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.349 ms) : 1329, 1370
.   : milestone, 1349,
appsec (1.719 ms) : 1695, 1742
.   : milestone, 1719,
appsec_no_iast (1.717 ms) : 1692, 1741
.   : milestone, 1717,
iast (1.475 ms) : 1452, 1498
.   : milestone, 1475,
profiling (1.525 ms) : 1499, 1551
.   : milestone, 1525,
tracing (1.452 ms) : 1428, 1476
.   : milestone, 1452,
section candidate
no_agent (1.355 ms) : 1336, 1374
.   : milestone, 1355,
appsec (1.702 ms) : 1677, 1726
.   : milestone, 1702,
appsec_no_iast (1.715 ms) : 1690, 1740
.   : milestone, 1715,
iast (1.465 ms) : 1443, 1487
.   : milestone, 1465,
profiling (1.472 ms) : 1447, 1496
.   : milestone, 1472,
tracing (1.475 ms) : 1451, 1500
.   : milestone, 1475,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.349 ms [1.329 ms, 1.37 ms] -
appsec 1.719 ms [1.695 ms, 1.742 ms] 369.545 µs (27.4%)
appsec_no_iast 1.717 ms [1.692 ms, 1.741 ms] 367.439 µs (27.2%)
iast 1.475 ms [1.452 ms, 1.498 ms] 126.026 µs (9.3%)
profiling 1.525 ms [1.499 ms, 1.551 ms] 176.043 µs (13.0%)
tracing 1.452 ms [1.428 ms, 1.476 ms] 102.845 µs (7.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.355 ms [1.336 ms, 1.374 ms] -
appsec 1.702 ms [1.677 ms, 1.726 ms] 346.462 µs (25.6%)
appsec_no_iast 1.715 ms [1.69 ms, 1.74 ms] 359.919 µs (26.6%)
iast 1.465 ms [1.443 ms, 1.487 ms] 109.718 µs (8.1%)
profiling 1.472 ms [1.447 ms, 1.496 ms] 116.403 µs (8.6%)
tracing 1.475 ms [1.451 ms, 1.5 ms] 120.315 µs (8.9%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/flaky-smoke-tests
git_commit_date 1728032175 1728040311
git_commit_sha 9db260f 6d5c892
release_version 1.41.0-SNAPSHOT~9db260fd16 1.41.0-SNAPSHOT~6d5c892d40
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1728042277 1728042277
ci_job_id 661533499 661533499
ci_pipeline_id 45811547 45811547
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.41.0-SNAPSHOT~6d5c892d40, baseline=1.41.0-SNAPSHOT~9db260fd16
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.615 s) : 15615000, 15615000
.   : milestone, 15615000,
appsec (14.994 s) : 14994000, 14994000
.   : milestone, 14994000,
iast (18.882 s) : 18882000, 18882000
.   : milestone, 18882000,
iast_GLOBAL (18.156 s) : 18156000, 18156000
.   : milestone, 18156000,
profiling (15.103 s) : 15103000, 15103000
.   : milestone, 15103000,
tracing (15.126 s) : 15126000, 15126000
.   : milestone, 15126000,
section candidate
no_agent (14.961 s) : 14961000, 14961000
.   : milestone, 14961000,
appsec (15.002 s) : 15002000, 15002000
.   : milestone, 15002000,
iast (18.772 s) : 18772000, 18772000
.   : milestone, 18772000,
iast_GLOBAL (17.873 s) : 17873000, 17873000
.   : milestone, 17873000,
profiling (15.112 s) : 15112000, 15112000
.   : milestone, 15112000,
tracing (15.393 s) : 15393000, 15393000
.   : milestone, 15393000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.615 s [15.615 s, 15.615 s] -
appsec 14.994 s [14.994 s, 14.994 s] -621.0 ms (-4.0%)
iast 18.882 s [18.882 s, 18.882 s] 3.267 s (20.9%)
iast_GLOBAL 18.156 s [18.156 s, 18.156 s] 2.541 s (16.3%)
profiling 15.103 s [15.103 s, 15.103 s] -512.0 ms (-3.3%)
tracing 15.126 s [15.126 s, 15.126 s] -489.0 ms (-3.1%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.961 s [14.961 s, 14.961 s] -
appsec 15.002 s [15.002 s, 15.002 s] 41.0 ms (0.3%)
iast 18.772 s [18.772 s, 18.772 s] 3.811 s (25.5%)
iast_GLOBAL 17.873 s [17.873 s, 17.873 s] 2.912 s (19.5%)
profiling 15.112 s [15.112 s, 15.112 s] 151.0 ms (1.0%)
tracing 15.393 s [15.393 s, 15.393 s] 432.0 ms (2.9%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.41.0-SNAPSHOT~6d5c892d40, baseline=1.41.0-SNAPSHOT~9db260fd16
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.461 ms) : 1450, 1472
.   : milestone, 1461,
appsec (2.299 ms) : 2258, 2339
.   : milestone, 2299,
iast (2.066 ms) : 2014, 2119
.   : milestone, 2066,
iast_GLOBAL (2.088 ms) : 2037, 2140
.   : milestone, 2088,
profiling (1.94 ms) : 1898, 1981
.   : milestone, 1940,
tracing (1.907 ms) : 1867, 1947
.   : milestone, 1907,
section candidate
no_agent (1.46 ms) : 1448, 1471
.   : milestone, 1460,
appsec (2.315 ms) : 2274, 2356
.   : milestone, 2315,
iast (2.048 ms) : 1996, 2099
.   : milestone, 2048,
iast_GLOBAL (2.105 ms) : 2053, 2157
.   : milestone, 2105,
profiling (1.921 ms) : 1880, 1962
.   : milestone, 1921,
tracing (1.903 ms) : 1863, 1943
.   : milestone, 1903,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.461 ms [1.45 ms, 1.472 ms] -
appsec 2.299 ms [2.258 ms, 2.339 ms] 837.662 µs (57.3%)
iast 2.066 ms [2.014 ms, 2.119 ms] 605.137 µs (41.4%)
iast_GLOBAL 2.088 ms [2.037 ms, 2.14 ms] 627.103 µs (42.9%)
profiling 1.94 ms [1.898 ms, 1.981 ms] 478.493 µs (32.8%)
tracing 1.907 ms [1.867 ms, 1.947 ms] 445.873 µs (30.5%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.46 ms [1.448 ms, 1.471 ms] -
appsec 2.315 ms [2.274 ms, 2.356 ms] 855.207 µs (58.6%)
iast 2.048 ms [1.996 ms, 2.099 ms] 587.684 µs (40.3%)
iast_GLOBAL 2.105 ms [2.053 ms, 2.157 ms] 645.095 µs (44.2%)
profiling 1.921 ms [1.88 ms, 1.962 ms] 461.055 µs (31.6%)
tracing 1.903 ms [1.863 ms, 1.943 ms] 443.262 µs (30.4%)

@jpbempel jpbempel merged commit f596731 into master Oct 7, 2024
102 of 103 checks passed
@jpbempel jpbempel deleted the jpbempel/flaky-smoke-tests branch October 7, 2024 15:24
@github-actions github-actions bot added this to the 1.41.0 milestone Oct 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cimi cimi cimi approved these changes

@PerfectSlayer PerfectSlayer PerfectSlayer approved these changes

@dougqh dougqh Awaiting requested review from dougqh dougqh is a code owner automatically assigned from DataDog/apm-java

Assignees
No one assigned
Labels
comp: debugger Dynamic Instrumentation comp: testing Testing tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
1.41.0
Development

Successfully merging this pull request may close these issues.
3 participants
@jpbempel @cimi @PerfectSlayer