Skip to content

Run tests on all jdks as part of nightly instead of every master pipeline #8900

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
May 29, 2025
Merged

Run tests on all jdks as part of nightly instead of every master pipeline #8900

merged 1 commit into from
May 29, 2025

Conversation

@randomanderson
Copy link
Contributor

@randomanderson randomanderson commented May 29, 2025

What Does This Do

Instead of running all tests on all jdks as part of every master pipeline, with this PR, those tests only run on nightly.

Motivation

The nondefault jdks have tests that are always failing. This has prevented master from having a green build in weeks.

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@randomanderson randomanderson requested a review from a team as a code owner May 29, 2025 15:57
@randomanderson randomanderson requested review from erikayasuda and removed request for a team May 29, 2025 15:57
@randomanderson randomanderson added tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels May 29, 2025
@pr-commenter
Copy link

pr-commenter bot commented May 29, 2025

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master landerson/default-jdks-on-master
git_commit_date 1746789389 1748534096
git_commit_sha ad6d5fe 144468b
release_version 1.50.0-SNAPSHOT~ad6d5fef42 1.50.0-SNAPSHOT~144468baa0
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1748536426 1748536426
ci_job_id 959587378 959587378
ci_pipeline_id 66443673 66443673
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-x3kj13mk-project-304-concurrent-0-0kapnqry 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-x3kj13mk-project-304-concurrent-0-0kapnqry 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 4 performance improvements and 2 performance regressions! Performance is the same for 53 metrics, 12 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:AppSec better
[-9.038ms; -5.743ms] or [-16.383%; -10.409%]		 47.778ms 55.168ms
scenario:startup:insecure-bank:tracing:Remote Config worse
[+25.969µs; +70.873µs] or [+3.764%; +10.273%]		 738.342µs 689.921µs
scenario:startup:petclinic:profiling:ProfilingAgent better
[-7.402ms; -2.470ms] or [-6.752%; -2.253%]		 104.686ms 109.621ms
scenario:startup:petclinic:profiling:GlobalTracer better
[-19.920ms; -17.477ms] or [-5.260%; -4.615%]		 360.003ms 378.702ms
scenario:startup:petclinic:profiling:AppSec worse
[+5.495ms; +8.978ms] or [+10.066%; +16.446%]		 61.826ms 54.590ms
scenario:startup:petclinic:profiling:Profiling better
[-7.403ms; -2.470ms] or [-6.752%; -2.252%]		 104.710ms 109.647ms
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.50.0-SNAPSHOT~144468baa0, baseline=1.50.0-SNAPSHOT~ad6d5fef42

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.021 s) : 0, 1021346
Total [baseline] (8.659 s) : 0, 8658917
Agent [candidate] (1.029 s) : 0, 1028635
Total [candidate] (8.707 s) : 0, 8707241
section iast
Agent [baseline] (1.147 s) : 0, 1146997
Total [baseline] (9.246 s) : 0, 9245521
Agent [candidate] (1.151 s) : 0, 1150627
Total [candidate] (9.248 s) : 0, 9247811
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.157 s) : 0, 1156791
Total [baseline] (9.273 s) : 0, 9273078
Agent [candidate] (1.16 s) : 0, 1159982
Total [candidate] (9.206 s) : 0, 9205909
section iast_TELEMETRY_OFF
Agent [baseline] (1.147 s) : 0, 1146922
Total [baseline] (9.205 s) : 0, 9204933
Agent [candidate] (1.157 s) : 0, 1157481
Total [candidate] (9.285 s) : 0, 9284778
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.021 s -
Agent iast 1.147 s 125.651 ms (12.3%)
Agent iast_HARDCODED_SECRET_DISABLED 1.157 s 135.445 ms (13.3%)
Agent iast_TELEMETRY_OFF 1.147 s 125.575 ms (12.3%)
Total tracing 8.659 s -
Total iast 9.246 s 586.604 ms (6.8%)
Total iast_HARDCODED_SECRET_DISABLED 9.273 s 614.161 ms (7.1%)
Total iast_TELEMETRY_OFF 9.205 s 546.016 ms (6.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.029 s -
Agent iast 1.151 s 121.992 ms (11.9%)
Agent iast_HARDCODED_SECRET_DISABLED 1.16 s 131.347 ms (12.8%)
Agent iast_TELEMETRY_OFF 1.157 s 128.846 ms (12.5%)
Total tracing 8.707 s -
Total iast 9.248 s 540.57 ms (6.2%)
Total iast_HARDCODED_SECRET_DISABLED 9.206 s 498.668 ms (5.7%)
Total iast_TELEMETRY_OFF 9.285 s 577.537 ms (6.6%) 
gantt
    title insecure-bank - break down per module: candidate=1.50.0-SNAPSHOT~144468baa0, baseline=1.50.0-SNAPSHOT~ad6d5fef42

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (682.26 ms) : 0, 682260
BytebuddyAgent [candidate] (688.615 ms) : 0, 688615
GlobalTracer [baseline] (239.965 ms) : 0, 239965
GlobalTracer [candidate] (242.465 ms) : 0, 242465
AppSec [baseline] (54.759 ms) : 0, 54759
AppSec [candidate] (55.602 ms) : 0, 55602
Debugger [baseline] (10.555 ms) : 0, 10555
Debugger [candidate] (7.054 ms) : 0, 7054
Remote Config [baseline] (689.921 µs) : 0, 690
Remote Config [candidate] (738.342 µs) : 0, 738
Telemetry [baseline] (9.733 ms) : 0, 9733
Telemetry [candidate] (10.46 ms) : 0, 10460
section iast
BytebuddyAgent [baseline] (800.548 ms) : 0, 800548
BytebuddyAgent [candidate] (802.746 ms) : 0, 802746
GlobalTracer [baseline] (230.486 ms) : 0, 230486
GlobalTracer [candidate] (231.351 ms) : 0, 231351
AppSec [baseline] (50.072 ms) : 0, 50072
AppSec [candidate] (51.926 ms) : 0, 51926
Debugger [baseline] (5.894 ms) : 0, 5894
Debugger [candidate] (5.838 ms) : 0, 5838
Remote Config [baseline] (583.984 µs) : 0, 584
Remote Config [candidate] (589.473 µs) : 0, 589
Telemetry [baseline] (7.854 ms) : 0, 7854
Telemetry [candidate] (7.87 ms) : 0, 7870
IAST [baseline] (28.095 ms) : 0, 28095
IAST [candidate] (26.867 ms) : 0, 26867
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (807.396 ms) : 0, 807396
BytebuddyAgent [candidate] (809.266 ms) : 0, 809266
GlobalTracer [baseline] (231.912 ms) : 0, 231912
GlobalTracer [candidate] (233.208 ms) : 0, 233208
AppSec [baseline] (49.053 ms) : 0, 49053
AppSec [candidate] (52.352 ms) : 0, 52352
Debugger [baseline] (5.958 ms) : 0, 5958
Debugger [candidate] (5.996 ms) : 0, 5996
Remote Config [baseline] (596.756 µs) : 0, 597
Remote Config [candidate] (590.555 µs) : 0, 591
Telemetry [baseline] (8.03 ms) : 0, 8030
Telemetry [candidate] (7.925 ms) : 0, 7925
IAST [baseline] (30.259 ms) : 0, 30259
IAST [candidate] (26.99 ms) : 0, 26990
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (799.787 ms) : 0, 799787
BytebuddyAgent [candidate] (809.084 ms) : 0, 809084
GlobalTracer [baseline] (230.792 ms) : 0, 230792
GlobalTracer [candidate] (232.411 ms) : 0, 232411
AppSec [baseline] (55.168 ms) : 0, 55168
AppSec [candidate] (47.778 ms) : 0, 47778
Debugger [baseline] (5.959 ms) : 0, 5959
Debugger [candidate] (5.883 ms) : 0, 5883
Remote Config [baseline] (607.743 µs) : 0, 608
Remote Config [candidate] (594.377 µs) : 0, 594
Telemetry [baseline] (7.761 ms) : 0, 7761
Telemetry [candidate] (7.86 ms) : 0, 7860
IAST [baseline] (23.312 ms) : 0, 23312
IAST [candidate] (30.2 ms) : 0, 30200
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.50.0-SNAPSHOT~144468baa0, baseline=1.50.0-SNAPSHOT~ad6d5fef42

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.028 s) : 0, 1028499
Total [baseline] (10.495 s) : 0, 10495285
Agent [candidate] (1.026 s) : 0, 1025756
Total [candidate] (10.521 s) : 0, 10521456
section appsec
Agent [baseline] (1.162 s) : 0, 1161897
Total [baseline] (10.716 s) : 0, 10715676
Agent [candidate] (1.17 s) : 0, 1169994
Total [candidate] (10.73 s) : 0, 10730301
section iast
Agent [baseline] (1.151 s) : 0, 1150728
Total [baseline] (10.88 s) : 0, 10879579
Agent [candidate] (1.151 s) : 0, 1150701
Total [candidate] (10.899 s) : 0, 10899260
section profiling
Agent [baseline] (1.284 s) : 0, 1283675
Total [baseline] (10.875 s) : 0, 10874742
Agent [candidate] (1.268 s) : 0, 1267738
Total [candidate] (10.836 s) : 0, 10836496
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.028 s -
Agent appsec 1.162 s 133.398 ms (13.0%)
Agent iast 1.151 s 122.229 ms (11.9%)
Agent profiling 1.284 s 255.177 ms (24.8%)
Total tracing 10.495 s -
Total appsec 10.716 s 220.391 ms (2.1%)
Total iast 10.88 s 384.294 ms (3.7%)
Total profiling 10.875 s 379.458 ms (3.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.026 s -
Agent appsec 1.17 s 144.237 ms (14.1%)
Agent iast 1.151 s 124.945 ms (12.2%)
Agent profiling 1.268 s 241.982 ms (23.6%)
Total tracing 10.521 s -
Total appsec 10.73 s 208.845 ms (2.0%)
Total iast 10.899 s 377.804 ms (3.6%)
Total profiling 10.836 s 315.04 ms (3.0%) 
gantt
    title petclinic - break down per module: candidate=1.50.0-SNAPSHOT~144468baa0, baseline=1.50.0-SNAPSHOT~ad6d5fef42

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (688.829 ms) : 0, 688829
BytebuddyAgent [candidate] (685.447 ms) : 0, 685447
GlobalTracer [baseline] (242.279 ms) : 0, 242279
GlobalTracer [candidate] (241.273 ms) : 0, 241273
AppSec [baseline] (55.925 ms) : 0, 55925
AppSec [candidate] (56.5 ms) : 0, 56500
Debugger [baseline] (7.799 ms) : 0, 7799
Debugger [candidate] (6.208 ms) : 0, 6208
Remote Config [baseline] (715.598 µs) : 0, 716
Remote Config [candidate] (716.891 µs) : 0, 717
Telemetry [baseline] (9.251 ms) : 0, 9251
Telemetry [candidate] (11.917 ms) : 0, 11917
section appsec
BytebuddyAgent [baseline] (700.498 ms) : 0, 700498
BytebuddyAgent [candidate] (705.347 ms) : 0, 705347
GlobalTracer [baseline] (236.887 ms) : 0, 236887
GlobalTracer [candidate] (239.286 ms) : 0, 239286
AppSec [baseline] (175.762 ms) : 0, 175762
AppSec [candidate] (176.684 ms) : 0, 176684
Debugger [baseline] (6.352 ms) : 0, 6352
Debugger [candidate] (5.999 ms) : 0, 5999
Remote Config [baseline] (617.957 µs) : 0, 618
Remote Config [candidate] (639.262 µs) : 0, 639
Telemetry [baseline] (7.345 ms) : 0, 7345
Telemetry [candidate] (7.348 ms) : 0, 7348
IAST [baseline] (21.725 ms) : 0, 21725
IAST [candidate] (22.059 ms) : 0, 22059
section iast
BytebuddyAgent [baseline] (803.869 ms) : 0, 803869
BytebuddyAgent [candidate] (803.206 ms) : 0, 803206
GlobalTracer [baseline] (230.956 ms) : 0, 230956
GlobalTracer [candidate] (231.261 ms) : 0, 231261
AppSec [baseline] (50.953 ms) : 0, 50953
AppSec [candidate] (50.068 ms) : 0, 50068
Debugger [baseline] (5.876 ms) : 0, 5876
Debugger [candidate] (5.886 ms) : 0, 5886
Remote Config [baseline] (587.449 µs) : 0, 587
Remote Config [candidate] (576.095 µs) : 0, 576
Telemetry [baseline] (7.798 ms) : 0, 7798
Telemetry [candidate] (7.817 ms) : 0, 7817
IAST [baseline] (27.177 ms) : 0, 27177
IAST [candidate] (27.614 ms) : 0, 27614
section profiling
BytebuddyAgent [baseline] (675.164 ms) : 0, 675164
BytebuddyAgent [candidate] (675.7 ms) : 0, 675700
GlobalTracer [baseline] (378.702 ms) : 0, 378702
GlobalTracer [candidate] (360.003 ms) : 0, 360003
AppSec [baseline] (54.59 ms) : 0, 54590
AppSec [candidate] (61.826 ms) : 0, 61826
Debugger [baseline] (6.121 ms) : 0, 6121
Debugger [candidate] (6.107 ms) : 0, 6107
Remote Config [baseline] (652.779 µs) : 0, 653
Remote Config [candidate] (637.936 µs) : 0, 638
Telemetry [baseline] (8.016 ms) : 0, 8016
Telemetry [candidate] (8.099 ms) : 0, 8099
ProfilingAgent [baseline] (109.621 ms) : 0, 109621
ProfilingAgent [candidate] (104.686 ms) : 0, 104686
Profiling [baseline] (109.647 ms) : 0, 109647
Profiling [candidate] (104.71 ms) : 0, 104710
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-05-29T16:04:04 2025-05-29T16:11:51
git_branch master landerson/default-jdks-on-master
git_commit_date 1746789389 1748534096
git_commit_sha ad6d5fe 144468b
release_version 1.50.0-SNAPSHOT~ad6d5fef42 1.50.0-SNAPSHOT~144468baa0
start_time 2025-05-29T16:03:50 2025-05-29T16:11:37
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1748535510 1748535510
ci_job_id 959587379 959587379
ci_pipeline_id 66443673 66443673
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-ukvdurnf-project-304-concurrent-0-89k8ej1y 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-ukvdurnf-project-304-concurrent-0-89k8ej1y 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 18 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~144468baa0, baseline=1.50.0-SNAPSHOT~ad6d5fef42
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.363 ms) : 1344, 1383
.   : milestone, 1363,
appsec (1.746 ms) : 1722, 1770
.   : milestone, 1746,
appsec_no_iast (1.722 ms) : 1699, 1746
.   : milestone, 1722,
code_origins (1.695 ms) : 1668, 1722
.   : milestone, 1695,
iast (1.517 ms) : 1493, 1540
.   : milestone, 1517,
profiling (1.533 ms) : 1508, 1559
.   : milestone, 1533,
tracing (1.523 ms) : 1497, 1549
.   : milestone, 1523,
section candidate
no_agent (1.369 ms) : 1348, 1390
.   : milestone, 1369,
appsec (1.752 ms) : 1728, 1776
.   : milestone, 1752,
appsec_no_iast (1.724 ms) : 1701, 1748
.   : milestone, 1724,
code_origins (1.669 ms) : 1642, 1696
.   : milestone, 1669,
iast (1.532 ms) : 1508, 1557
.   : milestone, 1532,
profiling (1.538 ms) : 1515, 1561
.   : milestone, 1538,
tracing (1.505 ms) : 1481, 1530
.   : milestone, 1505,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.363 ms [1.344 ms, 1.383 ms] -
appsec 1.746 ms [1.722 ms, 1.77 ms] 382.826 µs (28.1%)
appsec_no_iast 1.722 ms [1.699 ms, 1.746 ms] 358.933 µs (26.3%)
code_origins 1.695 ms [1.668 ms, 1.722 ms] 331.991 µs (24.4%)
iast 1.517 ms [1.493 ms, 1.54 ms] 153.769 µs (11.3%)
profiling 1.533 ms [1.508 ms, 1.559 ms] 170.175 µs (12.5%)
tracing 1.523 ms [1.497 ms, 1.549 ms] 159.813 µs (11.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.369 ms [1.348 ms, 1.39 ms] -
appsec 1.752 ms [1.728 ms, 1.776 ms] 383.057 µs (28.0%)
appsec_no_iast 1.724 ms [1.701 ms, 1.748 ms] 355.362 µs (26.0%)
code_origins 1.669 ms [1.642 ms, 1.696 ms] 299.882 µs (21.9%)
iast 1.532 ms [1.508 ms, 1.557 ms] 163.44 µs (11.9%)
profiling 1.538 ms [1.515 ms, 1.561 ms] 169.183 µs (12.4%)
tracing 1.505 ms [1.481 ms, 1.53 ms] 136.222 µs (10.0%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~144468baa0, baseline=1.50.0-SNAPSHOT~ad6d5fef42
    dateFormat X
    axisFormat %s
section baseline
no_agent (389.579 µs) : 369, 410
.   : milestone, 390,
iast (522.103 µs) : 500, 544
.   : milestone, 522,
iast_FULL (743.886 µs) : 722, 766
.   : milestone, 744,
iast_GLOBAL (574.843 µs) : 552, 597
.   : milestone, 575,
iast_HARDCODED_SECRET_DISABLED (526.342 µs) : 504, 549
.   : milestone, 526,
iast_INACTIVE (468.732 µs) : 446, 491
.   : milestone, 469,
iast_TELEMETRY_OFF (522.783 µs) : 500, 545
.   : milestone, 523,
tracing (467.638 µs) : 443, 492
.   : milestone, 468,
section candidate
no_agent (391.542 µs) : 372, 411
.   : milestone, 392,
iast (521.754 µs) : 499, 544
.   : milestone, 522,
iast_FULL (743.275 µs) : 721, 765
.   : milestone, 743,
iast_GLOBAL (573.135 µs) : 551, 595
.   : milestone, 573,
iast_HARDCODED_SECRET_DISABLED (528.316 µs) : 506, 550
.   : milestone, 528,
iast_INACTIVE (467.002 µs) : 445, 489
.   : milestone, 467,
iast_TELEMETRY_OFF (511.522 µs) : 489, 534
.   : milestone, 512,
tracing (466.75 µs) : 445, 489
.   : milestone, 467,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 389.579 µs [368.934 µs, 410.225 µs] -
iast 522.103 µs [500.392 µs, 543.814 µs] 132.523 µs (34.0%)
iast_FULL 743.886 µs [721.977 µs, 765.795 µs] 354.306 µs (90.9%)
iast_GLOBAL 574.843 µs [552.349 µs, 597.336 µs] 185.263 µs (47.6%)
iast_HARDCODED_SECRET_DISABLED 526.342 µs [504.107 µs, 548.578 µs] 136.763 µs (35.1%)
iast_INACTIVE 468.732 µs [446.219 µs, 491.245 µs] 79.153 µs (20.3%)
iast_TELEMETRY_OFF 522.783 µs [500.098 µs, 545.469 µs] 133.204 µs (34.2%)
tracing 467.638 µs [443.346 µs, 491.931 µs] 78.059 µs (20.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 391.542 µs [371.799 µs, 411.285 µs] -
iast 521.754 µs [499.189 µs, 544.319 µs] 130.212 µs (33.3%)
iast_FULL 743.275 µs [721.217 µs, 765.333 µs] 351.733 µs (89.8%)
iast_GLOBAL 573.135 µs [550.869 µs, 595.402 µs] 181.594 µs (46.4%)
iast_HARDCODED_SECRET_DISABLED 528.316 µs [506.467 µs, 550.165 µs] 136.774 µs (34.9%)
iast_INACTIVE 467.002 µs [444.793 µs, 489.21 µs] 75.46 µs (19.3%)
iast_TELEMETRY_OFF 511.522 µs [489.035 µs, 534.009 µs] 119.98 µs (30.6%)
tracing 466.75 µs [444.925 µs, 488.575 µs] 75.208 µs (19.2%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master landerson/default-jdks-on-master
git_commit_date 1746789389 1748534096
git_commit_sha ad6d5fe 144468b
release_version 1.50.0-SNAPSHOT~ad6d5fef42 1.50.0-SNAPSHOT~144468baa0
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1748536147 1748536147
ci_job_id 959587380 959587380
ci_pipeline_id 66443673 66443673
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-x3kj13mk-project-304-concurrent-1-attahkxx 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-x3kj13mk-project-304-concurrent-1-attahkxx 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~144468baa0, baseline=1.50.0-SNAPSHOT~ad6d5fef42
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.974 s) : 14974000, 14974000
.   : milestone, 14974000,
appsec (14.785 s) : 14785000, 14785000
.   : milestone, 14785000,
iast (19.131 s) : 19131000, 19131000
.   : milestone, 19131000,
iast_GLOBAL (18.017 s) : 18017000, 18017000
.   : milestone, 18017000,
profiling (14.838 s) : 14838000, 14838000
.   : milestone, 14838000,
tracing (15.041 s) : 15041000, 15041000
.   : milestone, 15041000,
section candidate
no_agent (15.482 s) : 15482000, 15482000
.   : milestone, 15482000,
appsec (14.942 s) : 14942000, 14942000
.   : milestone, 14942000,
iast (19.094 s) : 19094000, 19094000
.   : milestone, 19094000,
iast_GLOBAL (18.001 s) : 18001000, 18001000
.   : milestone, 18001000,
profiling (15.086 s) : 15086000, 15086000
.   : milestone, 15086000,
tracing (14.865 s) : 14865000, 14865000
.   : milestone, 14865000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.974 s [14.974 s, 14.974 s] -
appsec 14.785 s [14.785 s, 14.785 s] -189.0 ms (-1.3%)
iast 19.131 s [19.131 s, 19.131 s] 4.157 s (27.8%)
iast_GLOBAL 18.017 s [18.017 s, 18.017 s] 3.043 s (20.3%)
profiling 14.838 s [14.838 s, 14.838 s] -136.0 ms (-0.9%)
tracing 15.041 s [15.041 s, 15.041 s] 67.0 ms (0.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.482 s [15.482 s, 15.482 s] -
appsec 14.942 s [14.942 s, 14.942 s] -540.0 ms (-3.5%)
iast 19.094 s [19.094 s, 19.094 s] 3.612 s (23.3%)
iast_GLOBAL 18.001 s [18.001 s, 18.001 s] 2.519 s (16.3%)
profiling 15.086 s [15.086 s, 15.086 s] -396.0 ms (-2.6%)
tracing 14.865 s [14.865 s, 14.865 s] -617.0 ms (-4.0%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~144468baa0, baseline=1.50.0-SNAPSHOT~ad6d5fef42
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.475 ms) : 1463, 1486
.   : milestone, 1475,
appsec (2.387 ms) : 2339, 2435
.   : milestone, 2387,
iast (2.167 ms) : 2107, 2228
.   : milestone, 2167,
iast_GLOBAL (2.213 ms) : 2152, 2273
.   : milestone, 2213,
profiling (2.015 ms) : 1967, 2063
.   : milestone, 2015,
tracing (1.991 ms) : 1944, 2037
.   : milestone, 1991,
section candidate
no_agent (1.472 ms) : 1460, 1483
.   : milestone, 1472,
appsec (2.393 ms) : 2345, 2442
.   : milestone, 2393,
iast (2.171 ms) : 2110, 2231
.   : milestone, 2171,
iast_GLOBAL (2.206 ms) : 2145, 2267
.   : milestone, 2206,
profiling (2.043 ms) : 1993, 2093
.   : milestone, 2043,
tracing (1.991 ms) : 1944, 2038
.   : milestone, 1991,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.475 ms [1.463 ms, 1.486 ms] -
appsec 2.387 ms [2.339 ms, 2.435 ms] 912.237 µs (61.9%)
iast 2.167 ms [2.107 ms, 2.228 ms] 692.245 µs (46.9%)
iast_GLOBAL 2.213 ms [2.152 ms, 2.273 ms] 737.587 µs (50.0%)
profiling 2.015 ms [1.967 ms, 2.063 ms] 540.021 µs (36.6%)
tracing 1.991 ms [1.944 ms, 2.037 ms] 515.961 µs (35.0%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.472 ms [1.46 ms, 1.483 ms] -
appsec 2.393 ms [2.345 ms, 2.442 ms] 921.53 µs (62.6%)
iast 2.171 ms [2.11 ms, 2.231 ms] 698.838 µs (47.5%)
iast_GLOBAL 2.206 ms [2.145 ms, 2.267 ms] 734.484 µs (49.9%)
profiling 2.043 ms [1.993 ms, 2.093 ms] 571.035 µs (38.8%)
tracing 1.991 ms [1.944 ms, 2.038 ms] 519.52 µs (35.3%)

erikayasuda
erikayasuda approved these changes May 29, 2025
View reviewed changes
Copy link

@erikayasuda erikayasuda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is it about non-default JDKs that guarantees those test failures?

@randomanderson randomanderson merged commit 2fe20fe into master May 29, 2025
509 of 510 checks passed
@randomanderson randomanderson deleted the landerson/default-jdks-on-master branch May 29, 2025 20:37
@github-actions github-actions bot added this to the 1.50.0 milestone May 29, 2025
@randomanderson randomanderson mentioned this pull request Jun 3, 2025
Merged
bric3 added a commit that referenced this pull request Jun 4, 2025
@bric3
Revert "Only run all jdks as part of nightly (#8900)"
319fa31 
This reverts commit 2fe20fe.
@bric3 bric3 mentioned this pull request Jun 4, 2025
Merged
PerfectSlayer pushed a commit that referenced this pull request Jun 5, 2025
@bric3
Revert "Only run all jdks as part of nightly (#8900)" (#8925)
a9208b0 
This reverts commit 2fe20fe.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@erikayasuda erikayasuda erikayasuda approved these changes

Assignees

No one assigned

Labels

comp: tooling Build & Tooling tag: no release notes Changes to exclude from release notes

Projects

None yet

Milestone

1.50.0

Development

Successfully merging this pull request may close these issues.

3 participants

@randomanderson @erikayasuda