Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Assert test run status in CI Vis instrumentation tests #8316

Merged
merged 1 commit into from
Jan 31, 2025
Merged

Assert test run status in CI Vis instrumentation tests #8316

merged 1 commit into from
Jan 31, 2025

Conversation

nikita-tkachenko-datadog
Copy link
Contributor

@nikita-tkachenko-datadog nikita-tkachenko-datadog commented Jan 30, 2025 •
edited
Loading

What Does This Do

Updates CI Visibility instrumentation tests.
Now the tests assert the status of the run: successful or failed.

Motivation

Preparation for tests quarantining feature.
The main point of quarantining is avoiding failing the build if a quarantined test fails, so we will need to assert the status of the run.

Additional Notes

Also fixes a problem in Karate instrumentation: when doing auto-test retries the failure in the last attempt was not propagated to the build result.

Also removes a couple of redundant test cases for TestNG 6 (older TestNG versions do not support test retries).

Contributor Checklist

Jira ticket: SDTEST-1483

@nikita-tkachenko-datadog nikita-tkachenko-datadog added comp: testing Testing tag: no release notes Changes to exclude from release notes comp: ci visibility Continuous Integration Visibility labels Jan 30, 2025
@pr-commenter
Copy link

pr-commenter bot commented Jan 30, 2025

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/assert-test-run-status
git_commit_date 1738260668 1738266625
git_commit_sha 2b2be1e 3e8d229
release_version 1.46.0-SNAPSHOT~2b2be1ef8d 1.46.0-SNAPSHOT~3e8d229110
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1738269110 1738269110
ci_job_id 788225216 788225216
ci_pipeline_id 54504527 54504527
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-dy5wxjcg-project-304-concurrent-0-qpwterhj 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-dy5wxjcg-project-304-concurrent-0-qpwterhj 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 51 metrics, 12 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.46.0-SNAPSHOT~3e8d229110, baseline=1.46.0-SNAPSHOT~2b2be1ef8d

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.052 s) : 0, 1051970
Total [baseline] (10.534 s) : 0, 10533911
Agent [candidate] (1.041 s) : 0, 1041004
Total [candidate] (10.525 s) : 0, 10525107
section appsec
Agent [baseline] (1.187 s) : 0, 1186839
Total [baseline] (10.753 s) : 0, 10752764
Agent [candidate] (1.185 s) : 0, 1184621
Total [candidate] (10.854 s) : 0, 10853753
section iast
Agent [baseline] (1.184 s) : 0, 1183934
Total [baseline] (11.024 s) : 0, 11023738
Agent [candidate] (1.17 s) : 0, 1169502
Total [candidate] (10.936 s) : 0, 10936247
section profiling
Agent [baseline] (1.265 s) : 0, 1264633
Total [baseline] (10.863 s) : 0, 10863310
Agent [candidate] (1.258 s) : 0, 1258197
Total [candidate] (10.848 s) : 0, 10848434
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.052 s -
Agent appsec 1.187 s 134.87 ms (12.8%)
Agent iast 1.184 s 131.964 ms (12.5%)
Agent profiling 1.265 s 212.663 ms (20.2%)
Total tracing 10.534 s -
Total appsec 10.753 s 218.853 ms (2.1%)
Total iast 11.024 s 489.827 ms (4.7%)
Total profiling 10.863 s 329.399 ms (3.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.041 s -
Agent appsec 1.185 s 143.617 ms (13.8%)
Agent iast 1.17 s 128.498 ms (12.3%)
Agent profiling 1.258 s 217.194 ms (20.9%)
Total tracing 10.525 s -
Total appsec 10.854 s 328.646 ms (3.1%)
Total iast 10.936 s 411.14 ms (3.9%)
Total profiling 10.848 s 323.328 ms (3.1%) 
gantt
    title petclinic - break down per module: candidate=1.46.0-SNAPSHOT~3e8d229110, baseline=1.46.0-SNAPSHOT~2b2be1ef8d

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (724.639 ms) : 0, 724639
BytebuddyAgent [candidate] (715.996 ms) : 0, 715996
GlobalTracer [baseline] (243.409 ms) : 0, 243409
GlobalTracer [candidate] (240.195 ms) : 0, 240195
AppSec [baseline] (56.108 ms) : 0, 56108
AppSec [candidate] (55.211 ms) : 0, 55211
Remote Config [baseline] (727.932 µs) : 0, 728
Remote Config [candidate] (707.768 µs) : 0, 708
Telemetry [baseline] (11.689 ms) : 0, 11689
Telemetry [candidate] (13.602 ms) : 0, 13602
section appsec
BytebuddyAgent [baseline] (735.357 ms) : 0, 735357
BytebuddyAgent [candidate] (735.259 ms) : 0, 735259
GlobalTracer [baseline] (239.039 ms) : 0, 239039
GlobalTracer [candidate] (238.28 ms) : 0, 238280
AppSec [baseline] (177.124 ms) : 0, 177124
AppSec [candidate] (175.538 ms) : 0, 175538
Remote Config [baseline] (665.175 µs) : 0, 665
Remote Config [candidate] (655.035 µs) : 0, 655
Telemetry [baseline] (8.357 ms) : 0, 8357
Telemetry [candidate] (8.67 ms) : 0, 8670
IAST [baseline] (21.921 ms) : 0, 21921
IAST [candidate] (21.739 ms) : 0, 21739
section iast
BytebuddyAgent [baseline] (843.984 ms) : 0, 843984
BytebuddyAgent [candidate] (834.28 ms) : 0, 834280
GlobalTracer [baseline] (234.602 ms) : 0, 234602
GlobalTracer [candidate] (231.332 ms) : 0, 231332
AppSec [baseline] (53.602 ms) : 0, 53602
AppSec [candidate] (54.375 ms) : 0, 54375
Remote Config [baseline] (618.4 µs) : 0, 618
Remote Config [candidate] (616.125 µs) : 0, 616
Telemetry [baseline] (8.793 ms) : 0, 8793
Telemetry [candidate] (8.712 ms) : 0, 8712
IAST [baseline] (26.689 ms) : 0, 26689
IAST [candidate] (24.874 ms) : 0, 24874
section profiling
ProfilingAgent [baseline] (95.13 ms) : 0, 95130
ProfilingAgent [candidate] (95.23 ms) : 0, 95230
BytebuddyAgent [baseline] (709.873 ms) : 0, 709873
BytebuddyAgent [candidate] (705.863 ms) : 0, 705863
GlobalTracer [baseline] (352.384 ms) : 0, 352384
GlobalTracer [candidate] (351.33 ms) : 0, 351330
AppSec [baseline] (55.037 ms) : 0, 55037
AppSec [candidate] (53.879 ms) : 0, 53879
Remote Config [baseline] (704.912 µs) : 0, 705
Remote Config [candidate] (692.437 µs) : 0, 692
Telemetry [baseline] (8.863 ms) : 0, 8863
Telemetry [candidate] (8.87 ms) : 0, 8870
Profiling [baseline] (95.154 ms) : 0, 95154
Profiling [candidate] (95.254 ms) : 0, 95254
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.46.0-SNAPSHOT~3e8d229110, baseline=1.46.0-SNAPSHOT~2b2be1ef8d

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.038 s) : 0, 1038237
Total [baseline] (8.665 s) : 0, 8664700
Agent [candidate] (1.045 s) : 0, 1044657
Total [candidate] (8.671 s) : 0, 8670758
section iast
Agent [baseline] (1.175 s) : 0, 1174642
Total [baseline] (9.243 s) : 0, 9242534
Agent [candidate] (1.17 s) : 0, 1169805
Total [candidate] (9.218 s) : 0, 9217951
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.179 s) : 0, 1178541
Total [baseline] (9.228 s) : 0, 9227578
Agent [candidate] (1.171 s) : 0, 1170681
Total [candidate] (9.287 s) : 0, 9286971
section iast_TELEMETRY_OFF
Agent [baseline] (1.168 s) : 0, 1168159
Total [baseline] (9.223 s) : 0, 9222949
Agent [candidate] (1.171 s) : 0, 1171035
Total [candidate] (9.246 s) : 0, 9245562
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.038 s -
Agent iast 1.175 s 136.405 ms (13.1%)
Agent iast_HARDCODED_SECRET_DISABLED 1.179 s 140.304 ms (13.5%)
Agent iast_TELEMETRY_OFF 1.168 s 129.923 ms (12.5%)
Total tracing 8.665 s -
Total iast 9.243 s 577.835 ms (6.7%)
Total iast_HARDCODED_SECRET_DISABLED 9.228 s 562.878 ms (6.5%)
Total iast_TELEMETRY_OFF 9.223 s 558.249 ms (6.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.045 s -
Agent iast 1.17 s 125.148 ms (12.0%)
Agent iast_HARDCODED_SECRET_DISABLED 1.171 s 126.023 ms (12.1%)
Agent iast_TELEMETRY_OFF 1.171 s 126.377 ms (12.1%)
Total tracing 8.671 s -
Total iast 9.218 s 547.193 ms (6.3%)
Total iast_HARDCODED_SECRET_DISABLED 9.287 s 616.213 ms (7.1%)
Total iast_TELEMETRY_OFF 9.246 s 574.804 ms (6.6%) 
gantt
    title insecure-bank - break down per module: candidate=1.46.0-SNAPSHOT~3e8d229110, baseline=1.46.0-SNAPSHOT~2b2be1ef8d

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (714.988 ms) : 0, 714988
BytebuddyAgent [candidate] (718.328 ms) : 0, 718328
GlobalTracer [baseline] (240.348 ms) : 0, 240348
GlobalTracer [candidate] (240.91 ms) : 0, 240910
AppSec [baseline] (55.568 ms) : 0, 55568
AppSec [candidate] (56.342 ms) : 0, 56342
Remote Config [baseline] (719.541 µs) : 0, 720
Remote Config [candidate] (720.164 µs) : 0, 720
Telemetry [baseline] (11.381 ms) : 0, 11381
Telemetry [candidate] (13.09 ms) : 0, 13090
section iast
BytebuddyAgent [baseline] (839.476 ms) : 0, 839476
BytebuddyAgent [candidate] (834.643 ms) : 0, 834643
GlobalTracer [baseline] (231.157 ms) : 0, 231157
GlobalTracer [candidate] (230.805 ms) : 0, 230805
IAST [baseline] (26.349 ms) : 0, 26349
IAST [candidate] (24.2 ms) : 0, 24200
AppSec [baseline] (52.968 ms) : 0, 52968
AppSec [candidate] (55.6 ms) : 0, 55600
Remote Config [baseline] (608.634 µs) : 0, 609
Remote Config [candidate] (607.83 µs) : 0, 608
Telemetry [baseline] (8.649 ms) : 0, 8649
Telemetry [candidate] (8.643 ms) : 0, 8643
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (840.014 ms) : 0, 840014
BytebuddyAgent [candidate] (834.829 ms) : 0, 834829
GlobalTracer [baseline] (232.652 ms) : 0, 232652
GlobalTracer [candidate] (230.998 ms) : 0, 230998
IAST [baseline] (24.513 ms) : 0, 24513
IAST [candidate] (26.736 ms) : 0, 26736
AppSec [baseline] (56.452 ms) : 0, 56452
AppSec [candidate] (53.352 ms) : 0, 53352
Remote Config [baseline] (628.467 µs) : 0, 628
Remote Config [candidate] (621.154 µs) : 0, 621
Telemetry [baseline] (8.875 ms) : 0, 8875
Telemetry [candidate] (8.872 ms) : 0, 8872
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (832.997 ms) : 0, 832997
BytebuddyAgent [candidate] (835.349 ms) : 0, 835349
GlobalTracer [baseline] (231.545 ms) : 0, 231545
GlobalTracer [candidate] (231.919 ms) : 0, 231919
IAST [baseline] (25.951 ms) : 0, 25951
IAST [candidate] (26.077 ms) : 0, 26077
AppSec [baseline] (53.087 ms) : 0, 53087
AppSec [candidate] (53.117 ms) : 0, 53117
Remote Config [baseline] (616.667 µs) : 0, 617
Remote Config [candidate] (632.97 µs) : 0, 633
Telemetry [baseline] (8.591 ms) : 0, 8591
Telemetry [candidate] (8.591 ms) : 0, 8591
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-01-30T19:59:41 2025-01-30T20:06:44
git_branch master nikita-tkachenko/assert-test-run-status
git_commit_date 1738260668 1738266625
git_commit_sha 2b2be1e 3e8d229
release_version 1.46.0-SNAPSHOT~2b2be1ef8d 1.46.0-SNAPSHOT~3e8d229110
start_time 2025-01-30T19:59:28 2025-01-30T20:06:30
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1738267962 1738267962
ci_job_id 788225218 788225218
ci_pipeline_id 54504527 54504527
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-lyuztghe-project-304-concurrent-0-7wwgvxmc 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-lyuztghe-project-304-concurrent-0-7wwgvxmc 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~3e8d229110, baseline=1.46.0-SNAPSHOT~2b2be1ef8d
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.359 ms) : 1339, 1378
.   : milestone, 1359,
appsec (1.713 ms) : 1690, 1736
.   : milestone, 1713,
appsec_no_iast (1.733 ms) : 1708, 1758
.   : milestone, 1733,
iast (1.518 ms) : 1494, 1543
.   : milestone, 1518,
profiling (1.557 ms) : 1532, 1582
.   : milestone, 1557,
tracing (1.475 ms) : 1449, 1500
.   : milestone, 1475,
section candidate
no_agent (1.363 ms) : 1343, 1383
.   : milestone, 1363,
appsec (1.754 ms) : 1730, 1777
.   : milestone, 1754,
appsec_no_iast (1.774 ms) : 1749, 1799
.   : milestone, 1774,
iast (1.496 ms) : 1472, 1520
.   : milestone, 1496,
profiling (1.52 ms) : 1495, 1544
.   : milestone, 1520,
tracing (1.498 ms) : 1473, 1522
.   : milestone, 1498,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.359 ms [1.339 ms, 1.378 ms] -
appsec 1.713 ms [1.69 ms, 1.736 ms] 354.187 µs (26.1%)
appsec_no_iast 1.733 ms [1.708 ms, 1.758 ms] 374.634 µs (27.6%)
iast 1.518 ms [1.494 ms, 1.543 ms] 159.604 µs (11.7%)
profiling 1.557 ms [1.532 ms, 1.582 ms] 198.368 µs (14.6%)
tracing 1.475 ms [1.449 ms, 1.5 ms] 115.992 µs (8.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.363 ms [1.343 ms, 1.383 ms] -
appsec 1.754 ms [1.73 ms, 1.777 ms] 390.505 µs (28.6%)
appsec_no_iast 1.774 ms [1.749 ms, 1.799 ms] 410.799 µs (30.1%)
iast 1.496 ms [1.472 ms, 1.52 ms] 133.118 µs (9.8%)
profiling 1.52 ms [1.495 ms, 1.544 ms] 156.597 µs (11.5%)
tracing 1.498 ms [1.473 ms, 1.522 ms] 134.419 µs (9.9%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~3e8d229110, baseline=1.46.0-SNAPSHOT~2b2be1ef8d
    dateFormat X
    axisFormat %s
section baseline
no_agent (391.217 µs) : 371, 411
.   : milestone, 391,
iast (503.558 µs) : 480, 527
.   : milestone, 504,
iast_FULL (740.615 µs) : 719, 763
.   : milestone, 741,
iast_GLOBAL (550.85 µs) : 529, 573
.   : milestone, 551,
iast_HARDCODED_SECRET_DISABLED (503.625 µs) : 482, 526
.   : milestone, 504,
iast_INACTIVE (455.381 µs) : 434, 476
.   : milestone, 455,
iast_TELEMETRY_OFF (488.198 µs) : 466, 510
.   : milestone, 488,
tracing (451.496 µs) : 430, 473
.   : milestone, 451,
section candidate
no_agent (370.743 µs) : 350, 392
.   : milestone, 371,
iast (508.519 µs) : 485, 532
.   : milestone, 509,
iast_FULL (738.77 µs) : 717, 761
.   : milestone, 739,
iast_GLOBAL (561.757 µs) : 537, 587
.   : milestone, 562,
iast_HARDCODED_SECRET_DISABLED (506.566 µs) : 484, 529
.   : milestone, 507,
iast_INACTIVE (453.953 µs) : 433, 475
.   : milestone, 454,
iast_TELEMETRY_OFF (487.985 µs) : 465, 511
.   : milestone, 488,
tracing (451.073 µs) : 430, 472
.   : milestone, 451,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 391.217 µs [371.136 µs, 411.298 µs] -
iast 503.558 µs [480.121 µs, 526.995 µs] 112.341 µs (28.7%)
iast_FULL 740.615 µs [718.637 µs, 762.592 µs] 349.398 µs (89.3%)
iast_GLOBAL 550.85 µs [528.962 µs, 572.738 µs] 159.633 µs (40.8%)
iast_HARDCODED_SECRET_DISABLED 503.625 µs [481.574 µs, 525.675 µs] 112.408 µs (28.7%)
iast_INACTIVE 455.381 µs [434.383 µs, 476.38 µs] 64.165 µs (16.4%)
iast_TELEMETRY_OFF 488.198 µs [465.896 µs, 510.5 µs] 96.981 µs (24.8%)
tracing 451.496 µs [430.202 µs, 472.79 µs] 60.279 µs (15.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 370.743 µs [349.815 µs, 391.67 µs] -
iast 508.519 µs [485.1 µs, 531.938 µs] 137.776 µs (37.2%)
iast_FULL 738.77 µs [716.936 µs, 760.605 µs] 368.028 µs (99.3%)
iast_GLOBAL 561.757 µs [536.57 µs, 586.944 µs] 191.014 µs (51.5%)
iast_HARDCODED_SECRET_DISABLED 506.566 µs [484.341 µs, 528.791 µs] 135.823 µs (36.6%)
iast_INACTIVE 453.953 µs [432.904 µs, 475.002 µs] 83.21 µs (22.4%)
iast_TELEMETRY_OFF 487.985 µs [465.104 µs, 510.866 µs] 117.242 µs (31.6%)
tracing 451.073 µs [430.038 µs, 472.108 µs] 80.33 µs (21.7%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/assert-test-run-status
git_commit_date 1738260668 1738266625
git_commit_sha 2b2be1e 3e8d229
release_version 1.46.0-SNAPSHOT~2b2be1ef8d 1.46.0-SNAPSHOT~3e8d229110
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1738268651 1738268651
ci_job_id 788225220 788225220
ci_pipeline_id 54504527 54504527
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-dy5wxjcg-project-304-concurrent-1-hvkxf2m3 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-dy5wxjcg-project-304-concurrent-1-hvkxf2m3 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~3e8d229110, baseline=1.46.0-SNAPSHOT~2b2be1ef8d
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.475 ms) : 1464, 1487
.   : milestone, 1475,
appsec (2.352 ms) : 2310, 2395
.   : milestone, 2352,
iast (2.119 ms) : 2065, 2174
.   : milestone, 2119,
iast_GLOBAL (2.179 ms) : 2123, 2234
.   : milestone, 2179,
profiling (1.976 ms) : 1932, 2019
.   : milestone, 1976,
tracing (1.955 ms) : 1913, 1998
.   : milestone, 1955,
section candidate
no_agent (1.479 ms) : 1468, 1491
.   : milestone, 1479,
appsec (2.372 ms) : 2329, 2416
.   : milestone, 2372,
iast (2.12 ms) : 2065, 2175
.   : milestone, 2120,
iast_GLOBAL (2.161 ms) : 2106, 2217
.   : milestone, 2161,
profiling (2.473 ms) : 2283, 2662
.   : milestone, 2473,
tracing (1.961 ms) : 1918, 2003
.   : milestone, 1961,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.475 ms [1.464 ms, 1.487 ms] -
appsec 2.352 ms [2.31 ms, 2.395 ms] 877.194 µs (59.5%)
iast 2.119 ms [2.065 ms, 2.174 ms] 643.961 µs (43.6%)
iast_GLOBAL 2.179 ms [2.123 ms, 2.234 ms] 703.342 µs (47.7%)
profiling 1.976 ms [1.932 ms, 2.019 ms] 500.337 µs (33.9%)
tracing 1.955 ms [1.913 ms, 1.998 ms] 480.13 µs (32.5%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.479 ms [1.468 ms, 1.491 ms] -
appsec 2.372 ms [2.329 ms, 2.416 ms] 892.991 µs (60.4%)
iast 2.12 ms [2.065 ms, 2.175 ms] 640.643 µs (43.3%)
iast_GLOBAL 2.161 ms [2.106 ms, 2.217 ms] 682.022 µs (46.1%)
profiling 2.473 ms [2.283 ms, 2.662 ms] 993.266 µs (67.1%)
tracing 1.961 ms [1.918 ms, 2.003 ms] 481.264 µs (32.5%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~3e8d229110, baseline=1.46.0-SNAPSHOT~2b2be1ef8d
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.278 s) : 15278000, 15278000
.   : milestone, 15278000,
appsec (15.095 s) : 15095000, 15095000
.   : milestone, 15095000,
iast (18.21 s) : 18210000, 18210000
.   : milestone, 18210000,
iast_GLOBAL (18.162 s) : 18162000, 18162000
.   : milestone, 18162000,
profiling (15.193 s) : 15193000, 15193000
.   : milestone, 15193000,
tracing (15.088 s) : 15088000, 15088000
.   : milestone, 15088000,
section candidate
no_agent (14.754 s) : 14754000, 14754000
.   : milestone, 14754000,
appsec (14.994 s) : 14994000, 14994000
.   : milestone, 14994000,
iast (18.556 s) : 18556000, 18556000
.   : milestone, 18556000,
iast_GLOBAL (17.949 s) : 17949000, 17949000
.   : milestone, 17949000,
profiling (15.105 s) : 15105000, 15105000
.   : milestone, 15105000,
tracing (14.745 s) : 14745000, 14745000
.   : milestone, 14745000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.278 s [15.278 s, 15.278 s] -
appsec 15.095 s [15.095 s, 15.095 s] -183.0 ms (-1.2%)
iast 18.21 s [18.21 s, 18.21 s] 2.932 s (19.2%)
iast_GLOBAL 18.162 s [18.162 s, 18.162 s] 2.884 s (18.9%)
profiling 15.193 s [15.193 s, 15.193 s] -85.0 ms (-0.6%)
tracing 15.088 s [15.088 s, 15.088 s] -190.0 ms (-1.2%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.754 s [14.754 s, 14.754 s] -
appsec 14.994 s [14.994 s, 14.994 s] 240.0 ms (1.6%)
iast 18.556 s [18.556 s, 18.556 s] 3.802 s (25.8%)
iast_GLOBAL 17.949 s [17.949 s, 17.949 s] 3.195 s (21.7%)
profiling 15.105 s [15.105 s, 15.105 s] 351.0 ms (2.4%)
tracing 14.745 s [14.745 s, 14.745 s] -9.0 ms (-0.1%)

@nikita-tkachenko-datadog nikita-tkachenko-datadog marked this pull request as ready for review January 31, 2025 09:57
@nikita-tkachenko-datadog nikita-tkachenko-datadog requested a review from a team as a code owner January 31, 2025 09:57
@nikita-tkachenko-datadog nikita-tkachenko-datadog requested review from tonyredondo and daniel-mohedano and removed request for a team and tonyredondo January 31, 2025 09:57
@nikita-tkachenko-datadog nikita-tkachenko-datadog merged commit b24d153 into master Jan 31, 2025
201 of 203 checks passed
@nikita-tkachenko-datadog nikita-tkachenko-datadog deleted the nikita-tkachenko/assert-test-run-status branch January 31, 2025 10:18
@github-actions github-actions bot added this to the 1.47.0 milestone Jan 31, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@daniel-mohedano daniel-mohedano daniel-mohedano approved these changes

Assignees
No one assigned
Labels
comp: ci visibility Continuous Integration Visibility comp: testing Testing tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
1.47.0
Development

Successfully merging this pull request may close these issues.
2 participants
@nikita-tkachenko-datadog @daniel-mohedano