Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement quarantined tests tagging #8326

Merged
merged 5 commits into from
Feb 3, 2025
Merged

Implement quarantined tests tagging #8326

merged 5 commits into from
Feb 3, 2025

Conversation

nikita-tkachenko-datadog
Copy link
Contributor

@nikita-tkachenko-datadog nikita-tkachenko-datadog commented Feb 3, 2025 •
edited
Loading

What Does This Do

Implements additional tags related to tests quarantine feature:

  • test.is_quarantined for tests cases that are quarantined
  • test.has_failed_all_retries for test cases that were retried multiple times and failed all of the executions

Contributor Checklist

Jira ticket: SDTEST-1483

@nikita-tkachenko-datadog nikita-tkachenko-datadog changed the title Nikita tkachenko/quarantined tests tagging Implement quarantined tests tagging Feb 3, 2025
@pr-commenter
Copy link

pr-commenter bot commented Feb 3, 2025

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/quarantined-tests-tagging
git_commit_date 1738578316 1738592916
git_commit_sha 9af5347 c80ed84
release_version 1.47.0-SNAPSHOT~9af5347cda 1.47.0-SNAPSHOT~c80ed84ecc
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1738595502 1738595502
ci_job_id 791309694 791309694
ci_pipeline_id 54719662 54719662
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-yhuzh6uy-project-304-concurrent-0-cl4wqp5e 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-yhuzh6uy-project-304-concurrent-0-cl4wqp5e 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 52 metrics, 11 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.47.0-SNAPSHOT~c80ed84ecc, baseline=1.47.0-SNAPSHOT~9af5347cda

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.036 s) : 0, 1035731
Total [baseline] (10.496 s) : 0, 10495525
Agent [candidate] (1.041 s) : 0, 1040529
Total [candidate] (10.508 s) : 0, 10508084
section appsec
Agent [baseline] (1.182 s) : 0, 1182005
Total [baseline] (10.773 s) : 0, 10773059
Agent [candidate] (1.181 s) : 0, 1180887
Total [candidate] (10.749 s) : 0, 10749435
section iast
Agent [baseline] (1.168 s) : 0, 1168048
Total [baseline] (10.912 s) : 0, 10912426
Agent [candidate] (1.17 s) : 0, 1170445
Total [candidate] (10.961 s) : 0, 10961074
section profiling
Agent [baseline] (1.26 s) : 0, 1259908
Total [baseline] (10.897 s) : 0, 10896989
Agent [candidate] (1.264 s) : 0, 1263998
Total [candidate] (10.868 s) : 0, 10868094
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.036 s -
Agent appsec 1.182 s 146.274 ms (14.1%)
Agent iast 1.168 s 132.317 ms (12.8%)
Agent profiling 1.26 s 224.177 ms (21.6%)
Total tracing 10.496 s -
Total appsec 10.773 s 277.534 ms (2.6%)
Total iast 10.912 s 416.901 ms (4.0%)
Total profiling 10.897 s 401.463 ms (3.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.041 s -
Agent appsec 1.181 s 140.358 ms (13.5%)
Agent iast 1.17 s 129.916 ms (12.5%)
Agent profiling 1.264 s 223.468 ms (21.5%)
Total tracing 10.508 s -
Total appsec 10.749 s 241.351 ms (2.3%)
Total iast 10.961 s 452.99 ms (4.3%)
Total profiling 10.868 s 360.01 ms (3.4%) 
gantt
    title petclinic - break down per module: candidate=1.47.0-SNAPSHOT~c80ed84ecc, baseline=1.47.0-SNAPSHOT~9af5347cda

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (712.644 ms) : 0, 712644
BytebuddyAgent [candidate] (716.057 ms) : 0, 716057
GlobalTracer [baseline] (241.844 ms) : 0, 241844
GlobalTracer [candidate] (240.871 ms) : 0, 240871
AppSec [baseline] (55.29 ms) : 0, 55290
AppSec [candidate] (55.454 ms) : 0, 55454
Remote Config [baseline] (714.278 µs) : 0, 714
Remote Config [candidate] (719.344 µs) : 0, 719
Telemetry [baseline] (10.082 ms) : 0, 10082
Telemetry [candidate] (12.237 ms) : 0, 12237
section appsec
BytebuddyAgent [baseline] (730.411 ms) : 0, 730411
BytebuddyAgent [candidate] (731.763 ms) : 0, 731763
GlobalTracer [baseline] (240.466 ms) : 0, 240466
GlobalTracer [candidate] (237.537 ms) : 0, 237537
AppSec [baseline] (176.253 ms) : 0, 176253
AppSec [candidate] (175.963 ms) : 0, 175963
Remote Config [baseline] (665.808 µs) : 0, 666
Remote Config [candidate] (661.538 µs) : 0, 662
Telemetry [baseline] (8.287 ms) : 0, 8287
Telemetry [candidate] (9.067 ms) : 0, 9067
IAST [baseline] (21.488 ms) : 0, 21488
IAST [candidate] (21.531 ms) : 0, 21531
section iast
BytebuddyAgent [baseline] (831.134 ms) : 0, 831134
BytebuddyAgent [candidate] (833.687 ms) : 0, 833687
GlobalTracer [baseline] (233.064 ms) : 0, 233064
GlobalTracer [candidate] (232.313 ms) : 0, 232313
AppSec [baseline] (54.553 ms) : 0, 54553
AppSec [candidate] (52.334 ms) : 0, 52334
Remote Config [baseline] (614.629 µs) : 0, 615
Remote Config [candidate] (621.459 µs) : 0, 621
Telemetry [baseline] (8.688 ms) : 0, 8688
Telemetry [candidate] (8.848 ms) : 0, 8848
IAST [baseline] (24.77 ms) : 0, 24770
IAST [candidate] (27.436 ms) : 0, 27436
section profiling
BytebuddyAgent [baseline] (704.753 ms) : 0, 704753
BytebuddyAgent [candidate] (709.882 ms) : 0, 709882
GlobalTracer [baseline] (352.42 ms) : 0, 352420
GlobalTracer [candidate] (352.362 ms) : 0, 352362
AppSec [baseline] (55.467 ms) : 0, 55467
AppSec [candidate] (54.375 ms) : 0, 54375
Remote Config [baseline] (701.715 µs) : 0, 702
Remote Config [candidate] (698.14 µs) : 0, 698
Telemetry [baseline] (8.93 ms) : 0, 8930
Telemetry [candidate] (8.953 ms) : 0, 8953
ProfilingAgent [baseline] (95.492 ms) : 0, 95492
ProfilingAgent [candidate] (95.306 ms) : 0, 95306
Profiling [baseline] (95.516 ms) : 0, 95516
Profiling [candidate] (95.33 ms) : 0, 95330
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.47.0-SNAPSHOT~c80ed84ecc, baseline=1.47.0-SNAPSHOT~9af5347cda

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.047 s) : 0, 1047222
Total [baseline] (8.668 s) : 0, 8667784
Agent [candidate] (1.037 s) : 0, 1036702
Total [candidate] (8.599 s) : 0, 8599286
section iast
Agent [baseline] (1.169 s) : 0, 1169078
Total [baseline] (9.25 s) : 0, 9250194
Agent [candidate] (1.175 s) : 0, 1175254
Total [candidate] (9.24 s) : 0, 9240317
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.172 s) : 0, 1171954
Total [baseline] (9.22 s) : 0, 9220375
Agent [candidate] (1.176 s) : 0, 1176241
Total [candidate] (9.182 s) : 0, 9182232
section iast_TELEMETRY_OFF
Agent [baseline] (1.166 s) : 0, 1165755
Total [baseline] (9.173 s) : 0, 9172934
Agent [candidate] (1.165 s) : 0, 1164923
Total [candidate] (9.232 s) : 0, 9231720
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.047 s -
Agent iast 1.169 s 121.857 ms (11.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.172 s 124.733 ms (11.9%)
Agent iast_TELEMETRY_OFF 1.166 s 118.533 ms (11.3%)
Total tracing 8.668 s -
Total iast 9.25 s 582.41 ms (6.7%)
Total iast_HARDCODED_SECRET_DISABLED 9.22 s 552.592 ms (6.4%)
Total iast_TELEMETRY_OFF 9.173 s 505.15 ms (5.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.037 s -
Agent iast 1.175 s 138.552 ms (13.4%)
Agent iast_HARDCODED_SECRET_DISABLED 1.176 s 139.539 ms (13.5%)
Agent iast_TELEMETRY_OFF 1.165 s 128.221 ms (12.4%)
Total tracing 8.599 s -
Total iast 9.24 s 641.03 ms (7.5%)
Total iast_HARDCODED_SECRET_DISABLED 9.182 s 582.945 ms (6.8%)
Total iast_TELEMETRY_OFF 9.232 s 632.433 ms (7.4%) 
gantt
    title insecure-bank - break down per module: candidate=1.47.0-SNAPSHOT~c80ed84ecc, baseline=1.47.0-SNAPSHOT~9af5347cda

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (720.886 ms) : 0, 720886
BytebuddyAgent [candidate] (713.042 ms) : 0, 713042
GlobalTracer [baseline] (242.708 ms) : 0, 242708
GlobalTracer [candidate] (239.833 ms) : 0, 239833
AppSec [baseline] (56.042 ms) : 0, 56042
AppSec [candidate] (55.122 ms) : 0, 55122
Remote Config [baseline] (710.415 µs) : 0, 710
Remote Config [candidate] (712.957 µs) : 0, 713
Telemetry [baseline] (11.448 ms) : 0, 11448
Telemetry [candidate] (12.794 ms) : 0, 12794
section iast
BytebuddyAgent [baseline] (832.171 ms) : 0, 832171
BytebuddyAgent [candidate] (838.644 ms) : 0, 838644
GlobalTracer [baseline] (233.454 ms) : 0, 233454
GlobalTracer [candidate] (232.218 ms) : 0, 232218
AppSec [baseline] (51.9 ms) : 0, 51900
AppSec [candidate] (54.083 ms) : 0, 54083
Remote Config [baseline] (601.772 µs) : 0, 602
Remote Config [candidate] (612.217 µs) : 0, 612
Telemetry [baseline] (8.615 ms) : 0, 8615
Telemetry [candidate] (8.734 ms) : 0, 8734
IAST [baseline] (27.067 ms) : 0, 27067
IAST [candidate] (25.685 ms) : 0, 25685
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (833.893 ms) : 0, 833893
BytebuddyAgent [candidate] (839.331 ms) : 0, 839331
GlobalTracer [baseline] (233.574 ms) : 0, 233574
GlobalTracer [candidate] (232.049 ms) : 0, 232049
AppSec [baseline] (53.985 ms) : 0, 53985
AppSec [candidate] (54.169 ms) : 0, 54169
Remote Config [baseline] (616.341 µs) : 0, 616
Remote Config [candidate] (621.214 µs) : 0, 621
Telemetry [baseline] (8.755 ms) : 0, 8755
Telemetry [candidate] (8.814 ms) : 0, 8814
IAST [baseline] (25.833 ms) : 0, 25833
IAST [candidate] (25.897 ms) : 0, 25897
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (829.568 ms) : 0, 829568
BytebuddyAgent [candidate] (830.541 ms) : 0, 830541
GlobalTracer [baseline] (233.244 ms) : 0, 233244
GlobalTracer [candidate] (230.781 ms) : 0, 230781
AppSec [baseline] (52.707 ms) : 0, 52707
AppSec [candidate] (53.1 ms) : 0, 53100
Remote Config [baseline] (612.999 µs) : 0, 613
Remote Config [candidate] (611.421 µs) : 0, 611
Telemetry [baseline] (8.548 ms) : 0, 8548
Telemetry [candidate] (8.627 ms) : 0, 8627
IAST [baseline] (25.849 ms) : 0, 25849
IAST [candidate] (26.065 ms) : 0, 26065
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-02-03T14:41:37 2025-02-03T14:48:41
git_branch master nikita-tkachenko/quarantined-tests-tagging
git_commit_date 1738578316 1738592916
git_commit_sha 9af5347 c80ed84
release_version 1.47.0-SNAPSHOT~9af5347cda 1.47.0-SNAPSHOT~c80ed84ecc
start_time 2025-02-03T14:41:23 2025-02-03T14:48:27
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1738594478 1738594478
ci_job_id 791309695 791309695
ci_pipeline_id 54719662 54719662
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-kmuzcns6-project-304-concurrent-0-haicqptf 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-kmuzcns6-project-304-concurrent-0-haicqptf 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~c80ed84ecc, baseline=1.47.0-SNAPSHOT~9af5347cda
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.35 ms) : 1330, 1369
.   : milestone, 1350,
appsec (1.773 ms) : 1750, 1796
.   : milestone, 1773,
appsec_no_iast (1.74 ms) : 1715, 1765
.   : milestone, 1740,
iast (1.516 ms) : 1492, 1540
.   : milestone, 1516,
profiling (1.547 ms) : 1523, 1572
.   : milestone, 1547,
tracing (1.483 ms) : 1458, 1508
.   : milestone, 1483,
section candidate
no_agent (1.351 ms) : 1331, 1370
.   : milestone, 1351,
appsec (1.742 ms) : 1718, 1765
.   : milestone, 1742,
appsec_no_iast (1.732 ms) : 1707, 1757
.   : milestone, 1732,
iast (1.508 ms) : 1484, 1531
.   : milestone, 1508,
profiling (1.533 ms) : 1508, 1558
.   : milestone, 1533,
tracing (1.49 ms) : 1466, 1514
.   : milestone, 1490,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.35 ms [1.33 ms, 1.369 ms] -
appsec 1.773 ms [1.75 ms, 1.796 ms] 423.13 µs (31.4%)
appsec_no_iast 1.74 ms [1.715 ms, 1.765 ms] 390.771 µs (29.0%)
iast 1.516 ms [1.492 ms, 1.54 ms] 166.728 µs (12.4%)
profiling 1.547 ms [1.523 ms, 1.572 ms] 197.896 µs (14.7%)
tracing 1.483 ms [1.458 ms, 1.508 ms] 133.21 µs (9.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.351 ms [1.331 ms, 1.37 ms] -
appsec 1.742 ms [1.718 ms, 1.765 ms] 391.254 µs (29.0%)
appsec_no_iast 1.732 ms [1.707 ms, 1.757 ms] 381.445 µs (28.2%)
iast 1.508 ms [1.484 ms, 1.531 ms] 157.002 µs (11.6%)
profiling 1.533 ms [1.508 ms, 1.558 ms] 182.727 µs (13.5%)
tracing 1.49 ms [1.466 ms, 1.514 ms] 139.45 µs (10.3%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~c80ed84ecc, baseline=1.47.0-SNAPSHOT~9af5347cda
    dateFormat X
    axisFormat %s
section baseline
no_agent (377.07 µs) : 358, 397
.   : milestone, 377,
iast (508.068 µs) : 487, 530
.   : milestone, 508,
iast_FULL (737.896 µs) : 716, 760
.   : milestone, 738,
iast_GLOBAL (556.884 µs) : 534, 579
.   : milestone, 557,
iast_HARDCODED_SECRET_DISABLED (506.351 µs) : 484, 529
.   : milestone, 506,
iast_INACTIVE (457.25 µs) : 435, 479
.   : milestone, 457,
iast_TELEMETRY_OFF (493.415 µs) : 471, 516
.   : milestone, 493,
tracing (455.249 µs) : 434, 476
.   : milestone, 455,
section candidate
no_agent (376.016 µs) : 356, 396
.   : milestone, 376,
iast (499.651 µs) : 478, 521
.   : milestone, 500,
iast_FULL (738.0 µs) : 716, 760
.   : milestone, 738,
iast_GLOBAL (549.246 µs) : 528, 571
.   : milestone, 549,
iast_HARDCODED_SECRET_DISABLED (509.28 µs) : 487, 532
.   : milestone, 509,
iast_INACTIVE (458.829 µs) : 437, 480
.   : milestone, 459,
iast_TELEMETRY_OFF (496.154 µs) : 473, 520
.   : milestone, 496,
tracing (452.119 µs) : 431, 473
.   : milestone, 452,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 377.07 µs [357.527 µs, 396.612 µs] -
iast 508.068 µs [486.505 µs, 529.631 µs] 130.998 µs (34.7%)
iast_FULL 737.896 µs [715.867 µs, 759.924 µs] 360.826 µs (95.7%)
iast_GLOBAL 556.884 µs [534.46 µs, 579.308 µs] 179.815 µs (47.7%)
iast_HARDCODED_SECRET_DISABLED 506.351 µs [483.929 µs, 528.772 µs] 129.281 µs (34.3%)
iast_INACTIVE 457.25 µs [435.339 µs, 479.162 µs] 80.18 µs (21.3%)
iast_TELEMETRY_OFF 493.415 µs [470.921 µs, 515.908 µs] 116.345 µs (30.9%)
tracing 455.249 µs [434.136 µs, 476.363 µs] 78.179 µs (20.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 376.016 µs [356.266 µs, 395.766 µs] -
iast 499.651 µs [478.246 µs, 521.057 µs] 123.635 µs (32.9%)
iast_FULL 738.0 µs [716.159 µs, 759.841 µs] 361.983 µs (96.3%)
iast_GLOBAL 549.246 µs [527.525 µs, 570.967 µs] 173.23 µs (46.1%)
iast_HARDCODED_SECRET_DISABLED 509.28 µs [486.957 µs, 531.602 µs] 133.263 µs (35.4%)
iast_INACTIVE 458.829 µs [437.175 µs, 480.483 µs] 82.813 µs (22.0%)
iast_TELEMETRY_OFF 496.154 µs [472.579 µs, 519.729 µs] 120.138 µs (32.0%)
tracing 452.119 µs [431.096 µs, 473.141 µs] 76.102 µs (20.2%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/quarantined-tests-tagging
git_commit_date 1738578316 1738592916
git_commit_sha 9af5347 c80ed84
release_version 1.47.0-SNAPSHOT~9af5347cda 1.47.0-SNAPSHOT~c80ed84ecc
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1738595042 1738595042
ci_job_id 791309696 791309696
ci_pipeline_id 54719662 54719662
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-rzaamrpw-project-304-concurrent-0-iv5264yf 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-rzaamrpw-project-304-concurrent-0-iv5264yf 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~c80ed84ecc, baseline=1.47.0-SNAPSHOT~9af5347cda
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.94 s) : 14940000, 14940000
.   : milestone, 14940000,
appsec (15.321 s) : 15321000, 15321000
.   : milestone, 15321000,
iast (19.101 s) : 19101000, 19101000
.   : milestone, 19101000,
iast_GLOBAL (18.319 s) : 18319000, 18319000
.   : milestone, 18319000,
profiling (15.089 s) : 15089000, 15089000
.   : milestone, 15089000,
tracing (14.819 s) : 14819000, 14819000
.   : milestone, 14819000,
section candidate
no_agent (15.103 s) : 15103000, 15103000
.   : milestone, 15103000,
appsec (15.098 s) : 15098000, 15098000
.   : milestone, 15098000,
iast (18.511 s) : 18511000, 18511000
.   : milestone, 18511000,
iast_GLOBAL (18.185 s) : 18185000, 18185000
.   : milestone, 18185000,
profiling (15.169 s) : 15169000, 15169000
.   : milestone, 15169000,
tracing (14.886 s) : 14886000, 14886000
.   : milestone, 14886000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.94 s [14.94 s, 14.94 s] -
appsec 15.321 s [15.321 s, 15.321 s] 381.0 ms (2.6%)
iast 19.101 s [19.101 s, 19.101 s] 4.161 s (27.9%)
iast_GLOBAL 18.319 s [18.319 s, 18.319 s] 3.379 s (22.6%)
profiling 15.089 s [15.089 s, 15.089 s] 149.0 ms (1.0%)
tracing 14.819 s [14.819 s, 14.819 s] -121.0 ms (-0.8%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.103 s [15.103 s, 15.103 s] -
appsec 15.098 s [15.098 s, 15.098 s] -5.0 ms (-0.0%)
iast 18.511 s [18.511 s, 18.511 s] 3.408 s (22.6%)
iast_GLOBAL 18.185 s [18.185 s, 18.185 s] 3.082 s (20.4%)
profiling 15.169 s [15.169 s, 15.169 s] 66.0 ms (0.4%)
tracing 14.886 s [14.886 s, 14.886 s] -217.0 ms (-1.4%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~c80ed84ecc, baseline=1.47.0-SNAPSHOT~9af5347cda
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.477 ms) : 1466, 1489
.   : milestone, 1477,
appsec (2.365 ms) : 2322, 2408
.   : milestone, 2365,
iast (2.113 ms) : 2058, 2167
.   : milestone, 2113,
iast_GLOBAL (2.17 ms) : 2115, 2225
.   : milestone, 2170,
profiling (2.003 ms) : 1958, 2048
.   : milestone, 2003,
tracing (1.957 ms) : 1915, 1999
.   : milestone, 1957,
section candidate
no_agent (1.481 ms) : 1469, 1492
.   : milestone, 1481,
appsec (2.366 ms) : 2323, 2409
.   : milestone, 2366,
iast (2.129 ms) : 2074, 2183
.   : milestone, 2129,
iast_GLOBAL (2.161 ms) : 2106, 2216
.   : milestone, 2161,
profiling (1.971 ms) : 1927, 2014
.   : milestone, 1971,
tracing (1.953 ms) : 1911, 1994
.   : milestone, 1953,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.477 ms [1.466 ms, 1.489 ms] -
appsec 2.365 ms [2.322 ms, 2.408 ms] 888.009 µs (60.1%)
iast 2.113 ms [2.058 ms, 2.167 ms] 635.49 µs (43.0%)
iast_GLOBAL 2.17 ms [2.115 ms, 2.225 ms] 693.042 µs (46.9%)
profiling 2.003 ms [1.958 ms, 2.048 ms] 525.914 µs (35.6%)
tracing 1.957 ms [1.915 ms, 1.999 ms] 480.14 µs (32.5%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.481 ms [1.469 ms, 1.492 ms] -
appsec 2.366 ms [2.323 ms, 2.409 ms] 885.418 µs (59.8%)
iast 2.129 ms [2.074 ms, 2.183 ms] 648.241 µs (43.8%)
iast_GLOBAL 2.161 ms [2.106 ms, 2.216 ms] 680.369 µs (46.0%)
profiling 1.971 ms [1.927 ms, 2.014 ms] 490.126 µs (33.1%)
tracing 1.953 ms [1.911 ms, 1.994 ms] 471.937 µs (31.9%)

@nikita-tkachenko-datadog nikita-tkachenko-datadog marked this pull request as ready for review February 3, 2025 15:30
@nikita-tkachenko-datadog nikita-tkachenko-datadog merged commit 38b8211 into master Feb 3, 2025
202 of 204 checks passed
@nikita-tkachenko-datadog nikita-tkachenko-datadog deleted the nikita-tkachenko/quarantined-tests-tagging branch February 3, 2025 15:37
@github-actions github-actions bot added this to the 1.47.0 milestone Feb 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@daniel-mohedano daniel-mohedano daniel-mohedano approved these changes

@tonyredondo tonyredondo Awaiting requested review from tonyredondo tonyredondo is a code owner automatically assigned from DataDog/ci-app-libraries-java

@nayeem-kamal nayeem-kamal Awaiting requested review from nayeem-kamal nayeem-kamal is a code owner automatically assigned from DataDog/apm-java

Assignees
No one assigned
Labels
comp: ci visibility Continuous Integration Visibility type: enhancement
Projects
None yet
Milestone
1.47.0
Development

Successfully merging this pull request may close these issues.
2 participants
@nikita-tkachenko-datadog @daniel-mohedano