Parallelize CI Visibility settings requests #8299

nikita-tkachenko-datadog · 2025-01-29T11:43:32Z

What Does This Do

Obtaining CI Visibility settings from the backend is done with multiple requests:

request the list of enabled features
depending on enabled features, request skippable tests data
depending on enabled features, request known tests data
depending on enabled features, request flaky tests data
depending on enabled features, changed files data

Currently all of these requests are done sequentially.
This PR changes the logic so that all of the requests apart from the first one are done in parallel.

Motivation

Speeding up settings construction.

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: SDTEST-1483

pr-commenter · 2025-01-29T12:24:42Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/parallelize-settings-requests
git_commit_date	1738137590	1738150953
git_commit_sha	`f06d59c`	`2401bb9`
release_version	1.46.0-SNAPSHOT~f06d59c0dd	1.46.0-SNAPSHOT~2401bb9463

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1738153314	1738153314
ci_job_id	785194765	785194765
ci_pipeline_id	54301426	54301426
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-qympdhd1-project-304-concurrent-0-tyhozemx 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-qympdhd1-project-304-concurrent-0-tyhozemx 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 51 metrics, 12 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.46.0-SNAPSHOT~2401bb9463, baseline=1.46.0-SNAPSHOT~f06d59c0dd

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.039 s) : 0, 1038541
Total [baseline] (8.598 s) : 0, 8597851
Agent [candidate] (1.038 s) : 0, 1038117
Total [candidate] (8.604 s) : 0, 8603850
section iast
Agent [baseline] (1.175 s) : 0, 1175409
Total [baseline] (9.265 s) : 0, 9264827
Agent [candidate] (1.184 s) : 0, 1184098
Total [candidate] (9.27 s) : 0, 9269969
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.166 s) : 0, 1165772
Total [baseline] (9.212 s) : 0, 9211786
Agent [candidate] (1.184 s) : 0, 1184416
Total [candidate] (9.204 s) : 0, 9203508
section iast_TELEMETRY_OFF
Agent [baseline] (1.161 s) : 0, 1161470
Total [baseline] (9.244 s) : 0, 9244267
Agent [candidate] (1.162 s) : 0, 1161836
Total [candidate] (9.199 s) : 0, 9198843

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.039 s	-
Agent	iast	1.175 s	136.868 ms (13.2%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.166 s	127.231 ms (12.3%)
Agent	iast_TELEMETRY_OFF	1.161 s	122.929 ms (11.8%)
Total	tracing	8.598 s	-
Total	iast	9.265 s	666.975 ms (7.8%)
Total	iast_HARDCODED_SECRET_DISABLED	9.212 s	613.935 ms (7.1%)
Total	iast_TELEMETRY_OFF	9.244 s	646.416 ms (7.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.038 s	-
Agent	iast	1.184 s	145.981 ms (14.1%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.184 s	146.299 ms (14.1%)
Agent	iast_TELEMETRY_OFF	1.162 s	123.719 ms (11.9%)
Total	tracing	8.604 s	-
Total	iast	9.27 s	666.119 ms (7.7%)
Total	iast_HARDCODED_SECRET_DISABLED	9.204 s	599.659 ms (7.0%)
Total	iast_TELEMETRY_OFF	9.199 s	594.993 ms (6.9%)

gantt
    title insecure-bank - break down per module: candidate=1.46.0-SNAPSHOT~2401bb9463, baseline=1.46.0-SNAPSHOT~f06d59c0dd

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (714.297 ms) : 0, 714297
BytebuddyAgent [candidate] (713.928 ms) : 0, 713928
GlobalTracer [baseline] (239.365 ms) : 0, 239365
GlobalTracer [candidate] (239.776 ms) : 0, 239776
AppSec [baseline] (55.297 ms) : 0, 55297
AppSec [candidate] (55.557 ms) : 0, 55557
Remote Config [baseline] (715.96 µs) : 0, 716
Remote Config [candidate] (711.903 µs) : 0, 712
Telemetry [baseline] (13.64 ms) : 0, 13640
Telemetry [candidate] (12.931 ms) : 0, 12931
section iast
BytebuddyAgent [baseline] (838.537 ms) : 0, 838537
BytebuddyAgent [candidate] (845.242 ms) : 0, 845242
GlobalTracer [baseline] (231.888 ms) : 0, 231888
GlobalTracer [candidate] (233.157 ms) : 0, 233157
IAST [baseline] (24.355 ms) : 0, 24355
IAST [candidate] (25.17 ms) : 0, 25170
AppSec [baseline] (55.843 ms) : 0, 55843
AppSec [candidate] (55.426 ms) : 0, 55426
Remote Config [baseline] (628.949 µs) : 0, 629
Remote Config [candidate] (644.933 µs) : 0, 645
Telemetry [baseline] (8.831 ms) : 0, 8831
Telemetry [candidate] (8.98 ms) : 0, 8980
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (831.822 ms) : 0, 831822
BytebuddyAgent [candidate] (846.22 ms) : 0, 846220
GlobalTracer [baseline] (229.99 ms) : 0, 229990
GlobalTracer [candidate] (233.197 ms) : 0, 233197
IAST [baseline] (26.313 ms) : 0, 26313
IAST [candidate] (25.871 ms) : 0, 25871
AppSec [baseline] (52.957 ms) : 0, 52957
AppSec [candidate] (54.339 ms) : 0, 54339
Remote Config [baseline] (615.206 µs) : 0, 615
Remote Config [candidate] (622.877 µs) : 0, 623
Telemetry [baseline] (8.78 ms) : 0, 8780
Telemetry [candidate] (8.752 ms) : 0, 8752
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (828.839 ms) : 0, 828839
BytebuddyAgent [candidate] (828.558 ms) : 0, 828558
GlobalTracer [baseline] (229.789 ms) : 0, 229789
GlobalTracer [candidate] (229.705 ms) : 0, 229705
IAST [baseline] (26.377 ms) : 0, 26377
IAST [candidate] (25.839 ms) : 0, 25839
AppSec [baseline] (52.022 ms) : 0, 52022
AppSec [candidate] (53.289 ms) : 0, 53289
Remote Config [baseline] (613.249 µs) : 0, 613
Remote Config [candidate] (630.514 µs) : 0, 631
Telemetry [baseline] (8.522 ms) : 0, 8522
Telemetry [candidate] (8.619 ms) : 0, 8619

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.46.0-SNAPSHOT~2401bb9463, baseline=1.46.0-SNAPSHOT~f06d59c0dd

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.035 s) : 0, 1035412
Total [baseline] (10.417 s) : 0, 10417369
Agent [candidate] (1.037 s) : 0, 1036731
Total [candidate] (10.442 s) : 0, 10442150
section appsec
Agent [baseline] (1.178 s) : 0, 1177762
Total [baseline] (10.744 s) : 0, 10743710
Agent [candidate] (1.184 s) : 0, 1183671
Total [candidate] (10.759 s) : 0, 10758752
section iast
Agent [baseline] (1.176 s) : 0, 1176185
Total [baseline] (11.007 s) : 0, 11006869
Agent [candidate] (1.176 s) : 0, 1175926
Total [candidate] (10.939 s) : 0, 10939156
section profiling
Agent [baseline] (1.258 s) : 0, 1257612
Total [baseline] (10.849 s) : 0, 10849149
Agent [candidate] (1.265 s) : 0, 1265422
Total [candidate] (10.928 s) : 0, 10928069

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.035 s	-
Agent	appsec	1.178 s	142.35 ms (13.7%)
Agent	iast	1.176 s	140.774 ms (13.6%)
Agent	profiling	1.258 s	222.2 ms (21.5%)
Total	tracing	10.417 s	-
Total	appsec	10.744 s	326.341 ms (3.1%)
Total	iast	11.007 s	589.5 ms (5.7%)
Total	profiling	10.849 s	431.78 ms (4.1%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.037 s	-
Agent	appsec	1.184 s	146.94 ms (14.2%)
Agent	iast	1.176 s	139.195 ms (13.4%)
Agent	profiling	1.265 s	228.691 ms (22.1%)
Total	tracing	10.442 s	-
Total	appsec	10.759 s	316.602 ms (3.0%)
Total	iast	10.939 s	497.006 ms (4.8%)
Total	profiling	10.928 s	485.919 ms (4.7%)

gantt
    title petclinic - break down per module: candidate=1.46.0-SNAPSHOT~2401bb9463, baseline=1.46.0-SNAPSHOT~f06d59c0dd

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (713.805 ms) : 0, 713805
BytebuddyAgent [candidate] (714.887 ms) : 0, 714887
GlobalTracer [baseline] (238.967 ms) : 0, 238967
GlobalTracer [candidate] (239.972 ms) : 0, 239972
AppSec [baseline] (55.209 ms) : 0, 55209
AppSec [candidate] (55.148 ms) : 0, 55148
Remote Config [baseline] (719.217 µs) : 0, 719
Remote Config [candidate] (719.016 µs) : 0, 719
Telemetry [baseline] (11.566 ms) : 0, 11566
Telemetry [candidate] (10.806 ms) : 0, 10806
section appsec
BytebuddyAgent [baseline] (730.574 ms) : 0, 730574
BytebuddyAgent [candidate] (733.462 ms) : 0, 733462
GlobalTracer [baseline] (236.982 ms) : 0, 236982
GlobalTracer [candidate] (238.259 ms) : 0, 238259
IAST [baseline] (21.529 ms) : 0, 21529
IAST [candidate] (21.62 ms) : 0, 21620
AppSec [baseline] (175.252 ms) : 0, 175252
AppSec [candidate] (176.788 ms) : 0, 176788
Remote Config [baseline] (655.624 µs) : 0, 656
Remote Config [candidate] (660.228 µs) : 0, 660
Telemetry [baseline] (8.281 ms) : 0, 8281
Telemetry [candidate] (8.358 ms) : 0, 8358
section iast
BytebuddyAgent [baseline] (839.13 ms) : 0, 839130
BytebuddyAgent [candidate] (839.164 ms) : 0, 839164
GlobalTracer [baseline] (232.388 ms) : 0, 232388
GlobalTracer [candidate] (231.955 ms) : 0, 231955
IAST [baseline] (26.647 ms) : 0, 26647
IAST [candidate] (26.489 ms) : 0, 26489
AppSec [baseline] (53.391 ms) : 0, 53391
AppSec [candidate] (53.575 ms) : 0, 53575
Remote Config [baseline] (613.921 µs) : 0, 614
Remote Config [candidate] (617.118 µs) : 0, 617
Telemetry [baseline] (8.708 ms) : 0, 8708
Telemetry [candidate] (8.796 ms) : 0, 8796
section profiling
BytebuddyAgent [baseline] (704.947 ms) : 0, 704947
BytebuddyAgent [candidate] (709.421 ms) : 0, 709421
GlobalTracer [baseline] (350.391 ms) : 0, 350391
GlobalTracer [candidate] (352.202 ms) : 0, 352202
AppSec [baseline] (54.77 ms) : 0, 54770
AppSec [candidate] (54.814 ms) : 0, 54814
Remote Config [baseline] (693.777 µs) : 0, 694
Remote Config [candidate] (711.056 µs) : 0, 711
Telemetry [baseline] (8.891 ms) : 0, 8891
Telemetry [candidate] (8.989 ms) : 0, 8989
ProfilingAgent [baseline] (95.682 ms) : 0, 95682
ProfilingAgent [candidate] (96.638 ms) : 0, 96638
Profiling [baseline] (95.707 ms) : 0, 95707
Profiling [candidate] (96.662 ms) : 0, 96662

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-01-29T11:54:37	2025-01-29T12:01:39
git_branch	master	nikita-tkachenko/parallelize-settings-requests
git_commit_date	1738137590	1738150953
git_commit_sha	`f06d59c`	`2401bb9`
release_version	1.46.0-SNAPSHOT~f06d59c0dd	1.46.0-SNAPSHOT~2401bb9463
start_time	2025-01-29T11:54:24	2025-01-29T12:01:25

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1738152456	1738152456
ci_job_id	785194766	785194766
ci_pipeline_id	54301426	54301426
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-9roxoyme-project-304-concurrent-1-r1y35pd0 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-9roxoyme-project-304-concurrent-1-r1y35pd0 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 18 unstable metrics.

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~2401bb9463, baseline=1.46.0-SNAPSHOT~f06d59c0dd
    dateFormat X
    axisFormat %s
section baseline
no_agent (374.599 µs) : 355, 394
.   : milestone, 375,
iast (505.767 µs) : 483, 529
.   : milestone, 506,
iast_FULL (737.313 µs) : 715, 759
.   : milestone, 737,
iast_GLOBAL (546.546 µs) : 525, 568
.   : milestone, 547,
iast_HARDCODED_SECRET_DISABLED (503.892 µs) : 481, 527
.   : milestone, 504,
iast_INACTIVE (457.857 µs) : 436, 480
.   : milestone, 458,
iast_TELEMETRY_OFF (495.66 µs) : 473, 519
.   : milestone, 496,
tracing (453.499 µs) : 431, 476
.   : milestone, 453,
section candidate
no_agent (377.446 µs) : 358, 397
.   : milestone, 377,
iast (508.785 µs) : 486, 532
.   : milestone, 509,
iast_FULL (742.139 µs) : 720, 764
.   : milestone, 742,
iast_GLOBAL (558.897 µs) : 535, 583
.   : milestone, 559,
iast_HARDCODED_SECRET_DISABLED (510.611 µs) : 487, 534
.   : milestone, 511,
iast_INACTIVE (457.303 µs) : 436, 479
.   : milestone, 457,
iast_TELEMETRY_OFF (492.725 µs) : 470, 516
.   : milestone, 493,
tracing (451.443 µs) : 430, 473
.   : milestone, 451,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	374.599 µs [354.738 µs, 394.46 µs]	-
iast	505.767 µs [482.585 µs, 528.95 µs]	131.168 µs (35.0%)
iast_FULL	737.313 µs [715.459 µs, 759.166 µs]	362.713 µs (96.8%)
iast_GLOBAL	546.546 µs [525.173 µs, 567.918 µs]	171.947 µs (45.9%)
iast_HARDCODED_SECRET_DISABLED	503.892 µs [481.027 µs, 526.758 µs]	129.293 µs (34.5%)
iast_INACTIVE	457.857 µs [436.017 µs, 479.697 µs]	83.258 µs (22.2%)
iast_TELEMETRY_OFF	495.66 µs [472.506 µs, 518.813 µs]	121.061 µs (32.3%)
tracing	453.499 µs [431.367 µs, 475.63 µs]	78.9 µs (21.1%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	377.446 µs [357.845 µs, 397.047 µs]	-
iast	508.785 µs [485.532 µs, 532.038 µs]	131.339 µs (34.8%)
iast_FULL	742.139 µs [720.24 µs, 764.037 µs]	364.693 µs (96.6%)
iast_GLOBAL	558.897 µs [534.655 µs, 583.138 µs]	181.451 µs (48.1%)
iast_HARDCODED_SECRET_DISABLED	510.611 µs [487.292 µs, 533.931 µs]	133.165 µs (35.3%)
iast_INACTIVE	457.303 µs [436.046 µs, 478.56 µs]	79.857 µs (21.2%)
iast_TELEMETRY_OFF	492.725 µs [469.733 µs, 515.717 µs]	115.279 µs (30.5%)
tracing	451.443 µs [430.088 µs, 472.799 µs]	73.997 µs (19.6%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~2401bb9463, baseline=1.46.0-SNAPSHOT~f06d59c0dd
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.358 ms) : 1338, 1377
.   : milestone, 1358,
appsec (1.741 ms) : 1718, 1764
.   : milestone, 1741,
appsec_no_iast (1.736 ms) : 1711, 1761
.   : milestone, 1736,
iast (1.497 ms) : 1473, 1522
.   : milestone, 1497,
profiling (1.491 ms) : 1467, 1515
.   : milestone, 1491,
tracing (1.499 ms) : 1471, 1527
.   : milestone, 1499,
section candidate
no_agent (1.358 ms) : 1339, 1378
.   : milestone, 1358,
appsec (1.734 ms) : 1710, 1759
.   : milestone, 1734,
appsec_no_iast (1.769 ms) : 1746, 1793
.   : milestone, 1769,
iast (1.507 ms) : 1483, 1531
.   : milestone, 1507,
profiling (1.51 ms) : 1488, 1533
.   : milestone, 1510,
tracing (1.482 ms) : 1456, 1509
.   : milestone, 1482,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.358 ms [1.338 ms, 1.377 ms]	-
appsec	1.741 ms [1.718 ms, 1.764 ms]	383.371 µs (28.2%)
appsec_no_iast	1.736 ms [1.711 ms, 1.761 ms]	378.631 µs (27.9%)
iast	1.497 ms [1.473 ms, 1.522 ms]	139.844 µs (10.3%)
profiling	1.491 ms [1.467 ms, 1.515 ms]	133.555 µs (9.8%)
tracing	1.499 ms [1.471 ms, 1.527 ms]	141.581 µs (10.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.358 ms [1.339 ms, 1.378 ms]	-
appsec	1.734 ms [1.71 ms, 1.759 ms]	376.161 µs (27.7%)
appsec_no_iast	1.769 ms [1.746 ms, 1.793 ms]	411.107 µs (30.3%)
iast	1.507 ms [1.483 ms, 1.531 ms]	149.05 µs (11.0%)
profiling	1.51 ms [1.488 ms, 1.533 ms]	152.105 µs (11.2%)
tracing	1.482 ms [1.456 ms, 1.509 ms]	124.103 µs (9.1%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/parallelize-settings-requests
git_commit_date	1738137590	1738150953
git_commit_sha	`f06d59c`	`2401bb9`
release_version	1.46.0-SNAPSHOT~f06d59c0dd	1.46.0-SNAPSHOT~2401bb9463

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1738153003	1738153003
ci_job_id	785194767	785194767
ci_pipeline_id	54301426	54301426
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-ws5zmchx-project-304-concurrent-0-r1hk785r 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-ws5zmchx-project-304-concurrent-0-r1hk785r 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~2401bb9463, baseline=1.46.0-SNAPSHOT~f06d59c0dd
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.943 s) : 14943000, 14943000
.   : milestone, 14943000,
appsec (15.084 s) : 15084000, 15084000
.   : milestone, 15084000,
iast (18.802 s) : 18802000, 18802000
.   : milestone, 18802000,
iast_GLOBAL (18.229 s) : 18229000, 18229000
.   : milestone, 18229000,
profiling (15.057 s) : 15057000, 15057000
.   : milestone, 15057000,
tracing (14.629 s) : 14629000, 14629000
.   : milestone, 14629000,
section candidate
no_agent (14.716 s) : 14716000, 14716000
.   : milestone, 14716000,
appsec (14.838 s) : 14838000, 14838000
.   : milestone, 14838000,
iast (18.818 s) : 18818000, 18818000
.   : milestone, 18818000,
iast_GLOBAL (18.181 s) : 18181000, 18181000
.   : milestone, 18181000,
profiling (15.002 s) : 15002000, 15002000
.   : milestone, 15002000,
tracing (14.892 s) : 14892000, 14892000
.   : milestone, 14892000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.943 s [14.943 s, 14.943 s]	-
appsec	15.084 s [15.084 s, 15.084 s]	141.0 ms (0.9%)
iast	18.802 s [18.802 s, 18.802 s]	3.859 s (25.8%)
iast_GLOBAL	18.229 s [18.229 s, 18.229 s]	3.286 s (22.0%)
profiling	15.057 s [15.057 s, 15.057 s]	114.0 ms (0.8%)
tracing	14.629 s [14.629 s, 14.629 s]	-314.0 ms (-2.1%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.716 s [14.716 s, 14.716 s]	-
appsec	14.838 s [14.838 s, 14.838 s]	122.0 ms (0.8%)
iast	18.818 s [18.818 s, 18.818 s]	4.102 s (27.9%)
iast_GLOBAL	18.181 s [18.181 s, 18.181 s]	3.465 s (23.5%)
profiling	15.002 s [15.002 s, 15.002 s]	286.0 ms (1.9%)
tracing	14.892 s [14.892 s, 14.892 s]	176.0 ms (1.2%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~2401bb9463, baseline=1.46.0-SNAPSHOT~f06d59c0dd
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.458 ms) : 1447, 1469
.   : milestone, 1458,
appsec (2.35 ms) : 2306, 2393
.   : milestone, 2350,
iast (2.092 ms) : 2037, 2146
.   : milestone, 2092,
iast_GLOBAL (2.138 ms) : 2083, 2194
.   : milestone, 2138,
profiling (2.465 ms) : 2274, 2656
.   : milestone, 2465,
tracing (1.921 ms) : 1880, 1963
.   : milestone, 1921,
section candidate
no_agent (1.464 ms) : 1452, 1475
.   : milestone, 1464,
appsec (2.35 ms) : 2307, 2393
.   : milestone, 2350,
iast (2.096 ms) : 2041, 2151
.   : milestone, 2096,
iast_GLOBAL (2.149 ms) : 2094, 2204
.   : milestone, 2149,
profiling (1.95 ms) : 1906, 1993
.   : milestone, 1950,
tracing (1.93 ms) : 1888, 1972
.   : milestone, 1930,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.458 ms [1.447 ms, 1.469 ms]	-
appsec	2.35 ms [2.306 ms, 2.393 ms]	891.581 µs (61.2%)
iast	2.092 ms [2.037 ms, 2.146 ms]	633.848 µs (43.5%)
iast_GLOBAL	2.138 ms [2.083 ms, 2.194 ms]	680.419 µs (46.7%)
profiling	2.465 ms [2.274 ms, 2.656 ms]	1.007 ms (69.1%)
tracing	1.921 ms [1.88 ms, 1.963 ms]	463.225 µs (31.8%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.464 ms [1.452 ms, 1.475 ms]	-
appsec	2.35 ms [2.307 ms, 2.393 ms]	886.105 µs (60.5%)
iast	2.096 ms [2.041 ms, 2.151 ms]	632.336 µs (43.2%)
iast_GLOBAL	2.149 ms [2.094 ms, 2.204 ms]	685.451 µs (46.8%)
profiling	1.95 ms [1.906 ms, 1.993 ms]	485.872 µs (33.2%)
tracing	1.93 ms [1.888 ms, 1.972 ms]	466.053 µs (31.8%)

| Package | Type | Package file | Manager | Update | Change | |---|---|---|---|---|---| | [com.google.cloud:google-cloud-datastore](https://github.com/googleapis/java-datastore) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `2.25.4` -> `2.26.0` | | [com.datadoghq:dd-trace-api](https://github.com/datadog/dd-trace-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.45.2` -> `1.46.0` | | [com.datadoghq:dd-trace-ot](https://github.com/datadog/dd-trace-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.45.2` -> `1.46.0` | | [software.amazon.awssdk:sdk-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | | [software.amazon.awssdk:dynamodb-enhanced](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | | [software.amazon.awssdk:dynamodb](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | | [software.amazon.awssdk:aws-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | | [software.amazon.awssdk:bom](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | | [software.amazon.awssdk:auth](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | --- ### Release Notes <details> <summary>googleapis/java-datastore (com.google.cloud:google-cloud-datastore)</summary> ### [`v2.26.0`](https://github.com/googleapis/java-datastore/blob/HEAD/CHANGELOG.md#2260-2025-01-29) ##### Features - Add firestoreInDatastoreMode for datastore emulator ([#1698](googleapis/java-datastore#1698)) ([50f106d](googleapis/java-datastore@50f106d)) ##### Dependencies - Update dependency com.google.cloud:sdk-platform-java-config to v3.42.0 ([#1725](googleapis/java-datastore#1725)) ([1cbaf22](googleapis/java-datastore@1cbaf22)) </details> <details> <summary>datadog/dd-trace-java (com.datadoghq:dd-trace-api)</summary> ### [`v1.46.0`](https://github.com/DataDog/dd-trace-java/releases/tag/v1.46.0): 1.46.0 ##### Breaking Changes > \[!WARNING] > jnr-unixsocket is now an external dependency of dd-trace-ot and must be included when deploying dd-trace-ot. > \[!NOTE] > The API `TracerScope.setAsync(boolean)`, used to manually control asynchronous span propagation, does no more apply to the scope instance but to the active span scope. ##### Components ##### Application Security Management (IAST) - 🐛 Fix String.replace instrumentation for IAST ([#8281](DataDog/dd-trace-java#8281) - [@Mariovido](https://github.com/Mariovido)) - ✨ Apply the standard nomenclature to the stacktrace configs ([#8244](DataDog/dd-trace-java#8244) - [@jandro996](https://github.com/jandro996)) - 🐛 Exclude false positive weak randomness ([#8232](DataDog/dd-trace-java#8232) - [@jandro996](https://github.com/jandro996)) - ✨ Propagation of translateEscapes of String class ([#8186](DataDog/dd-trace-java#8186) - [@sezen-datadog](https://github.com/sezen-datadog)) - ✨ Add security control metrics ([#8175](DataDog/dd-trace-java#8175) - [@jandro996](https://github.com/jandro996)) - ✨ Increase IAST propagation to StringBuffer setLength ([#8128](DataDog/dd-trace-java#8128) - [@Mariovido](https://github.com/Mariovido)) - ✨ Add IAST taint tracking for DB values ([#8072](DataDog/dd-trace-java#8072) - [@Mariovido](https://github.com/Mariovido)) ##### Application Security Management (WAF) - 🐛 Prevents a NPE when there is no subscriber for user events ([#8258](DataDog/dd-trace-java#8258) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Apply the standard nomenclature to the stacktrace configs ([#8244](DataDog/dd-trace-java#8244) - [@jandro996](https://github.com/jandro996)) - 🐛 Ensure cached subscriptions are cleared on reconfiguration via RC ([#8229](DataDog/dd-trace-java#8229) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Add support for session tracking in Vertx ([#8167](DataDog/dd-trace-java#8167) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Create span tag: \_dd.appsec.rasp.timeout ([#8269](DataDog/dd-trace-java#8269) - [@Mariovido](https://github.com/Mariovido)) ##### Build & Tooling - 🐛 Ensure shaded helpers have unique names when injected into class-loaders ([#8192](DataDog/dd-trace-java#8192) - [@mcculls](https://github.com/mcculls)) ##### Configuration at Runtime - 🐛 Remove filtering of `DD_SERVICE` and `DD_ENV` from the tracer ([#8176](DataDog/dd-trace-java#8176) - [@mhlidd](https://github.com/mhlidd)) ##### Continuous Integration Visibility - 🧹 Generalize TestRetryPolicy to TestExecutionPolicy ([#8302](DataDog/dd-trace-java#8302) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🧹 Parallelize CI Visibility settings requests ([#8299](DataDog/dd-trace-java#8299) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🧹 Generalize test retry logic ([#8289](DataDog/dd-trace-java#8289) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🧹 Generalize tests skipping logic ([#8288](DataDog/dd-trace-java#8288) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🧹 Remove skip and shouldBeSkipped methods from TestEventsHandler in favor of isSkippable ([#8286](DataDog/dd-trace-java#8286) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨⚡ Optimize Git repository information computation ([#8270](DataDog/dd-trace-java#8270) - [@dougqh](https://github.com/dougqh)) - ✨ Always request known tests from the backend ([#8268](DataDog/dd-trace-java#8268) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Fix NPE when trying to get retry analyzer in Test NG ([#8253](DataDog/dd-trace-java#8253) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🐛 Set test framework and test framework version tags atomically ([#8252](DataDog/dd-trace-java#8252) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Add debug logging to Android Gradle module layout logic ([#8251](DataDog/dd-trace-java#8251) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🐛 Fix source and destination folders computation for Android Gradle projects ([#8190](DataDog/dd-trace-java#8190) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Add basic Scala Weaver sbt support ([#8189](DataDog/dd-trace-java#8189) - [@daniel-mohedano](https://github.com/daniel-mohedano)) - ✨ Implement impacted tests detection ([#8188](DataDog/dd-trace-java#8188) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) ##### Data Streams Monitoring - ✨ Change hash computation for protobuf to better represent impacting changes + save proto number in schema ([#8201](DataDog/dd-trace-java#8201) - [@vandonr](https://github.com/vandonr)) ##### Database Monitoring - Add peer service tag in dbm sql commenter ([#7913](DataDog/dd-trace-java#7913) - [@jordan-wong](https://github.com/jordan-wong)) ##### Dynamic Instrumentation - ✨ Add support for SymDB to scan directories ([#8306](DataDog/dd-trace-java#8306) - [@jpbempel](https://github.com/jpbempel)) - ✨ Add SymDB report for any jar scanning failures ([#8300](DataDog/dd-trace-java#8300) - [@jpbempel](https://github.com/jpbempel)) - ✨ Use two budgets depending on type ([#8283](DataDog/dd-trace-java#8283) - [@evanchooly](https://github.com/evanchooly)) - ✨ Institute a 10 snapshot per probe per trace budget ([#8277](DataDog/dd-trace-java#8277) - [@evanchooly](https://github.com/evanchooly)) - 🐛 Avoid double snapshots for Exception Replay ([#8273](DataDog/dd-trace-java#8273) - [@jpbempel](https://github.com/jpbempel)) - ✨ Simplify code origins. Separate out snapshot generation. ([#8263](DataDog/dd-trace-java#8263) - [@evanchooly](https://github.com/evanchooly)) - ✨ Add Exception probe custom instrumentation ([#8230](DataDog/dd-trace-java#8230) - [@jpbempel](https://github.com/jpbempel)) - ✨ Enhance log probes to honor debug session tags ([#8215](DataDog/dd-trace-java#8215) - [@evanchooly](https://github.com/evanchooly)) - 🐛 Don't redact env tokens from debugger probe snapshots ([#8211](DataDog/dd-trace-java#8211) - [@watson](https://github.com/watson)) - ✨⚡ Move Trace/SpanId capture at commit time ([#8184](DataDog/dd-trace-java#8184) - [@jpbempel](https://github.com/jpbempel)) - 🐛 Capture values at entry for method probe ([#8169](DataDog/dd-trace-java#8169) - [@jpbempel](https://github.com/jpbempel)) ##### JMX fetch - 🐛 Mute JMXFetch Shutdown in progress error ([#8068](DataDog/dd-trace-java#8068) - [@ygree](https://github.com/ygree)) ##### OpenTracing - ⚠️🧹 Make jnr-unixsocket an explicit dependency of dd-trace-ot ([#8307](DataDog/dd-trace-java#8307) - [@mcculls](https://github.com/mcculls)) ##### Profiling - 🐛 Avoid unsupported API call for creating folders on windows ([#8304](DataDog/dd-trace-java#8304) - [@jbachorik](https://github.com/jbachorik)) - ✨ Tag profiles for serverless ([#8279](DataDog/dd-trace-java#8279) - [@jbachorik](https://github.com/jbachorik)) - ✨ add queue type and length to queue events ([#8242](DataDog/dd-trace-java#8242) - [@richardstartin](https://github.com/richardstartin)) - 🐛 TempLocationManager Fixes and Improvements ([#8191](DataDog/dd-trace-java#8191) - [@jbachorik](https://github.com/jbachorik)) - ✨ Bump ddprof to 1.18.0 ([#8173](DataDog/dd-trace-java#8173) - [@jbachorik](https://github.com/jbachorik)) - ✨ Report profiler initialization and configuration errors to telemetry ([#8171](DataDog/dd-trace-java#8171) - [@jbachorik](https://github.com/jbachorik)) ##### Telemetry - ✨ Add pending traces report in tracer flares ([#8053](DataDog/dd-trace-java#8053) - [@mhlidd](https://github.com/mhlidd)) ##### Testing - ✨ Test http server requests in parallel ([#8222](DataDog/dd-trace-java#8222) - [@amarziali](https://github.com/amarziali)) ##### Trace context propagation - ✨ Add non default propagator registration ([#8310](DataDog/dd-trace-java#8310) - [@PerfectSlayer](https://github.com/PerfectSlayer)) ##### Tracer core - ✨ Probe for existence of IBMSASL or ACCP security providers ([#8276](DataDog/dd-trace-java#8276) - [@mcculls](https://github.com/mcculls)) - ✨⚡ Overhead improvement to agent feedback based sampling ([#8265](DataDog/dd-trace-java#8265) - [@dougqh](https://github.com/dougqh)) - 🧹 Move async propagation API from scope to tracer ([#8231](DataDog/dd-trace-java#8231) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - ✨ Introduce context propagation API ([#8161](DataDog/dd-trace-java#8161) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - ✨🧪 Use env-entry to add tags per webapp deployment ([#8138](DataDog/dd-trace-java#8138) - [@amarziali](https://github.com/amarziali)) - ✨ Introduce context helpers API ([#8134](DataDog/dd-trace-java#8134) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - ✨ Support IPv6 values for `DD_AGENT_HOST` and `DD_TRACE_AGENT_URL` ([#7984](DataDog/dd-trace-java#7984) - [@mhlidd](https://github.com/mhlidd)) ##### Instrumentations ##### Apache HttpComponents - 🐛 Properly finish spans and support latest apache httpclient5 ([#8272](DataDog/dd-trace-java#8272) - [@amarziali](https://github.com/amarziali)) ##### AWS Lambda instrumentation - 🐛 Properly capture lambda payloads for all handler types. ([#8264](DataDog/dd-trace-java#8264) - [@purple4reina](https://github.com/purple4reina)) ##### AWS S3 instrumentation - 💡 Create S3 instrumentation + add span pointers ([#8075](DataDog/dd-trace-java#8075) - [@nhulston](https://github.com/nhulston)) ##### AWS SDK instrumentation - 🐛 Revert "Add avoid double instrumenting lambda non-streaming handlers." ([#8247](DataDog/dd-trace-java#8247) - [@nhulston](https://github.com/nhulston)) ##### Cassandra - ✨ Allow extracting keyspace from statement result ([#8239](DataDog/dd-trace-java#8239) - [@amarziali](https://github.com/amarziali)) ##### Core Java language instrumentation - ✨ Propagation of translateEscapes of String class ([#8186](DataDog/dd-trace-java#8186) - [@sezen-datadog](https://github.com/sezen-datadog)) ##### Eclipse Vert.x instrumentation - 🐛 Fix vertx worker propagation and error handling ([#8237](DataDog/dd-trace-java#8237) - [@amarziali](https://github.com/amarziali)) - ✨ Support vertx 5 ([#8220](DataDog/dd-trace-java#8220) - [@amarziali](https://github.com/amarziali)) - ✨ Add support for session tracking in Vertx ([#8167](DataDog/dd-trace-java#8167) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) ##### Kafka instrumentation - 🐛 Prevent possible NPE calculating Kafka record header size ([#8292](DataDog/dd-trace-java#8292) - [@ygree](https://github.com/ygree)) ##### Mule instrumentation - 🐛 Fix crash using Mule with JPMS ([#8187](DataDog/dd-trace-java#8187) - [@amarziali](https://github.com/amarziali)) ##### Protocol Buffer instrumentation - ✨ Change hash computation for protobuf to better represent impacting changes + save proto number in schema ([#8201](DataDog/dd-trace-java#8201) - [@vandonr](https://github.com/vandonr)) ##### Spring instrumentation - 🐛 Preserve getQualifier from spring scheduling runnables ([#8293](DataDog/dd-trace-java#8293) - [@amarziali](https://github.com/amarziali)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 6pm every weekday,before 2am every weekday" in timezone Australia/Melbourne, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). GitOrigin-RevId: bb09d47e4eed77a003f630273b4d0a84003eb899

Parallelize CI Visibility settings requests

2401bb9

nikita-tkachenko-datadog added type: refactoring comp: ci visibility Continuous Integration Visibility labels Jan 29, 2025

nikita-tkachenko-datadog marked this pull request as ready for review January 29, 2025 12:57

nikita-tkachenko-datadog requested a review from a team as a code owner January 29, 2025 12:57

nikita-tkachenko-datadog requested review from juan-fernandez and daniel-mohedano January 29, 2025 12:57

daniel-mohedano approved these changes Jan 29, 2025

View reviewed changes

nikita-tkachenko-datadog merged commit e078e8a into master Jan 29, 2025
166 of 200 checks passed

nikita-tkachenko-datadog deleted the nikita-tkachenko/parallelize-settings-requests branch January 29, 2025 13:23

github-actions bot added this to the 1.46.0 milestone Jan 29, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Parallelize CI Visibility settings requests #8299

Parallelize CI Visibility settings requests #8299

nikita-tkachenko-datadog commented Jan 29, 2025 •

edited by jira bot

Loading

pr-commenter bot commented Jan 29, 2025

Parallelize CI Visibility settings requests #8299

Parallelize CI Visibility settings requests #8299

Conversation

nikita-tkachenko-datadog commented Jan 29, 2025 • edited by jira bot Loading

What Does This Do

Motivation

Contributor Checklist

pr-commenter bot commented Jan 29, 2025

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

nikita-tkachenko-datadog commented Jan 29, 2025 •

edited by jira bot

Loading