Set test framework and test framework version tags atomically #8252

nikita-tkachenko-datadog · 2025-01-20T12:51:14Z

What Does This Do

Updates the logic that sets test.framework and test.framework_version tags to do it atomically.

Motivation

When test framework name and version are propagated from test to test suite span, some merging is done.
It is possible for multiple tests from the same suite to be executed in parallel, which is why the two tags need to be set atomically.

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

pr-commenter · 2025-01-20T13:30:41Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/test-framework-tags-sync
git_commit_date	1737645585	1737649173
git_commit_sha	`991929d`	`32ad019`
release_version	1.46.0-SNAPSHOT~991929d244	1.46.0-SNAPSHOT~32ad019e3e

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1737651498	1737651498
ci_job_id	778053826	778053826
ci_pipeline_id	53816971	53816971
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-fgow3nxt-project-304-concurrent-0-y6ntbqyq 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-fgow3nxt-project-304-concurrent-0-y6ntbqyq 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 59 metrics, 4 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.46.0-SNAPSHOT~32ad019e3e, baseline=1.46.0-SNAPSHOT~991929d244

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.059 s) : 0, 1058753
Total [baseline] (8.682 s) : 0, 8682024
Agent [candidate] (1.076 s) : 0, 1075876
Total [candidate] (8.684 s) : 0, 8683922
section iast
Agent [baseline] (1.181 s) : 0, 1180691
Total [baseline] (9.203 s) : 0, 9202865
Agent [candidate] (1.195 s) : 0, 1194868
Total [candidate] (9.252 s) : 0, 9251913
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.191 s) : 0, 1190767
Total [baseline] (9.187 s) : 0, 9186752
Agent [candidate] (1.185 s) : 0, 1185480
Total [candidate] (9.2 s) : 0, 9200171
section iast_TELEMETRY_OFF
Agent [baseline] (1.178 s) : 0, 1178078
Total [baseline] (9.202 s) : 0, 9201734
Agent [candidate] (1.181 s) : 0, 1181403
Total [candidate] (9.232 s) : 0, 9232178

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.059 s	-
Agent	iast	1.181 s	121.939 ms (11.5%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.191 s	132.014 ms (12.5%)
Agent	iast_TELEMETRY_OFF	1.178 s	119.325 ms (11.3%)
Total	tracing	8.682 s	-
Total	iast	9.203 s	520.841 ms (6.0%)
Total	iast_HARDCODED_SECRET_DISABLED	9.187 s	504.728 ms (5.8%)
Total	iast_TELEMETRY_OFF	9.202 s	519.71 ms (6.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.076 s	-
Agent	iast	1.195 s	118.992 ms (11.1%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.185 s	109.605 ms (10.2%)
Agent	iast_TELEMETRY_OFF	1.181 s	105.527 ms (9.8%)
Total	tracing	8.684 s	-
Total	iast	9.252 s	567.991 ms (6.5%)
Total	iast_HARDCODED_SECRET_DISABLED	9.2 s	516.249 ms (5.9%)
Total	iast_TELEMETRY_OFF	9.232 s	548.256 ms (6.3%)

gantt
    title insecure-bank - break down per module: candidate=1.46.0-SNAPSHOT~32ad019e3e, baseline=1.46.0-SNAPSHOT~991929d244

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (715.808 ms) : 0, 715808
BytebuddyAgent [candidate] (728.704 ms) : 0, 728704
GlobalTracer [baseline] (256.47 ms) : 0, 256470
GlobalTracer [candidate] (260.168 ms) : 0, 260168
AppSec [baseline] (55.78 ms) : 0, 55780
AppSec [candidate] (57.072 ms) : 0, 57072
Remote Config [baseline] (753.147 µs) : 0, 753
Remote Config [candidate] (775.241 µs) : 0, 775
Telemetry [baseline] (14.98 ms) : 0, 14980
Telemetry [candidate] (13.938 ms) : 0, 13938
section iast
BytebuddyAgent [baseline] (830.671 ms) : 0, 830671
BytebuddyAgent [candidate] (840.589 ms) : 0, 840589
GlobalTracer [baseline] (246.229 ms) : 0, 246229
GlobalTracer [candidate] (249.14 ms) : 0, 249140
AppSec [baseline] (57.997 ms) : 0, 57997
AppSec [candidate] (58.486 ms) : 0, 58486
Remote Config [baseline] (648.477 µs) : 0, 648
Remote Config [candidate] (674.448 µs) : 0, 674
Telemetry [baseline] (8.696 ms) : 0, 8696
Telemetry [candidate] (8.929 ms) : 0, 8929
IAST [baseline] (21.5 ms) : 0, 21500
IAST [candidate] (21.803 ms) : 0, 21803
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (838.259 ms) : 0, 838259
BytebuddyAgent [candidate] (833.793 ms) : 0, 833793
GlobalTracer [baseline] (248.033 ms) : 0, 248033
GlobalTracer [candidate] (247.024 ms) : 0, 247024
AppSec [baseline] (58.301 ms) : 0, 58301
AppSec [candidate] (58.321 ms) : 0, 58321
Remote Config [baseline] (708.007 µs) : 0, 708
Remote Config [candidate] (675.451 µs) : 0, 675
Telemetry [baseline] (8.826 ms) : 0, 8826
Telemetry [candidate] (8.908 ms) : 0, 8908
IAST [baseline] (21.559 ms) : 0, 21559
IAST [candidate] (21.64 ms) : 0, 21640
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (828.903 ms) : 0, 828903
BytebuddyAgent [candidate] (830.692 ms) : 0, 830692
GlobalTracer [baseline] (246.074 ms) : 0, 246074
GlobalTracer [candidate] (247.045 ms) : 0, 247045
AppSec [baseline] (57.913 ms) : 0, 57913
AppSec [candidate] (57.993 ms) : 0, 57993
Remote Config [baseline] (671.66 µs) : 0, 672
Remote Config [candidate] (675.311 µs) : 0, 675
Telemetry [baseline] (8.699 ms) : 0, 8699
Telemetry [candidate] (8.65 ms) : 0, 8650
IAST [baseline] (20.777 ms) : 0, 20777
IAST [candidate] (21.309 ms) : 0, 21309

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.46.0-SNAPSHOT~32ad019e3e, baseline=1.46.0-SNAPSHOT~991929d244

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.071 s) : 0, 1070960
Total [baseline] (10.508 s) : 0, 10508498
Agent [candidate] (1.06 s) : 0, 1060443
Total [candidate] (10.496 s) : 0, 10496426
section appsec
Agent [baseline] (1.194 s) : 0, 1193758
Total [baseline] (10.836 s) : 0, 10835907
Agent [candidate] (1.19 s) : 0, 1189688
Total [candidate] (10.816 s) : 0, 10816246
section iast
Agent [baseline] (1.183 s) : 0, 1183250
Total [baseline] (10.964 s) : 0, 10963912
Agent [candidate] (1.184 s) : 0, 1183844
Total [candidate] (10.967 s) : 0, 10967305
section profiling
Agent [baseline] (1.255 s) : 0, 1255447
Total [baseline] (10.883 s) : 0, 10883474
Agent [candidate] (1.256 s) : 0, 1256046
Total [candidate] (10.837 s) : 0, 10836721

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.071 s	-
Agent	appsec	1.194 s	122.799 ms (11.5%)
Agent	iast	1.183 s	112.29 ms (10.5%)
Agent	profiling	1.255 s	184.488 ms (17.2%)
Total	tracing	10.508 s	-
Total	appsec	10.836 s	327.409 ms (3.1%)
Total	iast	10.964 s	455.414 ms (4.3%)
Total	profiling	10.883 s	374.976 ms (3.6%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.06 s	-
Agent	appsec	1.19 s	129.245 ms (12.2%)
Agent	iast	1.184 s	123.401 ms (11.6%)
Agent	profiling	1.256 s	195.603 ms (18.4%)
Total	tracing	10.496 s	-
Total	appsec	10.816 s	319.819 ms (3.0%)
Total	iast	10.967 s	470.878 ms (4.5%)
Total	profiling	10.837 s	340.295 ms (3.2%)

gantt
    title petclinic - break down per module: candidate=1.46.0-SNAPSHOT~32ad019e3e, baseline=1.46.0-SNAPSHOT~991929d244

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (725.476 ms) : 0, 725476
BytebuddyAgent [candidate] (719.577 ms) : 0, 719577
GlobalTracer [baseline] (258.862 ms) : 0, 258862
GlobalTracer [candidate] (255.982 ms) : 0, 255982
AppSec [baseline] (57.088 ms) : 0, 57088
AppSec [candidate] (57.022 ms) : 0, 57022
Remote Config [baseline] (761.457 µs) : 0, 761
Remote Config [candidate] (738.741 µs) : 0, 739
Telemetry [baseline] (13.583 ms) : 0, 13583
Telemetry [candidate] (12.021 ms) : 0, 12021
section appsec
BytebuddyAgent [baseline] (733.655 ms) : 0, 733655
BytebuddyAgent [candidate] (732.222 ms) : 0, 732222
GlobalTracer [baseline] (254.127 ms) : 0, 254127
GlobalTracer [candidate] (253.089 ms) : 0, 253089
AppSec [baseline] (172.157 ms) : 0, 172157
AppSec [candidate] (170.818 ms) : 0, 170818
Remote Config [baseline] (671.818 µs) : 0, 672
Remote Config [candidate] (660.581 µs) : 0, 661
Telemetry [baseline] (8.235 ms) : 0, 8235
Telemetry [candidate] (8.167 ms) : 0, 8167
IAST [baseline] (19.838 ms) : 0, 19838
IAST [candidate] (19.419 ms) : 0, 19419
section iast
BytebuddyAgent [baseline] (832.034 ms) : 0, 832034
BytebuddyAgent [candidate] (832.727 ms) : 0, 832727
GlobalTracer [baseline] (246.584 ms) : 0, 246584
GlobalTracer [candidate] (246.943 ms) : 0, 246943
AppSec [baseline] (58.326 ms) : 0, 58326
AppSec [candidate] (58.065 ms) : 0, 58065
Remote Config [baseline] (676.541 µs) : 0, 677
Remote Config [candidate] (660.671 µs) : 0, 661
Telemetry [baseline] (8.826 ms) : 0, 8826
Telemetry [candidate] (8.79 ms) : 0, 8790
IAST [baseline] (21.763 ms) : 0, 21763
IAST [candidate] (21.584 ms) : 0, 21584
section profiling
BytebuddyAgent [baseline] (703.822 ms) : 0, 703822
BytebuddyAgent [candidate] (705.239 ms) : 0, 705239
GlobalTracer [baseline] (349.226 ms) : 0, 349226
GlobalTracer [candidate] (349.73 ms) : 0, 349730
AppSec [baseline] (54.948 ms) : 0, 54948
AppSec [candidate] (54.021 ms) : 0, 54021
Remote Config [baseline] (709.851 µs) : 0, 710
Remote Config [candidate] (723.828 µs) : 0, 724
Telemetry [baseline] (8.753 ms) : 0, 8753
Telemetry [candidate] (8.804 ms) : 0, 8804
ProfilingAgent [baseline] (95.874 ms) : 0, 95874
ProfilingAgent [candidate] (95.354 ms) : 0, 95354
Profiling [baseline] (95.898 ms) : 0, 95898
Profiling [candidate] (95.377 ms) : 0, 95377

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-01-23T16:28:11	2025-01-23T16:35:17
git_branch	master	nikita-tkachenko/test-framework-tags-sync
git_commit_date	1737645585	1737649173
git_commit_sha	`991929d`	`32ad019`
release_version	1.46.0-SNAPSHOT~991929d244	1.46.0-SNAPSHOT~32ad019e3e
start_time	2025-01-23T16:27:57	2025-01-23T16:35:03

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1737650477	1737650477
ci_job_id	778053827	778053827
ci_pipeline_id	53816971	53816971
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-h9qrujh-project-304-concurrent-0-cq96yl3l 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-h9qrujh-project-304-concurrent-0-cq96yl3l 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 13 metrics, 15 unstable metrics.

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~32ad019e3e, baseline=1.46.0-SNAPSHOT~991929d244
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.357 ms) : 1337, 1376
.   : milestone, 1357,
appsec (1.757 ms) : 1734, 1781
.   : milestone, 1757,
appsec_no_iast (1.778 ms) : 1754, 1802
.   : milestone, 1778,
iast (1.505 ms) : 1481, 1529
.   : milestone, 1505,
profiling (1.543 ms) : 1517, 1569
.   : milestone, 1543,
tracing (1.503 ms) : 1478, 1527
.   : milestone, 1503,
section candidate
no_agent (1.364 ms) : 1345, 1384
.   : milestone, 1364,
appsec (1.744 ms) : 1720, 1767
.   : milestone, 1744,
appsec_no_iast (1.747 ms) : 1722, 1772
.   : milestone, 1747,
iast (1.517 ms) : 1492, 1542
.   : milestone, 1517,
profiling (1.543 ms) : 1519, 1567
.   : milestone, 1543,
tracing (1.509 ms) : 1485, 1533
.   : milestone, 1509,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.357 ms [1.337 ms, 1.376 ms]	-
appsec	1.757 ms [1.734 ms, 1.781 ms]	400.235 µs (29.5%)
appsec_no_iast	1.778 ms [1.754 ms, 1.802 ms]	421.251 µs (31.0%)
iast	1.505 ms [1.481 ms, 1.529 ms]	148.299 µs (10.9%)
profiling	1.543 ms [1.517 ms, 1.569 ms]	186.059 µs (13.7%)
tracing	1.503 ms [1.478 ms, 1.527 ms]	145.74 µs (10.7%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.364 ms [1.345 ms, 1.384 ms]	-
appsec	1.744 ms [1.72 ms, 1.767 ms]	379.416 µs (27.8%)
appsec_no_iast	1.747 ms [1.722 ms, 1.772 ms]	382.762 µs (28.1%)
iast	1.517 ms [1.492 ms, 1.542 ms]	152.965 µs (11.2%)
profiling	1.543 ms [1.519 ms, 1.567 ms]	179.001 µs (13.1%)
tracing	1.509 ms [1.485 ms, 1.533 ms]	144.663 µs (10.6%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~32ad019e3e, baseline=1.46.0-SNAPSHOT~991929d244
    dateFormat X
    axisFormat %s
section baseline
no_agent (386.355 µs) : 363, 410
.   : milestone, 386,
iast (510.134 µs) : 488, 532
.   : milestone, 510,
iast_FULL (749.749 µs) : 728, 772
.   : milestone, 750,
iast_GLOBAL (557.967 µs) : 535, 581
.   : milestone, 558,
iast_HARDCODED_SECRET_DISABLED (508.6 µs) : 487, 530
.   : milestone, 509,
iast_INACTIVE (471.054 µs) : 450, 492
.   : milestone, 471,
iast_TELEMETRY_OFF (498.223 µs) : 476, 520
.   : milestone, 498,
tracing (456.864 µs) : 436, 478
.   : milestone, 457,
section candidate
no_agent (369.672 µs) : 350, 389
.   : milestone, 370,
iast (517.556 µs) : 495, 540
.   : milestone, 518,
iast_FULL (745.058 µs) : 723, 767
.   : milestone, 745,
iast_GLOBAL (561.538 µs) : 539, 584
.   : milestone, 562,
iast_HARDCODED_SECRET_DISABLED (511.099 µs) : 489, 533
.   : milestone, 511,
iast_INACTIVE (463.316 µs) : 441, 485
.   : milestone, 463,
iast_TELEMETRY_OFF (499.451 µs) : 478, 521
.   : milestone, 499,
tracing (453.289 µs) : 433, 474
.   : milestone, 453,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	386.355 µs [363.13 µs, 409.579 µs]	-
iast	510.134 µs [488.25 µs, 532.018 µs]	123.779 µs (32.0%)
iast_FULL	749.749 µs [727.78 µs, 771.718 µs]	363.395 µs (94.1%)
iast_GLOBAL	557.967 µs [534.96 µs, 580.975 µs]	171.613 µs (44.4%)
iast_HARDCODED_SECRET_DISABLED	508.6 µs [487.019 µs, 530.181 µs]	122.245 µs (31.6%)
iast_INACTIVE	471.054 µs [449.76 µs, 492.348 µs]	84.699 µs (21.9%)
iast_TELEMETRY_OFF	498.223 µs [476.123 µs, 520.324 µs]	111.869 µs (29.0%)
tracing	456.864 µs [436.054 µs, 477.674 µs]	70.509 µs (18.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	369.672 µs [350.045 µs, 389.299 µs]	-
iast	517.556 µs [495.444 µs, 539.668 µs]	147.884 µs (40.0%)
iast_FULL	745.058 µs [723.039 µs, 767.078 µs]	375.386 µs (101.5%)
iast_GLOBAL	561.538 µs [539.05 µs, 584.026 µs]	191.866 µs (51.9%)
iast_HARDCODED_SECRET_DISABLED	511.099 µs [489.348 µs, 532.85 µs]	141.427 µs (38.3%)
iast_INACTIVE	463.316 µs [441.448 µs, 485.184 µs]	93.644 µs (25.3%)
iast_TELEMETRY_OFF	499.451 µs [477.666 µs, 521.236 µs]	129.779 µs (35.1%)
tracing	453.289 µs [432.534 µs, 474.044 µs]	83.617 µs (22.6%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/test-framework-tags-sync
git_commit_date	1737645585	1737649173
git_commit_sha	`991929d`	`32ad019`
release_version	1.46.0-SNAPSHOT~991929d244	1.46.0-SNAPSHOT~32ad019e3e

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1737651038	1737651038
ci_job_id	778053828	778053828
ci_pipeline_id	53816971	53816971
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-fgow3nxt-project-304-concurrent-1-e82jzvlv 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-fgow3nxt-project-304-concurrent-1-e82jzvlv 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~32ad019e3e, baseline=1.46.0-SNAPSHOT~991929d244
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.472 ms) : 1460, 1483
.   : milestone, 1472,
appsec (2.356 ms) : 2312, 2399
.   : milestone, 2356,
iast (2.118 ms) : 2062, 2173
.   : milestone, 2118,
iast_GLOBAL (2.161 ms) : 2106, 2216
.   : milestone, 2161,
profiling (1.961 ms) : 1917, 2004
.   : milestone, 1961,
tracing (1.956 ms) : 1914, 1999
.   : milestone, 1956,
section candidate
no_agent (1.463 ms) : 1452, 1474
.   : milestone, 1463,
appsec (2.355 ms) : 2312, 2399
.   : milestone, 2355,
iast (2.117 ms) : 2062, 2171
.   : milestone, 2117,
iast_GLOBAL (2.156 ms) : 2100, 2211
.   : milestone, 2156,
profiling (1.967 ms) : 1923, 2011
.   : milestone, 1967,
tracing (1.947 ms) : 1905, 1989
.   : milestone, 1947,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.472 ms [1.46 ms, 1.483 ms]	-
appsec	2.356 ms [2.312 ms, 2.399 ms]	884.132 µs (60.1%)
iast	2.118 ms [2.062 ms, 2.173 ms]	646.088 µs (43.9%)
iast_GLOBAL	2.161 ms [2.106 ms, 2.216 ms]	689.538 µs (46.9%)
profiling	1.961 ms [1.917 ms, 2.004 ms]	489.181 µs (33.2%)
tracing	1.956 ms [1.914 ms, 1.999 ms]	484.972 µs (33.0%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.463 ms [1.452 ms, 1.474 ms]	-
appsec	2.355 ms [2.312 ms, 2.399 ms]	892.391 µs (61.0%)
iast	2.117 ms [2.062 ms, 2.171 ms]	653.634 µs (44.7%)
iast_GLOBAL	2.156 ms [2.1 ms, 2.211 ms]	692.757 µs (47.4%)
profiling	1.967 ms [1.923 ms, 2.011 ms]	503.971 µs (34.4%)
tracing	1.947 ms [1.905 ms, 1.989 ms]	484.187 µs (33.1%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~32ad019e3e, baseline=1.46.0-SNAPSHOT~991929d244
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.422 s) : 15422000, 15422000
.   : milestone, 15422000,
appsec (15.149 s) : 15149000, 15149000
.   : milestone, 15149000,
iast (18.649 s) : 18649000, 18649000
.   : milestone, 18649000,
iast_GLOBAL (17.833 s) : 17833000, 17833000
.   : milestone, 17833000,
profiling (15.439 s) : 15439000, 15439000
.   : milestone, 15439000,
tracing (14.798 s) : 14798000, 14798000
.   : milestone, 14798000,
section candidate
no_agent (15.184 s) : 15184000, 15184000
.   : milestone, 15184000,
appsec (14.979 s) : 14979000, 14979000
.   : milestone, 14979000,
iast (18.346 s) : 18346000, 18346000
.   : milestone, 18346000,
iast_GLOBAL (18.245 s) : 18245000, 18245000
.   : milestone, 18245000,
profiling (15.833 s) : 15833000, 15833000
.   : milestone, 15833000,
tracing (14.942 s) : 14942000, 14942000
.   : milestone, 14942000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.422 s [15.422 s, 15.422 s]	-
appsec	15.149 s [15.149 s, 15.149 s]	-273.0 ms (-1.8%)
iast	18.649 s [18.649 s, 18.649 s]	3.227 s (20.9%)
iast_GLOBAL	17.833 s [17.833 s, 17.833 s]	2.411 s (15.6%)
profiling	15.439 s [15.439 s, 15.439 s]	17.0 ms (0.1%)
tracing	14.798 s [14.798 s, 14.798 s]	-624.0 ms (-4.0%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.184 s [15.184 s, 15.184 s]	-
appsec	14.979 s [14.979 s, 14.979 s]	-205.0 ms (-1.4%)
iast	18.346 s [18.346 s, 18.346 s]	3.162 s (20.8%)
iast_GLOBAL	18.245 s [18.245 s, 18.245 s]	3.061 s (20.2%)
profiling	15.833 s [15.833 s, 15.833 s]	649.0 ms (4.3%)
tracing	14.942 s [14.942 s, 14.942 s]	-242.0 ms (-1.6%)

| Package | Type | Package file | Manager | Update | Change | |---|---|---|---|---|---| | [com.google.cloud:google-cloud-datastore](https://github.com/googleapis/java-datastore) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `2.25.4` -> `2.26.0` | | [com.datadoghq:dd-trace-api](https://github.com/datadog/dd-trace-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.45.2` -> `1.46.0` | | [com.datadoghq:dd-trace-ot](https://github.com/datadog/dd-trace-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.45.2` -> `1.46.0` | | [software.amazon.awssdk:sdk-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | | [software.amazon.awssdk:dynamodb-enhanced](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | | [software.amazon.awssdk:dynamodb](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | | [software.amazon.awssdk:aws-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | | [software.amazon.awssdk:bom](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | | [software.amazon.awssdk:auth](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | --- ### Release Notes <details> <summary>googleapis/java-datastore (com.google.cloud:google-cloud-datastore)</summary> ### [`v2.26.0`](https://github.com/googleapis/java-datastore/blob/HEAD/CHANGELOG.md#2260-2025-01-29) ##### Features - Add firestoreInDatastoreMode for datastore emulator ([#1698](googleapis/java-datastore#1698)) ([50f106d](googleapis/java-datastore@50f106d)) ##### Dependencies - Update dependency com.google.cloud:sdk-platform-java-config to v3.42.0 ([#1725](googleapis/java-datastore#1725)) ([1cbaf22](googleapis/java-datastore@1cbaf22)) </details> <details> <summary>datadog/dd-trace-java (com.datadoghq:dd-trace-api)</summary> ### [`v1.46.0`](https://github.com/DataDog/dd-trace-java/releases/tag/v1.46.0): 1.46.0 ##### Breaking Changes > \[!WARNING] > jnr-unixsocket is now an external dependency of dd-trace-ot and must be included when deploying dd-trace-ot. > \[!NOTE] > The API `TracerScope.setAsync(boolean)`, used to manually control asynchronous span propagation, does no more apply to the scope instance but to the active span scope. ##### Components ##### Application Security Management (IAST) - 🐛 Fix String.replace instrumentation for IAST ([#8281](DataDog/dd-trace-java#8281) - [@Mariovido](https://github.com/Mariovido)) - ✨ Apply the standard nomenclature to the stacktrace configs ([#8244](DataDog/dd-trace-java#8244) - [@jandro996](https://github.com/jandro996)) - 🐛 Exclude false positive weak randomness ([#8232](DataDog/dd-trace-java#8232) - [@jandro996](https://github.com/jandro996)) - ✨ Propagation of translateEscapes of String class ([#8186](DataDog/dd-trace-java#8186) - [@sezen-datadog](https://github.com/sezen-datadog)) - ✨ Add security control metrics ([#8175](DataDog/dd-trace-java#8175) - [@jandro996](https://github.com/jandro996)) - ✨ Increase IAST propagation to StringBuffer setLength ([#8128](DataDog/dd-trace-java#8128) - [@Mariovido](https://github.com/Mariovido)) - ✨ Add IAST taint tracking for DB values ([#8072](DataDog/dd-trace-java#8072) - [@Mariovido](https://github.com/Mariovido)) ##### Application Security Management (WAF) - 🐛 Prevents a NPE when there is no subscriber for user events ([#8258](DataDog/dd-trace-java#8258) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Apply the standard nomenclature to the stacktrace configs ([#8244](DataDog/dd-trace-java#8244) - [@jandro996](https://github.com/jandro996)) - 🐛 Ensure cached subscriptions are cleared on reconfiguration via RC ([#8229](DataDog/dd-trace-java#8229) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Add support for session tracking in Vertx ([#8167](DataDog/dd-trace-java#8167) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Create span tag: \_dd.appsec.rasp.timeout ([#8269](DataDog/dd-trace-java#8269) - [@Mariovido](https://github.com/Mariovido)) ##### Build & Tooling - 🐛 Ensure shaded helpers have unique names when injected into class-loaders ([#8192](DataDog/dd-trace-java#8192) - [@mcculls](https://github.com/mcculls)) ##### Configuration at Runtime - 🐛 Remove filtering of `DD_SERVICE` and `DD_ENV` from the tracer ([#8176](DataDog/dd-trace-java#8176) - [@mhlidd](https://github.com/mhlidd)) ##### Continuous Integration Visibility - 🧹 Generalize TestRetryPolicy to TestExecutionPolicy ([#8302](DataDog/dd-trace-java#8302) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🧹 Parallelize CI Visibility settings requests ([#8299](DataDog/dd-trace-java#8299) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🧹 Generalize test retry logic ([#8289](DataDog/dd-trace-java#8289) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🧹 Generalize tests skipping logic ([#8288](DataDog/dd-trace-java#8288) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🧹 Remove skip and shouldBeSkipped methods from TestEventsHandler in favor of isSkippable ([#8286](DataDog/dd-trace-java#8286) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨⚡ Optimize Git repository information computation ([#8270](DataDog/dd-trace-java#8270) - [@dougqh](https://github.com/dougqh)) - ✨ Always request known tests from the backend ([#8268](DataDog/dd-trace-java#8268) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Fix NPE when trying to get retry analyzer in Test NG ([#8253](DataDog/dd-trace-java#8253) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🐛 Set test framework and test framework version tags atomically ([#8252](DataDog/dd-trace-java#8252) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Add debug logging to Android Gradle module layout logic ([#8251](DataDog/dd-trace-java#8251) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🐛 Fix source and destination folders computation for Android Gradle projects ([#8190](DataDog/dd-trace-java#8190) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Add basic Scala Weaver sbt support ([#8189](DataDog/dd-trace-java#8189) - [@daniel-mohedano](https://github.com/daniel-mohedano)) - ✨ Implement impacted tests detection ([#8188](DataDog/dd-trace-java#8188) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) ##### Data Streams Monitoring - ✨ Change hash computation for protobuf to better represent impacting changes + save proto number in schema ([#8201](DataDog/dd-trace-java#8201) - [@vandonr](https://github.com/vandonr)) ##### Database Monitoring - Add peer service tag in dbm sql commenter ([#7913](DataDog/dd-trace-java#7913) - [@jordan-wong](https://github.com/jordan-wong)) ##### Dynamic Instrumentation - ✨ Add support for SymDB to scan directories ([#8306](DataDog/dd-trace-java#8306) - [@jpbempel](https://github.com/jpbempel)) - ✨ Add SymDB report for any jar scanning failures ([#8300](DataDog/dd-trace-java#8300) - [@jpbempel](https://github.com/jpbempel)) - ✨ Use two budgets depending on type ([#8283](DataDog/dd-trace-java#8283) - [@evanchooly](https://github.com/evanchooly)) - ✨ Institute a 10 snapshot per probe per trace budget ([#8277](DataDog/dd-trace-java#8277) - [@evanchooly](https://github.com/evanchooly)) - 🐛 Avoid double snapshots for Exception Replay ([#8273](DataDog/dd-trace-java#8273) - [@jpbempel](https://github.com/jpbempel)) - ✨ Simplify code origins. Separate out snapshot generation. ([#8263](DataDog/dd-trace-java#8263) - [@evanchooly](https://github.com/evanchooly)) - ✨ Add Exception probe custom instrumentation ([#8230](DataDog/dd-trace-java#8230) - [@jpbempel](https://github.com/jpbempel)) - ✨ Enhance log probes to honor debug session tags ([#8215](DataDog/dd-trace-java#8215) - [@evanchooly](https://github.com/evanchooly)) - 🐛 Don't redact env tokens from debugger probe snapshots ([#8211](DataDog/dd-trace-java#8211) - [@watson](https://github.com/watson)) - ✨⚡ Move Trace/SpanId capture at commit time ([#8184](DataDog/dd-trace-java#8184) - [@jpbempel](https://github.com/jpbempel)) - 🐛 Capture values at entry for method probe ([#8169](DataDog/dd-trace-java#8169) - [@jpbempel](https://github.com/jpbempel)) ##### JMX fetch - 🐛 Mute JMXFetch Shutdown in progress error ([#8068](DataDog/dd-trace-java#8068) - [@ygree](https://github.com/ygree)) ##### OpenTracing - ⚠️🧹 Make jnr-unixsocket an explicit dependency of dd-trace-ot ([#8307](DataDog/dd-trace-java#8307) - [@mcculls](https://github.com/mcculls)) ##### Profiling - 🐛 Avoid unsupported API call for creating folders on windows ([#8304](DataDog/dd-trace-java#8304) - [@jbachorik](https://github.com/jbachorik)) - ✨ Tag profiles for serverless ([#8279](DataDog/dd-trace-java#8279) - [@jbachorik](https://github.com/jbachorik)) - ✨ add queue type and length to queue events ([#8242](DataDog/dd-trace-java#8242) - [@richardstartin](https://github.com/richardstartin)) - 🐛 TempLocationManager Fixes and Improvements ([#8191](DataDog/dd-trace-java#8191) - [@jbachorik](https://github.com/jbachorik)) - ✨ Bump ddprof to 1.18.0 ([#8173](DataDog/dd-trace-java#8173) - [@jbachorik](https://github.com/jbachorik)) - ✨ Report profiler initialization and configuration errors to telemetry ([#8171](DataDog/dd-trace-java#8171) - [@jbachorik](https://github.com/jbachorik)) ##### Telemetry - ✨ Add pending traces report in tracer flares ([#8053](DataDog/dd-trace-java#8053) - [@mhlidd](https://github.com/mhlidd)) ##### Testing - ✨ Test http server requests in parallel ([#8222](DataDog/dd-trace-java#8222) - [@amarziali](https://github.com/amarziali)) ##### Trace context propagation - ✨ Add non default propagator registration ([#8310](DataDog/dd-trace-java#8310) - [@PerfectSlayer](https://github.com/PerfectSlayer)) ##### Tracer core - ✨ Probe for existence of IBMSASL or ACCP security providers ([#8276](DataDog/dd-trace-java#8276) - [@mcculls](https://github.com/mcculls)) - ✨⚡ Overhead improvement to agent feedback based sampling ([#8265](DataDog/dd-trace-java#8265) - [@dougqh](https://github.com/dougqh)) - 🧹 Move async propagation API from scope to tracer ([#8231](DataDog/dd-trace-java#8231) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - ✨ Introduce context propagation API ([#8161](DataDog/dd-trace-java#8161) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - ✨🧪 Use env-entry to add tags per webapp deployment ([#8138](DataDog/dd-trace-java#8138) - [@amarziali](https://github.com/amarziali)) - ✨ Introduce context helpers API ([#8134](DataDog/dd-trace-java#8134) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - ✨ Support IPv6 values for `DD_AGENT_HOST` and `DD_TRACE_AGENT_URL` ([#7984](DataDog/dd-trace-java#7984) - [@mhlidd](https://github.com/mhlidd)) ##### Instrumentations ##### Apache HttpComponents - 🐛 Properly finish spans and support latest apache httpclient5 ([#8272](DataDog/dd-trace-java#8272) - [@amarziali](https://github.com/amarziali)) ##### AWS Lambda instrumentation - 🐛 Properly capture lambda payloads for all handler types. ([#8264](DataDog/dd-trace-java#8264) - [@purple4reina](https://github.com/purple4reina)) ##### AWS S3 instrumentation - 💡 Create S3 instrumentation + add span pointers ([#8075](DataDog/dd-trace-java#8075) - [@nhulston](https://github.com/nhulston)) ##### AWS SDK instrumentation - 🐛 Revert "Add avoid double instrumenting lambda non-streaming handlers." ([#8247](DataDog/dd-trace-java#8247) - [@nhulston](https://github.com/nhulston)) ##### Cassandra - ✨ Allow extracting keyspace from statement result ([#8239](DataDog/dd-trace-java#8239) - [@amarziali](https://github.com/amarziali)) ##### Core Java language instrumentation - ✨ Propagation of translateEscapes of String class ([#8186](DataDog/dd-trace-java#8186) - [@sezen-datadog](https://github.com/sezen-datadog)) ##### Eclipse Vert.x instrumentation - 🐛 Fix vertx worker propagation and error handling ([#8237](DataDog/dd-trace-java#8237) - [@amarziali](https://github.com/amarziali)) - ✨ Support vertx 5 ([#8220](DataDog/dd-trace-java#8220) - [@amarziali](https://github.com/amarziali)) - ✨ Add support for session tracking in Vertx ([#8167](DataDog/dd-trace-java#8167) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) ##### Kafka instrumentation - 🐛 Prevent possible NPE calculating Kafka record header size ([#8292](DataDog/dd-trace-java#8292) - [@ygree](https://github.com/ygree)) ##### Mule instrumentation - 🐛 Fix crash using Mule with JPMS ([#8187](DataDog/dd-trace-java#8187) - [@amarziali](https://github.com/amarziali)) ##### Protocol Buffer instrumentation - ✨ Change hash computation for protobuf to better represent impacting changes + save proto number in schema ([#8201](DataDog/dd-trace-java#8201) - [@vandonr](https://github.com/vandonr)) ##### Spring instrumentation - 🐛 Preserve getQualifier from spring scheduling runnables ([#8293](DataDog/dd-trace-java#8293) - [@amarziali](https://github.com/amarziali)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 6pm every weekday,before 2am every weekday" in timezone Australia/Melbourne, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). GitOrigin-RevId: bb09d47e4eed77a003f630273b4d0a84003eb899

nikita-tkachenko-datadog added type: bug comp: ci visibility Continuous Integration Visibility labels Jan 20, 2025

nikita-tkachenko-datadog marked this pull request as ready for review January 20, 2025 13:38

nikita-tkachenko-datadog requested review from a team as code owners January 20, 2025 13:38

nikita-tkachenko-datadog requested review from anmarchenko and ygree January 20, 2025 13:38

juan-fernandez approved these changes Jan 20, 2025

View reviewed changes

Set test framework and test framework version tags atomically

a83cee4

nikita-tkachenko-datadog force-pushed the nikita-tkachenko/test-framework-tags-sync branch from 20a5542 to a83cee4 Compare January 22, 2025 16:06

Merge branch 'master' into nikita-tkachenko/test-framework-tags-sync

765343e

anmarchenko approved these changes Jan 23, 2025

View reviewed changes

nikita-tkachenko-datadog added 3 commits January 23, 2025 11:46

Merge branch 'master' into nikita-tkachenko/test-framework-tags-sync

079a8c0

Merge branch 'master' into nikita-tkachenko/test-framework-tags-sync

d81799f

Merge branch 'master' into nikita-tkachenko/test-framework-tags-sync

32ad019

PerfectSlayer added type: bug and removed type: bug labels Jan 24, 2025

nikita-tkachenko-datadog merged commit 3bbea85 into master Jan 24, 2025
174 of 175 checks passed

nikita-tkachenko-datadog deleted the nikita-tkachenko/test-framework-tags-sync branch January 24, 2025 12:16

github-actions bot added this to the 1.46.0 milestone Jan 24, 2025

nikita-tkachenko-datadog mentioned this pull request Jan 28, 2025

Do not set test framework version tag if the value is null #8295

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Set test framework and test framework version tags atomically #8252

Set test framework and test framework version tags atomically #8252

nikita-tkachenko-datadog commented Jan 20, 2025

pr-commenter bot commented Jan 20, 2025 •

edited

Loading

Set test framework and test framework version tags atomically #8252

Set test framework and test framework version tags atomically #8252

Conversation

nikita-tkachenko-datadog commented Jan 20, 2025

What Does This Do

Motivation

Contributor Checklist

pr-commenter bot commented Jan 20, 2025 • edited Loading

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

pr-commenter bot commented Jan 20, 2025 •

edited

Loading