Switch some unused Mocks to Stubs to avoid recording actions #6372

bantonsson · 2023-12-15T12:55:04Z

What Does This Do

Changes some Mock instances to Stub instances when the interactions are not validated.

Motivation

Had an OOM failure locally when running :dd-trace-core:test, and the heap dump showed org.spockframework.mock.runtime.InteractionScope holding on to 190 MB in DDAgentWriterCombinedTest.

Additional Notes

Changed all applicable Mock calls of CoreTracer and PendingTrace in all tests, since those where the culprits in DDAgentWriterCombinedTest.

pr-commenter · 2023-12-15T13:20:34Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	ban/mock-is-not-stub
git_commit_date	1702652090	1702657916
git_commit_sha	`7a1a4ca`	`014f0bc`
release_version	1.26.0-SNAPSHOT~7a1a4cac29	1.26.0-SNAPSHOT~014f0bc440

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1702660551	1702660551
ci_job_id	391005648	391005648
ci_pipeline_id	25171493	25171493
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 49 metrics, 5 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.26.0-SNAPSHOT~014f0bc440, baseline=1.26.0-SNAPSHOT~7a1a4cac29

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.045 s) : 0, 1044707
Total [baseline] (8.724 s) : 0, 8724440
Agent [candidate] (1.048 s) : 0, 1047563
Total [candidate] (8.723 s) : 0, 8722864
section iast
Agent [baseline] (1.162 s) : 0, 1161580
Total [baseline] (9.245 s) : 0, 9245263
Agent [candidate] (1.173 s) : 0, 1172810
Total [candidate] (9.276 s) : 0, 9275627
section iast_TELEMETRY_OFF
Agent [baseline] (1.157 s) : 0, 1157233
Total [baseline] (9.269 s) : 0, 9269254
Agent [candidate] (1.161 s) : 0, 1160775
Total [candidate] (9.239 s) : 0, 9238781

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.045 s	-
Agent	iast	1.162 s	116.874 ms (11.2%)
Agent	iast_TELEMETRY_OFF	1.157 s	112.527 ms (10.8%)
Total	tracing	8.724 s	-
Total	iast	9.245 s	520.823 ms (6.0%)
Total	iast_TELEMETRY_OFF	9.269 s	544.814 ms (6.2%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.048 s	-
Agent	iast	1.173 s	125.247 ms (12.0%)
Agent	iast_TELEMETRY_OFF	1.161 s	113.212 ms (10.8%)
Total	tracing	8.723 s	-
Total	iast	9.276 s	552.762 ms (6.3%)
Total	iast_TELEMETRY_OFF	9.239 s	515.917 ms (5.9%)

gantt
    title insecure-bank - break down per module: candidate=1.26.0-SNAPSHOT~014f0bc440, baseline=1.26.0-SNAPSHOT~7a1a4cac29

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (646.271 ms) : 0, 646271
BytebuddyAgent [candidate] (646.978 ms) : 0, 646978
GlobalTracer [baseline] (305.463 ms) : 0, 305463
GlobalTracer [candidate] (307.142 ms) : 0, 307142
AppSec [baseline] (50.772 ms) : 0, 50772
AppSec [candidate] (51.394 ms) : 0, 51394
Remote Config [baseline] (681.708 µs) : 0, 682
Remote Config [candidate] (675.997 µs) : 0, 676
Telemetry [baseline] (7.301 ms) : 0, 7301
Telemetry [candidate] (7.233 ms) : 0, 7233
section iast
BytebuddyAgent [baseline] (765.734 ms) : 0, 765734
BytebuddyAgent [candidate] (773.899 ms) : 0, 773899
GlobalTracer [baseline] (284.121 ms) : 0, 284121
GlobalTracer [candidate] (286.851 ms) : 0, 286851
AppSec [baseline] (49.011 ms) : 0, 49011
AppSec [candidate] (49.047 ms) : 0, 49047
IAST [baseline] (20.672 ms) : 0, 20672
IAST [candidate] (20.628 ms) : 0, 20628
Remote Config [baseline] (618.182 µs) : 0, 618
Remote Config [candidate] (619.998 µs) : 0, 620
Telemetry [baseline] (7.202 ms) : 0, 7202
Telemetry [candidate] (7.179 ms) : 0, 7179
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (760.432 ms) : 0, 760432
BytebuddyAgent [candidate] (761.987 ms) : 0, 761987
GlobalTracer [baseline] (285.245 ms) : 0, 285245
GlobalTracer [candidate] (285.884 ms) : 0, 285884
AppSec [baseline] (49.188 ms) : 0, 49188
AppSec [candidate] (49.692 ms) : 0, 49692
IAST [baseline] (20.874 ms) : 0, 20874
IAST [candidate] (21.624 ms) : 0, 21624
Remote Config [baseline] (626.488 µs) : 0, 626
Remote Config [candidate] (630.795 µs) : 0, 631
Telemetry [baseline] (6.556 ms) : 0, 6556
Telemetry [candidate] (6.65 ms) : 0, 6650

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.26.0-SNAPSHOT~014f0bc440, baseline=1.26.0-SNAPSHOT~7a1a4cac29

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.047 s) : 0, 1047359
Total [baseline] (9.302 s) : 0, 9301739
Agent [candidate] (1.049 s) : 0, 1049287
Total [candidate] (9.378 s) : 0, 9378027
section appsec
Agent [baseline] (1.143 s) : 0, 1143147
Total [baseline] (9.421 s) : 0, 9420842
Agent [candidate] (1.147 s) : 0, 1146812
Total [candidate] (9.465 s) : 0, 9465457
section iast
Agent [baseline] (1.165 s) : 0, 1165013
Total [baseline] (9.551 s) : 0, 9551327
Agent [candidate] (1.168 s) : 0, 1167915
Total [candidate] (9.547 s) : 0, 9546930
section profiling
Agent [baseline] (1.251 s) : 0, 1251187
Total [baseline] (9.681 s) : 0, 9680546
Agent [candidate] (1.25 s) : 0, 1249634
Total [candidate] (9.703 s) : 0, 9703181

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.047 s	-
Agent	appsec	1.143 s	95.788 ms (9.1%)
Agent	iast	1.165 s	117.653 ms (11.2%)
Agent	profiling	1.251 s	203.827 ms (19.5%)
Total	tracing	9.302 s	-
Total	appsec	9.421 s	119.102 ms (1.3%)
Total	iast	9.551 s	249.588 ms (2.7%)
Total	profiling	9.681 s	378.807 ms (4.1%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.049 s	-
Agent	appsec	1.147 s	97.525 ms (9.3%)
Agent	iast	1.168 s	118.628 ms (11.3%)
Agent	profiling	1.25 s	200.347 ms (19.1%)
Total	tracing	9.378 s	-
Total	appsec	9.465 s	87.43 ms (0.9%)
Total	iast	9.547 s	168.903 ms (1.8%)
Total	profiling	9.703 s	325.154 ms (3.5%)

gantt
    title petclinic - break down per module: candidate=1.26.0-SNAPSHOT~014f0bc440, baseline=1.26.0-SNAPSHOT~7a1a4cac29

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (647.326 ms) : 0, 647326
BytebuddyAgent [candidate] (648.564 ms) : 0, 648564
GlobalTracer [baseline] (306.597 ms) : 0, 306597
GlobalTracer [candidate] (307.434 ms) : 0, 307434
AppSec [baseline] (51.409 ms) : 0, 51409
AppSec [candidate] (51.185 ms) : 0, 51185
Remote Config [baseline] (666.313 µs) : 0, 666
Remote Config [candidate] (667.591 µs) : 0, 668
Telemetry [baseline] (7.141 ms) : 0, 7141
Telemetry [candidate] (7.2 ms) : 0, 7200
section appsec
BytebuddyAgent [baseline] (645.732 ms) : 0, 645732
BytebuddyAgent [candidate] (648.164 ms) : 0, 648164
GlobalTracer [baseline] (306.451 ms) : 0, 306451
GlobalTracer [candidate] (307.554 ms) : 0, 307554
AppSec [baseline] (149.293 ms) : 0, 149293
AppSec [candidate] (149.373 ms) : 0, 149373
Remote Config [baseline] (645.252 µs) : 0, 645
Remote Config [candidate] (640.157 µs) : 0, 640
Telemetry [baseline] (6.818 ms) : 0, 6818
Telemetry [candidate] (6.845 ms) : 0, 6845
section iast
BytebuddyAgent [baseline] (768.07 ms) : 0, 768070
BytebuddyAgent [candidate] (770.256 ms) : 0, 770256
GlobalTracer [baseline] (284.306 ms) : 0, 284306
GlobalTracer [candidate] (285.617 ms) : 0, 285617
AppSec [baseline] (48.891 ms) : 0, 48891
AppSec [candidate] (49.198 ms) : 0, 49198
IAST [baseline] (22.258 ms) : 0, 22258
IAST [candidate] (21.346 ms) : 0, 21346
Remote Config [baseline] (608.154 µs) : 0, 608
Remote Config [candidate] (630.473 µs) : 0, 630
Telemetry [baseline] (6.463 ms) : 0, 6463
Telemetry [candidate] (6.507 ms) : 0, 6507
section profiling
BytebuddyAgent [baseline] (661.034 ms) : 0, 661034
BytebuddyAgent [candidate] (661.873 ms) : 0, 661873
GlobalTracer [baseline] (380.067 ms) : 0, 380067
GlobalTracer [candidate] (378.408 ms) : 0, 378408
AppSec [baseline] (51.761 ms) : 0, 51761
AppSec [candidate] (51.437 ms) : 0, 51437
Remote Config [baseline] (672.496 µs) : 0, 672
Remote Config [candidate] (673.054 µs) : 0, 673
Telemetry [baseline] (7.56 ms) : 0, 7560
Telemetry [candidate] (7.5 ms) : 0, 7500
ProfilingAgent [baseline] (95.763 ms) : 0, 95763
ProfilingAgent [candidate] (95.238 ms) : 0, 95238
Profiling [baseline] (95.788 ms) : 0, 95788
Profiling [candidate] (95.264 ms) : 0, 95264

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2023-12-15T16:55:03	2023-12-15T17:11:38
git_branch	master	ban/mock-is-not-stub
git_commit_date	1702652090	1702657916
git_commit_sha	`7a1a4ca`	`014f0bc`
release_version	1.26.0-SNAPSHOT~7a1a4cac29	1.26.0-SNAPSHOT~014f0bc440
start_time	2023-12-15T16:54:50	2023-12-15T17:11:25

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1702660551	1702660551
ci_job_id	391005648	391005648
ci_pipeline_id	25171493	25171493
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 8 metrics, 14 unstable metrics.

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.26.0-SNAPSHOT~014f0bc440, baseline=1.26.0-SNAPSHOT~7a1a4cac29
    dateFormat X
    axisFormat %s
section baseline
no_agent (363.35 µs) : 343, 383
.   : milestone, 363,
iast (472.956 µs) : 452, 494
.   : milestone, 473,
iast_FULL (533.909 µs) : 514, 554
.   : milestone, 534,
iast_INACTIVE (441.668 µs) : 420, 463
.   : milestone, 442,
iast_TELEMETRY_OFF (464.598 µs) : 444, 485
.   : milestone, 465,
tracing (444.357 µs) : 423, 465
.   : milestone, 444,
section candidate
no_agent (365.087 µs) : 344, 386
.   : milestone, 365,
iast (466.913 µs) : 447, 487
.   : milestone, 467,
iast_FULL (531.985 µs) : 512, 552
.   : milestone, 532,
iast_INACTIVE (459.977 µs) : 439, 481
.   : milestone, 460,
iast_TELEMETRY_OFF (461.987 µs) : 441, 483
.   : milestone, 462,
tracing (442.184 µs) : 421, 463
.   : milestone, 442,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	363.35 µs [343.46 µs, 383.24 µs]	-
iast	472.956 µs [451.999 µs, 493.912 µs]	109.606 µs (30.2%)
iast_FULL	533.909 µs [513.792 µs, 554.026 µs]	170.559 µs (46.9%)
iast_INACTIVE	441.668 µs [420.474 µs, 462.862 µs]	78.318 µs (21.6%)
iast_TELEMETRY_OFF	464.598 µs [443.908 µs, 485.289 µs]	101.248 µs (27.9%)
tracing	444.357 µs [423.418 µs, 465.296 µs]	81.007 µs (22.3%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	365.087 µs [343.905 µs, 386.269 µs]	-
iast	466.913 µs [446.811 µs, 487.014 µs]	101.826 µs (27.9%)
iast_FULL	531.985 µs [511.569 µs, 552.401 µs]	166.898 µs (45.7%)
iast_INACTIVE	459.977 µs [438.877 µs, 481.076 µs]	94.89 µs (26.0%)
iast_TELEMETRY_OFF	461.987 µs [441.037 µs, 482.937 µs]	96.9 µs (26.5%)
tracing	442.184 µs [421.299 µs, 463.069 µs]	77.097 µs (21.1%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.26.0-SNAPSHOT~014f0bc440, baseline=1.26.0-SNAPSHOT~7a1a4cac29
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.351 ms) : 1332, 1370
.   : milestone, 1351,
appsec (1.752 ms) : 1727, 1776
.   : milestone, 1752,
iast (1.54 ms) : 1516, 1564
.   : milestone, 1540,
profiling (1.531 ms) : 1506, 1557
.   : milestone, 1531,
tracing (1.476 ms) : 1452, 1501
.   : milestone, 1476,
section candidate
no_agent (1.349 ms) : 1330, 1368
.   : milestone, 1349,
appsec (1.752 ms) : 1725, 1778
.   : milestone, 1752,
iast (1.495 ms) : 1471, 1520
.   : milestone, 1495,
profiling (1.514 ms) : 1489, 1539
.   : milestone, 1514,
tracing (1.491 ms) : 1466, 1516
.   : milestone, 1491,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.351 ms [1.332 ms, 1.37 ms]	-
appsec	1.752 ms [1.727 ms, 1.776 ms]	400.432 µs (29.6%)
iast	1.54 ms [1.516 ms, 1.564 ms]	189.138 µs (14.0%)
profiling	1.531 ms [1.506 ms, 1.557 ms]	180.313 µs (13.3%)
tracing	1.476 ms [1.452 ms, 1.501 ms]	125.206 µs (9.3%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.349 ms [1.33 ms, 1.368 ms]	-
appsec	1.752 ms [1.725 ms, 1.778 ms]	402.643 µs (29.8%)
iast	1.495 ms [1.471 ms, 1.52 ms]	146.562 µs (10.9%)
profiling	1.514 ms [1.489 ms, 1.539 ms]	164.738 µs (12.2%)
tracing	1.491 ms [1.466 ms, 1.516 ms]	142.13 µs (10.5%)

bantonsson force-pushed the ban/mock-is-not-stub branch from 5dfeac3 to c991b2f Compare December 15, 2023 12:55

ValentinZakharov mentioned this pull request Dec 15, 2023

Replaced unused Mocks with Stubs in AppSec #6374

Merged

bantonsson force-pushed the ban/mock-is-not-stub branch from c991b2f to 6ffa19d Compare December 15, 2023 16:28

Switch some unused Mocks to Stubs to avoid recording actions

014f0bc

bantonsson force-pushed the ban/mock-is-not-stub branch from 6ffa19d to 014f0bc Compare December 15, 2023 16:32

ValentinZakharov mentioned this pull request Dec 15, 2023

Replaced unused Mocks with Stubs in IAST #6377

Merged

bantonsson marked this pull request as ready for review December 15, 2023 17:06

bantonsson requested a review from a team as a code owner December 15, 2023 17:06

bantonsson requested review from mcculls and ygree December 15, 2023 17:06

PerfectSlayer approved these changes Dec 18, 2023

View reviewed changes

PerfectSlayer added comp: testing Testing tag: no release notes Changes to exclude from release notes labels Dec 18, 2023

bantonsson merged commit 1f9ada9 into master Dec 18, 2023
10 of 11 checks passed

bantonsson deleted the ban/mock-is-not-stub branch December 18, 2023 07:44

github-actions bot added this to the 1.26.0 milestone Dec 18, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Switch some unused Mocks to Stubs to avoid recording actions #6372

Switch some unused Mocks to Stubs to avoid recording actions #6372

bantonsson commented Dec 15, 2023

pr-commenter bot commented Dec 15, 2023 •

edited

Loading

Switch some unused Mocks to Stubs to avoid recording actions #6372

Switch some unused Mocks to Stubs to avoid recording actions #6372

Conversation

bantonsson commented Dec 15, 2023

What Does This Do

Motivation

Additional Notes

pr-commenter bot commented Dec 15, 2023 • edited Loading

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

pr-commenter bot commented Dec 15, 2023 •

edited

Loading