Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Replaced unused Mocks with Stubs in IAST #6377

Merged
merged 1 commit into from
Dec 18, 2023
Merged

Replaced unused Mocks with Stubs in IAST #6377

merged 1 commit into from
Dec 18, 2023

Conversation

ValentinZakharov
Copy link
Contributor

What Does This Do

Changes some Mock instances to Stub instances when the interactions are not validated.

Motivation

Inspired by initial improvement done for core tracer in #6372

Additional Notes

@ValentinZakharov ValentinZakharov added comp: testing Testing tag: no release notes Changes to exclude from release notes comp: asm iast Application Security Management (IAST) labels Dec 15, 2023
@ValentinZakharov ValentinZakharov self-assigned this Dec 15, 2023
@github-actions GitHub Actions
Copy link
Contributor

Hi! 👋 Looks like you updated a Git Submodule.
If this was not intentional please make sure to:

@ValentinZakharov ValentinZakharov force-pushed the vzakharov/mocks_to_stubs_iast branch from 0419897 to 74be0bd Compare December 15, 2023 16:58
@pr-commenter
Copy link

pr-commenter bot commented Dec 15, 2023 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master vzakharov/mocks_to_stubs_iast
git_commit_date 1702652090 1702659464
git_commit_sha 7a1a4ca 74be0bd
release_version 1.26.0-SNAPSHOT~7a1a4cac29 1.26.0-SNAPSHOT~74be0bda04
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1702662108 1702662108
ci_job_id 391030044 391030044
ci_pipeline_id 25173331 25173331
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 47 metrics, 7 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.26.0-SNAPSHOT~74be0bda04, baseline=1.26.0-SNAPSHOT~7a1a4cac29

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.048 s) : 0, 1048203
Total [baseline] (9.358 s) : 0, 9358443
Agent [candidate] (1.055 s) : 0, 1054669
Total [candidate] (9.347 s) : 0, 9346756
section appsec
Agent [baseline] (1.154 s) : 0, 1154130
Total [baseline] (9.505 s) : 0, 9504927
Agent [candidate] (1.148 s) : 0, 1148346
Total [candidate] (9.472 s) : 0, 9471642
section iast
Agent [baseline] (1.166 s) : 0, 1166085
Total [baseline] (9.637 s) : 0, 9636686
Agent [candidate] (1.171 s) : 0, 1171135
Total [candidate] (9.555 s) : 0, 9555017
section profiling
Agent [baseline] (1.244 s) : 0, 1244371
Total [baseline] (9.63 s) : 0, 9630059
Agent [candidate] (1.245 s) : 0, 1244653
Total [candidate] (9.6 s) : 0, 9600117
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.048 s -
Agent appsec 1.154 s 105.926 ms (10.1%)
Agent iast 1.166 s 117.881 ms (11.2%)
Agent profiling 1.244 s 196.168 ms (18.7%)
Total tracing 9.358 s -
Total appsec 9.505 s 146.484 ms (1.6%)
Total iast 9.637 s 278.242 ms (3.0%)
Total profiling 9.63 s 271.615 ms (2.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.055 s -
Agent appsec 1.148 s 93.677 ms (8.9%)
Agent iast 1.171 s 116.466 ms (11.0%)
Agent profiling 1.245 s 189.985 ms (18.0%)
Total tracing 9.347 s -
Total appsec 9.472 s 124.886 ms (1.3%)
Total iast 9.555 s 208.261 ms (2.2%)
Total profiling 9.6 s 253.36 ms (2.7%) 
gantt
    title petclinic - break down per module: candidate=1.26.0-SNAPSHOT~74be0bda04, baseline=1.26.0-SNAPSHOT~7a1a4cac29

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (648.789 ms) : 0, 648789
BytebuddyAgent [candidate] (653.362 ms) : 0, 653362
GlobalTracer [baseline] (306.263 ms) : 0, 306263
GlobalTracer [candidate] (307.552 ms) : 0, 307552
AppSec [baseline] (50.855 ms) : 0, 50855
AppSec [candidate] (51.148 ms) : 0, 51148
Remote Config [baseline] (666.531 µs) : 0, 667
Remote Config [candidate] (685.701 µs) : 0, 686
Telemetry [baseline] (7.253 ms) : 0, 7253
Telemetry [candidate] (7.232 ms) : 0, 7232
section appsec
BytebuddyAgent [baseline] (653.034 ms) : 0, 653034
BytebuddyAgent [candidate] (648.937 ms) : 0, 648937
GlobalTracer [baseline] (308.897 ms) : 0, 308897
GlobalTracer [candidate] (308.046 ms) : 0, 308046
AppSec [baseline] (150.042 ms) : 0, 150042
AppSec [candidate] (149.52 ms) : 0, 149520
Remote Config [baseline] (650.469 µs) : 0, 650
Remote Config [candidate] (653.0 µs) : 0, 653
Telemetry [baseline] (6.895 ms) : 0, 6895
Telemetry [candidate] (6.843 ms) : 0, 6843
section iast
BytebuddyAgent [baseline] (768.942 ms) : 0, 768942
BytebuddyAgent [candidate] (772.027 ms) : 0, 772027
GlobalTracer [baseline] (285.067 ms) : 0, 285067
GlobalTracer [candidate] (286.065 ms) : 0, 286065
AppSec [baseline] (49.165 ms) : 0, 49165
AppSec [candidate] (49.284 ms) : 0, 49284
Remote Config [baseline] (612.576 µs) : 0, 613
Remote Config [candidate] (630.472 µs) : 0, 630
Telemetry [baseline] (8.072 ms) : 0, 8072
Telemetry [candidate] (7.435 ms) : 0, 7435
IAST [baseline] (19.939 ms) : 0, 19939
IAST [candidate] (21.317 ms) : 0, 21317
section profiling
BytebuddyAgent [baseline] (658.633 ms) : 0, 658633
BytebuddyAgent [candidate] (659.581 ms) : 0, 659581
GlobalTracer [baseline] (377.049 ms) : 0, 377049
GlobalTracer [candidate] (377.063 ms) : 0, 377063
AppSec [baseline] (51.462 ms) : 0, 51462
AppSec [candidate] (51.23 ms) : 0, 51230
Remote Config [baseline] (667.136 µs) : 0, 667
Remote Config [candidate] (651.945 µs) : 0, 652
Telemetry [baseline] (7.441 ms) : 0, 7441
Telemetry [candidate] (7.409 ms) : 0, 7409
ProfilingAgent [baseline] (94.682 ms) : 0, 94682
ProfilingAgent [candidate] (94.374 ms) : 0, 94374
Profiling [baseline] (94.707 ms) : 0, 94707
Profiling [candidate] (94.399 ms) : 0, 94399
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.26.0-SNAPSHOT~74be0bda04, baseline=1.26.0-SNAPSHOT~7a1a4cac29

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.049 s) : 0, 1048690
Total [baseline] (8.714 s) : 0, 8714128
Agent [candidate] (1.05 s) : 0, 1049972
Total [candidate] (8.717 s) : 0, 8716771
section iast
Agent [baseline] (1.166 s) : 0, 1165756
Total [baseline] (9.262 s) : 0, 9262262
Agent [candidate] (1.175 s) : 0, 1175045
Total [candidate] (9.263 s) : 0, 9262570
section iast_TELEMETRY_OFF
Agent [baseline] (1.157 s) : 0, 1157140
Total [baseline] (9.25 s) : 0, 9250132
Agent [candidate] (1.169 s) : 0, 1168777
Total [candidate] (9.251 s) : 0, 9250916
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.049 s -
Agent iast 1.166 s 117.066 ms (11.2%)
Agent iast_TELEMETRY_OFF 1.157 s 108.45 ms (10.3%)
Total tracing 8.714 s -
Total iast 9.262 s 548.133 ms (6.3%)
Total iast_TELEMETRY_OFF 9.25 s 536.003 ms (6.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.05 s -
Agent iast 1.175 s 125.073 ms (11.9%)
Agent iast_TELEMETRY_OFF 1.169 s 118.805 ms (11.3%)
Total tracing 8.717 s -
Total iast 9.263 s 545.799 ms (6.3%)
Total iast_TELEMETRY_OFF 9.251 s 534.145 ms (6.1%) 
gantt
    title insecure-bank - break down per module: candidate=1.26.0-SNAPSHOT~74be0bda04, baseline=1.26.0-SNAPSHOT~7a1a4cac29

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (648.662 ms) : 0, 648662
BytebuddyAgent [candidate] (648.799 ms) : 0, 648799
GlobalTracer [baseline] (306.549 ms) : 0, 306549
GlobalTracer [candidate] (308.075 ms) : 0, 308075
AppSec [baseline] (51.254 ms) : 0, 51254
AppSec [candidate] (50.914 ms) : 0, 50914
Remote Config [baseline] (667.527 µs) : 0, 668
Remote Config [candidate] (679.496 µs) : 0, 679
Telemetry [baseline] (7.187 ms) : 0, 7187
Telemetry [candidate] (7.241 ms) : 0, 7241
section iast
BytebuddyAgent [baseline] (768.996 ms) : 0, 768996
BytebuddyAgent [candidate] (775.02 ms) : 0, 775020
GlobalTracer [baseline] (284.689 ms) : 0, 284689
GlobalTracer [candidate] (287.815 ms) : 0, 287815
AppSec [baseline] (49.071 ms) : 0, 49071
AppSec [candidate] (49.235 ms) : 0, 49235
IAST [baseline] (19.113 ms) : 0, 19113
IAST [candidate] (20.643 ms) : 0, 20643
Remote Config [baseline] (617.868 µs) : 0, 618
Remote Config [candidate] (609.092 µs) : 0, 609
Telemetry [baseline] (8.756 ms) : 0, 8756
Telemetry [candidate] (7.199 ms) : 0, 7199
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (761.408 ms) : 0, 761408
BytebuddyAgent [candidate] (768.175 ms) : 0, 768175
GlobalTracer [baseline] (285.339 ms) : 0, 285339
GlobalTracer [candidate] (287.682 ms) : 0, 287682
AppSec [baseline] (48.896 ms) : 0, 48896
AppSec [candidate] (49.44 ms) : 0, 49440
IAST [baseline] (19.251 ms) : 0, 19251
IAST [candidate] (21.632 ms) : 0, 21632
Remote Config [baseline] (624.121 µs) : 0, 624
Remote Config [candidate] (623.117 µs) : 0, 623
Telemetry [baseline] (7.343 ms) : 0, 7343
Telemetry [candidate] (6.631 ms) : 0, 6631
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2023-12-15T17:21:01 2023-12-15T17:37:33
git_branch master vzakharov/mocks_to_stubs_iast
git_commit_date 1702652090 1702659464
git_commit_sha 7a1a4ca 74be0bd
release_version 1.26.0-SNAPSHOT~7a1a4cac29 1.26.0-SNAPSHOT~74be0bda04
start_time 2023-12-15T17:20:48 2023-12-15T17:37:20
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1702662108 1702662108
ci_job_id 391030044 391030044
ci_pipeline_id 25173331 25173331
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 8 metrics, 14 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.26.0-SNAPSHOT~74be0bda04, baseline=1.26.0-SNAPSHOT~7a1a4cac29
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.348 ms) : 1329, 1368
.   : milestone, 1348,
appsec (1.76 ms) : 1735, 1786
.   : milestone, 1760,
iast (1.514 ms) : 1489, 1539
.   : milestone, 1514,
profiling (1.509 ms) : 1483, 1534
.   : milestone, 1509,
tracing (1.488 ms) : 1463, 1513
.   : milestone, 1488,
section candidate
no_agent (1.34 ms) : 1321, 1360
.   : milestone, 1340,
appsec (1.757 ms) : 1733, 1782
.   : milestone, 1757,
iast (1.505 ms) : 1481, 1530
.   : milestone, 1505,
profiling (1.545 ms) : 1519, 1572
.   : milestone, 1545,
tracing (1.475 ms) : 1451, 1499
.   : milestone, 1475,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.348 ms [1.329 ms, 1.368 ms] -
appsec 1.76 ms [1.735 ms, 1.786 ms] 412.076 µs (30.6%)
iast 1.514 ms [1.489 ms, 1.539 ms] 165.645 µs (12.3%)
profiling 1.509 ms [1.483 ms, 1.534 ms] 160.181 µs (11.9%)
tracing 1.488 ms [1.463 ms, 1.513 ms] 139.341 µs (10.3%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.34 ms [1.321 ms, 1.36 ms] -
appsec 1.757 ms [1.733 ms, 1.782 ms] 416.677 µs (31.1%)
iast 1.505 ms [1.481 ms, 1.53 ms] 164.676 µs (12.3%)
profiling 1.545 ms [1.519 ms, 1.572 ms] 204.969 µs (15.3%)
tracing 1.475 ms [1.451 ms, 1.499 ms] 134.493 µs (10.0%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.26.0-SNAPSHOT~74be0bda04, baseline=1.26.0-SNAPSHOT~7a1a4cac29
    dateFormat X
    axisFormat %s
section baseline
no_agent (365.814 µs) : 346, 386
.   : milestone, 366,
iast (480.963 µs) : 460, 502
.   : milestone, 481,
iast_FULL (537.478 µs) : 517, 558
.   : milestone, 537,
iast_INACTIVE (449.335 µs) : 429, 470
.   : milestone, 449,
iast_TELEMETRY_OFF (463.572 µs) : 443, 484
.   : milestone, 464,
tracing (450.557 µs) : 429, 472
.   : milestone, 451,
section candidate
no_agent (364.381 µs) : 344, 385
.   : milestone, 364,
iast (467.547 µs) : 447, 488
.   : milestone, 468,
iast_FULL (534.574 µs) : 514, 555
.   : milestone, 535,
iast_INACTIVE (449.581 µs) : 428, 471
.   : milestone, 450,
iast_TELEMETRY_OFF (474.293 µs) : 453, 496
.   : milestone, 474,
tracing (440.289 µs) : 420, 461
.   : milestone, 440,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 365.814 µs [345.918 µs, 385.71 µs] -
iast 480.963 µs [459.78 µs, 502.145 µs] 115.149 µs (31.5%)
iast_FULL 537.478 µs [516.958 µs, 557.999 µs] 171.665 µs (46.9%)
iast_INACTIVE 449.335 µs [428.538 µs, 470.131 µs] 83.521 µs (22.8%)
iast_TELEMETRY_OFF 463.572 µs [443.016 µs, 484.127 µs] 97.758 µs (26.7%)
tracing 450.557 µs [428.745 µs, 472.369 µs] 84.743 µs (23.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 364.381 µs [344.043 µs, 384.719 µs] -
iast 467.547 µs [447.387 µs, 487.707 µs] 103.166 µs (28.3%)
iast_FULL 534.574 µs [513.75 µs, 555.399 µs] 170.193 µs (46.7%)
iast_INACTIVE 449.581 µs [428.263 µs, 470.898 µs] 85.199 µs (23.4%)
iast_TELEMETRY_OFF 474.293 µs [452.83 µs, 495.755 µs] 109.911 µs (30.2%)
tracing 440.289 µs [419.93 µs, 460.647 µs] 75.907 µs (20.8%)

@ValentinZakharov ValentinZakharov marked this pull request as ready for review December 18, 2023 00:12
@ValentinZakharov ValentinZakharov requested a review from a team as a code owner December 18, 2023 00:12
@ValentinZakharov ValentinZakharov merged commit ce0c8c4 into master Dec 18, 2023
11 checks passed
@ValentinZakharov ValentinZakharov deleted the vzakharov/mocks_to_stubs_iast branch December 18, 2023 15:30
@github-actions github-actions bot added this to the 1.26.0 milestone Dec 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@smola smola smola approved these changes

@cataphract cataphract Awaiting requested review from cataphract cataphract is a code owner automatically assigned from DataDog/asm-java

@manuel-alvarez-alvarez manuel-alvarez-alvarez Awaiting requested review from manuel-alvarez-alvarez manuel-alvarez-alvarez is a code owner automatically assigned from DataDog/asm-java

Assignees

@ValentinZakharov ValentinZakharov

Labels
comp: asm iast Application Security Management (IAST) comp: testing Testing tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
1.26.0
Development

Successfully merging this pull request may close these issues.
2 participants
@ValentinZakharov @smola