Skip to content

Conversation

@uurien
Copy link
Collaborator

@uurien uurien commented Jul 6, 2023

What does this PR do?

Execute less iterations and request in iast test to reduce the time spent on testing.

Motivation

Sometimes benchmark timeout

@github-actions
Copy link

github-actions bot commented Jul 6, 2023

Overall package size

Self size: 4.87 MB
Deduped: 57.86 MB
No deduping: 57.95 MB

Dependency sizes

name version self size total size
@datadog/native-iast-taint-tracking 1.5.0 14.86 MB 14.86 MB
@datadog/native-appsec 3.2.0 13.38 MB 13.39 MB
@datadog/pprof 3.0.0 10.54 MB 11.39 MB
protobufjs 7.2.4 2.74 MB 6.52 MB
@datadog/native-iast-rewriter 2.0.1 2.09 MB 2.1 MB
@opentelemetry/core 1.14.0 872.87 kB 1.47 MB
@datadog/native-metrics 2.0.0 898.77 kB 1.3 MB
@opentelemetry/api 1.4.1 780.32 kB 780.32 kB
msgpack-lite 0.1.26 201.16 kB 281.59 kB
opentracing 0.14.7 194.81 kB 194.81 kB
semver 7.5.3 93.39 kB 123.79 kB
@datadog/sketches-js 2.1.0 109.9 kB 109.9 kB
lodash.sortby 4.7.0 75.76 kB 75.76 kB
lru-cache 7.14.0 74.95 kB 74.95 kB
ipaddr.js 2.0.1 59.52 kB 59.52 kB
int64-buffer 0.1.10 49.18 kB 49.18 kB
ignore 5.2.0 48.87 kB 48.87 kB
import-in-the-middle 1.3.5 34.34 kB 38.81 kB
istanbul-lib-coverage 3.2.0 29.34 kB 29.34 kB
retry 0.10.1 27.44 kB 27.44 kB
lodash.uniq 4.5.0 25.01 kB 25.01 kB
limiter 1.1.5 23.17 kB 23.17 kB
lodash.kebabcase 4.1.1 17.75 kB 17.75 kB
lodash.pick 4.4.0 16.33 kB 16.33 kB
node-abort-controller 3.0.1 14.33 kB 14.33 kB
crypto-randomuuid 1.0.0 11.18 kB 11.18 kB
diagnostics_channel 1.1.0 7.07 kB 7.07 kB
path-to-regexp 0.1.7 6.78 kB 6.78 kB
koalas 1.0.2 6.47 kB 6.47 kB
methods 1.1.2 5.29 kB 5.29 kB
module-details-from-path 1.0.3 4.47 kB 4.47 kB

🤖 This report was automatically generated by heaviest-objects-in-the-universe

@codecov
Copy link

codecov bot commented Jul 6, 2023

Codecov Report

Merging #3354 (07a0d45) into master (ea81490) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #3354   +/-   ##
=======================================
  Coverage   84.71%   84.71%           
=======================================
  Files         211      211           
  Lines        8256     8256           
  Branches       33       33           
=======================================
  Hits         6994     6994           
  Misses       1262     1262           

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@pr-commenter
Copy link

pr-commenter bot commented Jul 6, 2023

Benchmarks

Benchmark execution time: 2023-07-13 07:09:10

Comparing candidate commit 07a0d45 in PR branch ugaitz/reduce-iast-benchmark-time with baseline commit ea81490 in branch master.

Found 36 performance improvements and 1 performance regressions! Performance is the same for 431 metrics, 24 unstable metrics.

scenario:appsec-iast-no-vulnerability-control-16

  • 🟥 cpu_usage_percentage [+5.077%; +7.841%]
  • 🟩 cpu_user_time [-142.779ms; -124.748ms] or [-23.459%; -20.496%]
  • 🟩 execution_time [-214.338ms; -193.081ms] or [-28.180%; -25.386%]
  • 🟩 instructions [+0G instructions; +0G instructions] or [-16.864%; -16.768%]
  • 🟩 max_rss_usage [-7.101KB; -6.637KB] or [-10.176%; -9.511%]

scenario:appsec-iast-no-vulnerability-control-18

  • 🟩 cpu_user_time [-130.140ms; -115.006ms] or [-20.917%; -18.485%]
  • 🟩 execution_time [-180.080ms; -156.150ms] or [-23.728%; -20.575%]
  • 🟩 instructions [+0G instructions; +0G instructions] or [-14.719%; -14.500%]
  • 🟩 max_rss_usage [-8.350KB; -8.002KB] or [-10.287%; -9.859%]

scenario:appsec-iast-no-vulnerability-iast-enabled-always-active-16

  • 🟩 instructions [+0G instructions; +0G instructions] or [-23.880%; -23.331%]

scenario:appsec-iast-no-vulnerability-iast-enabled-always-active-18

  • 🟩 cpu_user_time [-0.192s; -0.162s] or [-16.737%; -14.126%]
  • 🟩 execution_time [-0.210s; -0.172s] or [-16.209%; -13.310%]
  • 🟩 instructions [+0G instructions; +0G instructions] or [-20.840%; -20.086%]

scenario:appsec-iast-no-vulnerability-iast-enabled-default-config-16

  • 🟩 instructions [+0G instructions; +0G instructions] or [-20.066%; -19.011%]

scenario:appsec-iast-no-vulnerability-iast-enabled-default-config-18

  • 🟩 cpu_user_time [-0.154s; -0.132s] or [-14.172%; -12.119%]
  • 🟩 execution_time [-0.191s; -0.161s] or [-15.357%; -12.968%]
  • 🟩 instructions [+0G instructions; +0G instructions] or [-15.819%; -14.727%]

scenario:appsec-iast-with-vulnerability-control-16

  • 🟩 cpu_user_time [-0.351s; -0.321s] or [-31.640%; -28.915%]
  • 🟩 execution_time [-0.613s; -0.559s] or [-37.439%; -34.147%]
  • 🟩 instructions [+0G instructions; +0G instructions] or [-32.137%; -31.905%]

scenario:appsec-iast-with-vulnerability-control-18

  • 🟩 cpu_user_time [-0.367s; -0.331s] or [-32.165%; -29.026%]
  • 🟩 execution_time [-0.617s; -0.542s] or [-36.352%; -31.950%]
  • 🟩 instructions [+0G instructions; +0G instructions] or [-32.063%; -31.735%]

scenario:appsec-iast-with-vulnerability-iast-enabled-always-active-16

  • 🟩 cpu_user_time [-0.510s; -0.373s] or [-12.354%; -9.026%]
  • 🟩 execution_time [-0.901s; -0.697s] or [-17.922%; -13.874%]
  • 🟩 instructions [-1G instructions; -1G instructions] or [-35.146%; -34.463%]
  • 🟩 max_rss_usage [-14.421KB; -11.176KB] or [-11.645%; -9.025%]

scenario:appsec-iast-with-vulnerability-iast-enabled-always-active-18

  • 🟩 cpu_user_time [-0.498s; -0.410s] or [-26.329%; -21.678%]
  • 🟩 execution_time [-0.910s; -0.717s] or [-32.867%; -25.904%]
  • 🟩 instructions [-1G instructions; -1G instructions] or [-35.335%; -34.916%]
  • 🟩 max_rss_usage [-9.325KB; -6.851KB] or [-7.743%; -5.689%]

scenario:appsec-iast-with-vulnerability-iast-enabled-default-config-16

  • 🟩 cpu_user_time [-0.510s; -0.444s] or [-12.691%; -11.046%]
  • 🟩 execution_time [-0.837s; -0.665s] or [-17.452%; -13.861%]
  • 🟩 instructions [-1G instructions; -1G instructions] or [-33.111%; -31.917%]

scenario:appsec-iast-with-vulnerability-iast-enabled-default-config-18

  • 🟩 cpu_user_time [-0.457s; -0.392s] or [-26.644%; -22.838%]
  • 🟩 execution_time [-0.810s; -0.650s] or [-32.381%; -25.983%]
  • 🟩 instructions [-1G instructions; -1G instructions] or [-33.195%; -31.651%]

@uurien uurien force-pushed the ugaitz/reduce-iast-benchmark-time branch from fc2010c to 49d2075 Compare July 12, 2023 13:34
@uurien uurien force-pushed the ugaitz/reduce-iast-benchmark-time branch from 49d2075 to 07a0d45 Compare July 13, 2023 07:00
@uurien uurien marked this pull request as ready for review July 13, 2023 07:00
@uurien uurien requested a review from a team as a code owner July 13, 2023 07:00
@uurien uurien merged commit d450941 into master Jul 13, 2023
@uurien uurien deleted the ugaitz/reduce-iast-benchmark-time branch July 13, 2023 07:30
Qard pushed a commit that referenced this pull request Jul 13, 2023
Qard pushed a commit that referenced this pull request Jul 13, 2023
Qard pushed a commit that referenced this pull request Jul 13, 2023
Qard pushed a commit that referenced this pull request Jul 14, 2023
Qard pushed a commit that referenced this pull request Jul 14, 2023
Qard pushed a commit that referenced this pull request Jul 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants