v2.41.0 proposal #3384

Qard · 2023-07-11T21:15:59Z

Features

profiling: Adapt to new pprof-nodejs API ([profiling] Adapt to new pprof-nodejs API #3368)
iast: Report IAST metrics via telemetry (Report IAST metrics via telemetry #2805)
ci-visibility: Change test.type in browser tests ([ci-visibility] Change test.type in browser tests #3358)

Bug Fixes

ci-visibility: Intelligent test runner: only report skipped flag if tests were actually skipped ([ci-visibility] Intelligent test runner: only report skipped flag if tests were actually skipped #3369)
tracer: add dc store binding polyfill and migrate http2 (add dc store binding polyfill and migrate http2 #3284)
tracer: grpc: Convert grpc to use tracing channel (Convert grpc to use tracing channel #3366)
ci-visibility: validate CI tags before sending them (validate CI tags before sending them #3310)
tracer: express: recognize regex with flags (express: recognize regex with flags #3301)

github-actions · 2023-07-11T21:17:07Z

Overall package size

Self size: 4.88 MB
Deduped: 68.09 MB
No deduping: 68.19 MB

Dependency sizes

name	version	self size	total size
@datadog/native-iast-taint-tracking	1.5.0	14.86 MB	14.86 MB
@datadog/native-appsec	3.2.0	13.38 MB	13.39 MB
@datadog/pprof	3.0.0	10.54 MB	11.39 MB
@datadog/native-metrics	1.6.0	7.88 MB	7.89 MB
protobufjs	7.2.4	2.74 MB	6.62 MB
@types/node	18.11.19	3.58 MB	3.58 MB
@datadog/native-iast-rewriter	2.0.1	2.09 MB	2.1 MB
@opentelemetry/core	1.3.1	784.66 kB	1.37 MB
@opentelemetry/api	1.4.1	780.32 kB	780.32 kB
msgpack-lite	0.1.26	201.16 kB	272.06 kB
opentracing	0.14.7	194.81 kB	194.81 kB
lru-cache	7.18.3	133.92 kB	133.92 kB
semver	7.5.3	93.39 kB	123.79 kB
@datadog/sketches-js	2.1.0	109.9 kB	109.9 kB
lodash.sortby	4.7.0	75.76 kB	75.76 kB
ipaddr.js	2.0.1	59.52 kB	59.52 kB
ignore	5.2.4	51.22 kB	51.22 kB
int64-buffer	0.1.10	49.18 kB	49.18 kB
import-in-the-middle	1.3.5	34.34 kB	38.81 kB
istanbul-lib-coverage	3.2.0	29.34 kB	29.34 kB
retry	0.10.1	27.44 kB	27.44 kB
lodash.uniq	4.5.0	25.01 kB	25.01 kB
limiter	1.1.5	23.17 kB	23.17 kB
lodash.kebabcase	4.1.1	17.75 kB	17.75 kB
node-abort-controller	3.1.1	16.89 kB	16.89 kB
lodash.pick	4.4.0	16.33 kB	16.33 kB
crypto-randomuuid	1.0.0	11.18 kB	11.18 kB
diagnostics_channel	1.1.0	7.07 kB	7.07 kB
path-to-regexp	0.1.7	6.78 kB	6.78 kB
koalas	1.0.2	6.47 kB	6.47 kB
methods	1.1.2	5.29 kB	5.29 kB
module-details-from-path	1.0.3	4.47 kB	4.47 kB

_{🤖 This report was automatically generated by heaviest-objects-in-the-universe}

codecov · 2023-07-11T21:18:35Z

Codecov Report

Merging #3384 (411b885) into v2.x (fbc2899) will decrease coverage by 8.56%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             v2.x    #3384      +/-   ##
==========================================
- Coverage   93.51%   84.95%   -8.56%     
==========================================
  Files         222      212      -10     
  Lines        8829     8415     -414     
  Branches        0       33      +33     
==========================================
- Hits         8256     7149    -1107     
- Misses        573     1266     +693

Impacted Files	Coverage Δ
integration-tests/ci-visibility/test/sum.js	`69.19% <ø> (ø)`
packages/datadog-instrumentations/src/mocha.js	`0.00% <0.00%> (-98.64%)`	⬇️
packages/datadog-plugin-fastify/src/index.js	`100.00% <ø> (ø)`

... and 332 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

* add dc store binding polyfill and migrate http2 * fix error handling for synchronous exceptions * OutboundPlugin#tagPeerService() --------- Co-authored-by: Thomas Hunter II <[email protected]>

* ciapp-3571: add url validator * ciapp-3571: add url validator and change tests from ci-spec * fix variables to follow camelcase * Remove valid import Co-authored-by: Juan Antonio Fernández de Alba <[email protected]> * Remove unnecessary check Co-authored-by: Juan Antonio Fernández de Alba <[email protected]> * Remove unnecessary check for CI_PIPELINE_URL Co-authored-by: Juan Antonio Fernández de Alba <[email protected]> * Format typo Co-authored-by: Juan Antonio Fernández de Alba <[email protected]> * Use validateMetadata return directly Co-authored-by: Juan Antonio Fernández de Alba <[email protected]> * use new ci-spec JSONs * fix spacing and sort imports * add test to validateMetadata * Update JSONs and refactor URL checking * Update JSONs and refactor URL checking * Add URL validation tests * Apply suggestions from code review Co-authored-by: Juan Antonio Fernández de Alba <[email protected]> * Add tag to test suite --------- Co-authored-by: Juan Antonio Fernández de Alba <[email protected]>

* Add framework type checking * Improve code readability Co-authored-by: Juan Antonio Fernández de Alba <[email protected]> * Fix imports * Fix styling Co-authored-by: Juan Antonio Fernández de Alba <[email protected]> * Add cypress and playwright tests * Change browser conditions to detect all playwright instances * Fix include string * Fix some suites still being reported as test type --------- Co-authored-by: Juan Antonio Fernández de Alba <[email protected]>

* Remove cpu-experimental profiler * Adapt to API changes in pprof-nodejs

…tests were actually skipped (#3369)

Co-authored-by: Ugaitz Urien <[email protected]>

* Convert grpc to use tracing channel * Allow addError to accept explicit span rather than delegating to activeSpan

juan-fernandez

looks good from ci visibility's perspective. Only feedback is to change "civis" to ci-visibility in the release notes

Supplying a `DD_TRACE_PEER_SERVICE_MAPPING` allows users to modify the `peer.service` tag in the same manner that `DD_SERVICE_MAPPING` does for service name. If the `peer.service` value was matched by the mapping, the original value of the tag before remapping is reported in `_dd.peer.service.remapped_from`.

pr-commenter · 2023-07-13T23:12:14Z

Benchmarks

Benchmark execution time: 2023-07-14 00:41:29

Comparing candidate commit c959a66 in PR branch v2.41.0-proposal with baseline commit 25579aa in branch v2.x.

Found 54 performance improvements and 0 performance regressions! Performance is the same for 648 metrics, 33 unstable metrics.

scenario:appsec-iast-no-vulnerability-control-12

🟩 cpu_user_time [-132.605ms; -109.556ms] or [-20.791%; -17.177%]
🟩 execution_time [-176.469ms; -151.063ms] or [-21.908%; -18.754%]
🟩 instructions [+0G instructions; +0G instructions] or [-16.803%; -16.582%]
🟩 max_rss_usage [-6.630KB; -6.357KB] or [-10.496%; -10.064%]

scenario:appsec-iast-no-vulnerability-control-14

🟩 cpu_user_time [-157.724ms; -136.926ms] or [-24.181%; -20.992%]
🟩 execution_time [-195.964ms; -172.717ms] or [-23.960%; -21.118%]
🟩 instructions [+0G instructions; +0G instructions] or [-17.592%; -17.307%]
🟩 max_rss_usage [-6.449KB; -6.030KB] or [-10.039%; -9.387%]

scenario:appsec-iast-no-vulnerability-control-16

🟩 cpu_user_time [-144.655ms; -124.313ms] or [-22.700%; -19.508%]
🟩 execution_time [-174.390ms; -139.626ms] or [-21.957%; -17.580%]
🟩 instructions [+0G instructions; +0G instructions] or [-15.933%; -15.715%]
🟩 max_rss_usage [-6.432KB; -5.912KB] or [-9.090%; -8.356%]

scenario:appsec-iast-no-vulnerability-iast-enabled-always-active-12

🟩 cpu_user_time [-0.245s; -0.193s] or [-6.427%; -5.072%]
🟩 instructions [+0G instructions; +0G instructions] or [-20.725%; -20.054%]

scenario:appsec-iast-no-vulnerability-iast-enabled-always-active-14

🟩 cpu_user_time [-0.239s; -0.182s] or [-7.392%; -5.603%]
🟩 instructions [+0G instructions; +0G instructions] or [-22.501%; -21.721%]

scenario:appsec-iast-no-vulnerability-iast-enabled-always-active-16

🟩 instructions [+0G instructions; +0G instructions] or [-23.717%; -23.124%]

scenario:appsec-iast-no-vulnerability-iast-enabled-default-config-12

🟩 instructions [+0G instructions; +0G instructions] or [-16.201%; -15.367%]

scenario:appsec-iast-no-vulnerability-iast-enabled-default-config-14

🟩 instructions [+0G instructions; +0G instructions] or [-18.178%; -17.136%]

scenario:appsec-iast-no-vulnerability-iast-enabled-default-config-16

🟩 instructions [+0G instructions; +0G instructions] or [-19.990%; -18.702%]

scenario:appsec-iast-with-vulnerability-control-12

🟩 cpu_user_time [-0.403s; -0.355s] or [-34.612%; -30.501%]
🟩 execution_time [-0.585s; -0.520s] or [-35.564%; -31.582%]
🟩 instructions [+0G instructions; +0G instructions] or [-31.870%; -31.359%]
🟩 max_rss_usage [-4.153KB; -3.755KB] or [-6.158%; -5.567%]

scenario:appsec-iast-with-vulnerability-control-14

🟩 cpu_user_time [-0.407s; -0.361s] or [-34.739%; -30.758%]
🟩 execution_time [-0.604s; -0.537s] or [-36.004%; -32.009%]
🟩 instructions [+0G instructions; +0G instructions] or [-32.152%; -31.719%]
🟩 max_rss_usage [-4.548KB; -3.986KB] or [-6.699%; -5.871%]

scenario:appsec-iast-with-vulnerability-control-16

🟩 cpu_user_time [-0.376s; -0.330s] or [-32.211%; -28.323%]
🟩 execution_time [-0.608s; -0.536s] or [-35.400%; -31.231%]
🟩 instructions [+0G instructions; +0G instructions] or [-31.946%; -31.496%]
🟩 max_rss_usage [-4.640KB; -4.172KB] or [-6.136%; -5.517%]

scenario:appsec-iast-with-vulnerability-iast-enabled-always-active-12

🟩 cpu_user_time [-0.529s; -0.376s] or [-11.252%; -7.990%]
🟩 execution_time [-0.936s; -0.711s] or [-16.621%; -12.634%]
🟩 instructions [-1G instructions; -1G instructions] or [-34.350%; -33.958%]
🟩 max_rss_usage [-11.379KB; -9.444KB] or [-9.818%; -8.149%]

scenario:appsec-iast-with-vulnerability-iast-enabled-always-active-14

🟩 cpu_user_time [-0.571s; -0.437s] or [-13.786%; -10.556%]
🟩 execution_time [-0.937s; -0.726s] or [-18.693%; -14.477%]
🟩 instructions [-1G instructions; -1G instructions] or [-35.591%; -35.129%]
🟩 max_rss_usage [-10.660KB; -8.282KB] or [-9.585%; -7.447%]

scenario:appsec-iast-with-vulnerability-iast-enabled-always-active-16

🟩 cpu_user_time [-0.543s; -0.381s] or [-13.066%; -9.167%]
🟩 execution_time [-0.933s; -0.708s] or [-18.457%; -14.016%]
🟩 instructions [-1G instructions; -1G instructions] or [-35.104%; -34.456%]
🟩 max_rss_usage [-9.609KB; -6.361KB] or [-8.065%; -5.339%]

scenario:appsec-iast-with-vulnerability-iast-enabled-default-config-12

🟩 cpu_user_time [-0.508s; -0.437s] or [-11.083%; -9.542%]
🟩 execution_time [-0.775s; -0.588s] or [-14.443%; -10.960%]
🟩 instructions [-1G instructions; -1G instructions] or [-31.824%; -30.215%]

scenario:appsec-iast-with-vulnerability-iast-enabled-default-config-14

🟩 cpu_user_time [-0.514s; -0.431s] or [-12.922%; -10.838%]
🟩 execution_time [-0.809s; -0.622s] or [-16.955%; -13.036%]
🟩 instructions [-1G instructions; -1G instructions] or [-32.429%; -30.903%]

scenario:appsec-iast-with-vulnerability-iast-enabled-default-config-16

🟩 cpu_user_time [-0.490s; -0.422s] or [-12.123%; -10.440%]
🟩 execution_time [-0.865s; -0.661s] or [-17.872%; -13.672%]
🟩 instructions [-1G instructions; -1G instructions] or [-32.927%; -31.746%]

scenario:log-without-log-16

🟩 execution_time [-18.907ms; -17.420ms] or [-5.448%; -5.019%]

Qard added the semver-minor label Jul 11, 2023

Qard requested review from a team as code owners July 11, 2023 21:16

rochdev and others added 9 commits July 11, 2023 14:40

add dc store binding polyfill and migrate http2 (#3284)

9348f7f

* add dc store binding polyfill and migrate http2 * fix error handling for synchronous exceptions * OutboundPlugin#tagPeerService() --------- Co-authored-by: Thomas Hunter II <[email protected]>

express: recognize regex with flags (#3301)

66c23a7

dep: bump protobufjs to 7.2.4 (#3374)

def00f0

[profiling] Adapt to new pprof-nodejs API (#3368)

791d33a

* Remove cpu-experimental profiler * Adapt to API changes in pprof-nodejs

[ci-visibility] Intelligent test runner: only report skipped flag if …

e6dc87d

…tests were actually skipped (#3369)

Report IAST metrics via telemetry (#2805)

9681f51

Co-authored-by: Ugaitz Urien <[email protected]>

Convert grpc to use tracing channel (#3366)

96efa33

* Convert grpc to use tracing channel * Allow addError to accept explicit span rather than delegating to activeSpan

Qard force-pushed the v2.41.0-proposal branch from e3c6b99 to 7475b20 Compare July 11, 2023 21:40

juan-fernandez previously approved these changes Jul 12, 2023

View reviewed changes

uurien previously approved these changes Jul 12, 2023

View reviewed changes

Qard force-pushed the v2.41.0-proposal branch from 7475b20 to ea0f171 Compare July 12, 2023 20:37

tlhunter previously approved these changes Jul 13, 2023

View reviewed changes

tlhunter and others added 4 commits July 13, 2023 15:15

docs: add link to openai docs (#3381)

287e7d8

Reduce times in appsec-iast tests (#3354)

72911a2

benchmarks: add node 12, 14, 20, upgrade node 14, 16 (#3388)

b88ba1f

Qard dismissed stale reviews from tlhunter, juan-fernandez, and uurien via 83990b9 July 13, 2023 22:15

Qard force-pushed the v2.41.0-proposal branch from ea0f171 to 83990b9 Compare July 13, 2023 22:15

tlhunter and others added 2 commits July 13, 2023 17:22

test v2.x against node 12, 14, 16 (#3390)

a6c6763

v2.41.0

c959a66

Qard force-pushed the v2.41.0-proposal branch from 411b885 to c959a66 Compare July 14, 2023 00:22

tlhunter approved these changes Jul 14, 2023

View reviewed changes

uurien approved these changes Jul 14, 2023

View reviewed changes

Qard merged commit 4b02e9e into v2.x Jul 14, 2023

Qard deleted the v2.41.0-proposal branch July 14, 2023 16:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

v2.41.0 proposal #3384

v2.41.0 proposal #3384

Uh oh!

Qard commented Jul 11, 2023 •

edited

Loading

Uh oh!

github-actions bot commented Jul 11, 2023 •

edited

Loading

Uh oh!

codecov bot commented Jul 11, 2023 •

edited

Loading

Uh oh!

juan-fernandez left a comment

Uh oh!

pr-commenter bot commented Jul 13, 2023 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

v2.41.0 proposal #3384

v2.41.0 proposal #3384

Uh oh!

Conversation

Qard commented Jul 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Features

Bug Fixes

Uh oh!

github-actions bot commented Jul 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overall package size

Dependency sizes

Uh oh!

codecov bot commented Jul 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

juan-fernandez left a comment

Choose a reason for hiding this comment

Uh oh!

pr-commenter bot commented Jul 13, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

scenario:appsec-iast-no-vulnerability-control-12

scenario:appsec-iast-no-vulnerability-control-14

scenario:appsec-iast-no-vulnerability-control-16

scenario:appsec-iast-no-vulnerability-iast-enabled-always-active-12

scenario:appsec-iast-no-vulnerability-iast-enabled-always-active-14

scenario:appsec-iast-no-vulnerability-iast-enabled-always-active-16

scenario:appsec-iast-no-vulnerability-iast-enabled-default-config-12

scenario:appsec-iast-no-vulnerability-iast-enabled-default-config-14

scenario:appsec-iast-no-vulnerability-iast-enabled-default-config-16

scenario:appsec-iast-with-vulnerability-control-12

scenario:appsec-iast-with-vulnerability-control-14

scenario:appsec-iast-with-vulnerability-control-16

scenario:appsec-iast-with-vulnerability-iast-enabled-always-active-12

scenario:appsec-iast-with-vulnerability-iast-enabled-always-active-14

scenario:appsec-iast-with-vulnerability-iast-enabled-always-active-16

scenario:appsec-iast-with-vulnerability-iast-enabled-default-config-12

scenario:appsec-iast-with-vulnerability-iast-enabled-default-config-14

scenario:appsec-iast-with-vulnerability-iast-enabled-default-config-16

scenario:log-without-log-16

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

Qard commented Jul 11, 2023 •

edited

Loading

github-actions bot commented Jul 11, 2023 •

edited

Loading

codecov bot commented Jul 11, 2023 •

edited

Loading

pr-commenter bot commented Jul 13, 2023 •

edited

Loading