diff --git a/docs/component_deduplication.md b/docs/component_deduplication.md new file mode 100644 index 000000000..0bda64222 --- /dev/null +++ b/docs/component_deduplication.md @@ -0,0 +1,14 @@ +# Component Deduplication + +Read [NodeJS internals](nodejs_internals.md) first. + +NPM does the needed graph de-duplications internally already when it generates the affective module layout in the file-system. +See [`npm dedupe` docs](https://github.com/npm/cli/blob/latest/docs/lib/content/commands/npm-dedupe.md). + +Idea: Additional logic how module de-deduplication could be done will come to the conclusion that +it is either invalid per definition, due to the previously described rules of graph/node identity that applies here, +or that it is unnecessary, because it was already done by NPM. + +This idea shall be falsified. +See [Milestone: after-the-fact component deduplication](https://github.com/CycloneDX/cyclonedx-node-npm/milestone/2) +See [Discussion: describe how component de-duplication works](https://github.com/CycloneDX/cyclonedx-node-npm/discussions/307) diff --git a/docs/nodejs_internals.md b/docs/nodejs_internals.md new file mode 100644 index 000000000..55b6a45d4 --- /dev/null +++ b/docs/nodejs_internals.md @@ -0,0 +1,52 @@ +# _NodeJS_ Internals + +## Basics + +A package might have a name, a version, and dependencies. +This information is usually stored in a `package.json` file. + +A package might have submodules or subpackages. +These are usually stored in a `node_modules` folder next to the `package.json`. + +## How _NodeJS_'s module/package resolution works + +_NodeJS_'s module/package system is file-system based. It works regardless of package dependencies. +When code in package `foo` tries to use/require/access code from a different package `bar`, +then _NodeJS_ will look in `foo`'s own/direct `node_modules` folder. +If it did not find any `bar` there, then NodeJS traverses all folders upwards and does the same lookup there, +until it finds any `bar`. +This file-based loading behavior happens regardless of components' "dependency graph". +This loading behavior is - as described - not flat but hierarchical. + +See [NodeJS docs](https://nodejs.org/api/packages.html#introduction) + +## Implications + +Based on this module resolution system it might appear that one complex tree might have multiple individual +instances of module `bar`. +Each of these instances might have a different content. +If two of these instances had equal content - on a module basis - they are still not the same module, +as their own `node_modules` might be different, which causes submodules being not the same. +If two of these instances had equal content - on a module basis - they are still not the same module, +as their position in the global module-resolution-tree might be different and therefore causes this very instances +to have different dependencies in the first place. +So two modules at different paths with equal file contents are most likely not the same. + +Imagine each NodeJS-module as a node in a directed graph. +Each node has a set of properties. Properties represent file-content(checksums), module-name, and so on. +A directed edge in this graph represents module access in terms of node's module-resolution-system. +Therefore, the graph is not implicit, so that no transitive module-resolution is to be expected. +If a module A can (by any means) load module B, then a directed edge from A to B must exist. +This graph is per definition in the format of a directed tree. +In that graph two directed edges E1 and E2 are equal, if and only if: +a) E1's start-node equals E2's end-node, and +b) E1's end-node equals E2's end-node. +In that graph two nodes N1 and N2 are equal, if and only if: +a) N1's set of node properties is equal to N2's set of node properties, and +b) N1's set of directed edges is equal to N2's set of directed edges. + +In graph following the given definition, two NodeJS-modules described as nodes can be de-duplicated if they are equal. + +## Examples + +Find an example in [Results Examples](result.md#examples-and-visualisation ) diff --git a/docs/result.md b/docs/result.md index 4f86aadac..8e48af1fa 100644 --- a/docs/result.md +++ b/docs/result.md @@ -2,20 +2,106 @@ This document will describe, how certain SBOM results should be deducted. -## Basics +## Preamble -A package might have a name, a version, and dependencies. -This information is usually stored in a `package.json` file. +Read [NodeJs Internals](nodejs_internals.md) first. -## Project -> `bom.metadata.component` +## Examples and Visualisation + +Let the dependencies be in a non-range manner. +Let component `strip-ansi@7.0.1` require in a range manner: `ansi-regex@^6`. + +### Dependency Graph + +```mermaid +graph TB + R((application)) + A((some-module)) + B((other-module)) + C1((strip-ansi
7.0.1)) + C2((strip-ansi
7.0.1)) + D1((ansi-regex
6.0.1)) + D2((ansi-regex
6.0.0)) + D3((ansi-regex
5.0.1)) + R --> A + R --> B + R --> D3 + A --> C1 + B --> C2 + A --> D1 + B --> D2 + C1 --> D1 + C2 --> D2 +``` + +### A corresponding File-System Tree + +```text +application +└── node_modules + ├── ansi-regex <- @5.0.1 + ├── other-module + │ └── node_modules + │ ├── ansi-regex <- @6.0.0 + │ └── strip-ansi <- @7.0.1 + └── some-module + └── node_modules + ├── ansi-regex <- @6.0.1 + └── strip-ansi <- @7.0.1 +``` + +### The corresponding Module Resolution Graph + +```mermaid +graph LR + R((application)) + A((some-module)) + B((other-module)) + C1((strip-ansi
7.0.1)) + C2((strip-ansi
7.0.1)) + D1((ansi-regex
6.0.1)) + D2((ansi-regex
6.0.0)) + D3((ansi-regex
5.0.1)) + R --> A + R --> B + R --> D3 + A --- B + B --- D3 + D3 --- A + A --- C1 + B --- C2 + A --- D1 + B --- D2 + C1 --- D1 + C2 --- D2 + C1 --> B + C2 --> A + D1 --> B + D2 --> A +``` + +### The resulting CycloneDX SBOM + +... to be described + +### Component De-duplication + +NPM does the needed graph de-duplications internally already when it generates the affective module layout in the file system. +See [`npm dedupe` docs](https://github.com/npm/cli/blob/latest/docs/lib/content/commands/npm-dedupe.md). + +See also: [Component De-duplication](component_deduplication.md) + +---- + +## Project => `bom.metadata.component` ... to be described -## Package -> `...component` +## Package => `...component` ... to be described -## Bundled dependencies -> `...component.components` +## Bundled dependencies => `...component.components` Some projects might have [`bundleDependencies`](https://docs.npmjs.com/cli/v8/configuring-npm/package-json#bundledependencies), which means, that dependencies are part of a package