Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions docs/component_deduplication.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Component Deduplication

Read [NodeJS internals](nodejs_internals.md) first.

NPM does the needed graph de-duplications internally already when it generates the affective module layout in the file-system.
See [`npm dedupe` docs](https://github.com/npm/cli/blob/latest/docs/lib/content/commands/npm-dedupe.md).

Idea: Additional logic how module de-deduplication could be done will come to the conclusion that
it is either invalid per definition, due to the previously described rules of graph/node identity that applies here,
or that it is unnecessary, because it was already done by NPM.

This idea shall be falsified.
See [Milestone: after-the-fact component deduplication](https://github.com/CycloneDX/cyclonedx-node-npm/milestone/2)
See [Discussion: describe how component de-duplication works](https://github.com/CycloneDX/cyclonedx-node-npm/discussions/307)
52 changes: 52 additions & 0 deletions docs/nodejs_internals.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# _NodeJS_ Internals

## Basics

A package might have a name, a version, and dependencies.
This information is usually stored in a `package.json` file.

A package might have submodules or subpackages.
These are usually stored in a `node_modules` folder next to the `package.json`.

## How _NodeJS_'s module/package resolution works

_NodeJS_'s module/package system is file-system based. It works regardless of package dependencies.
When code in package `foo` tries to use/require/access code from a different package `bar`,
then _NodeJS_ will look in `foo`'s own/direct `node_modules` folder.
If it did not find any `bar` there, then NodeJS traverses all folders upwards and does the same lookup there,
until it finds any `bar`.
This file-based loading behavior happens regardless of components' "dependency graph".
This loading behavior is - as described - not flat but hierarchical.

See [NodeJS docs](https://nodejs.org/api/packages.html#introduction)

## Implications

Based on this module resolution system it might appear that one complex tree might have multiple individual
instances of module `bar`.
Each of these instances might have a different content.
If two of these instances had equal content - on a module basis - they are still not the same module,
as their own `node_modules` might be different, which causes submodules being not the same.
If two of these instances had equal content - on a module basis - they are still not the same module,
as their position in the global module-resolution-tree might be different and therefore causes this very instances
to have different dependencies in the first place.
So two modules at different paths with equal file contents are most likely not the same.

Imagine each NodeJS-module as a node in a directed graph.
Each node has a set of properties. Properties represent file-content(checksums), module-name, and so on.
A directed edge in this graph represents module access in terms of node's module-resolution-system.
Therefore, the graph is not implicit, so that no transitive module-resolution is to be expected.
If a module A can (by any means) load module B, then a directed edge from A to B must exist.
This graph is per definition in the format of a directed tree.
In that graph two directed edges E1 and E2 are equal, if and only if:
a) E1's start-node equals E2's end-node, and
b) E1's end-node equals E2's end-node.
In that graph two nodes N1 and N2 are equal, if and only if:
a) N1's set of node properties is equal to N2's set of node properties, and
b) N1's set of directed edges is equal to N2's set of directed edges.

In graph following the given definition, two NodeJS-modules described as nodes can be de-duplicated if they are equal.

## Examples

Find an example in [Results Examples](result.md#examples-and-visualisation )
98 changes: 92 additions & 6 deletions docs/result.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,106 @@

This document will describe, how certain SBOM results should be deducted.

## Basics
## Preamble

A package might have a name, a version, and dependencies.
This information is usually stored in a `package.json` file.
Read [NodeJs Internals](nodejs_internals.md) first.

## Project -> `bom.metadata.component`
## Examples and Visualisation

Let the dependencies be in a non-range manner.
Let component `strip-ansi@7.0.1` require in a range manner: `ansi-regex@^6`.

### Dependency Graph

```mermaid
graph TB
R((application))
A((some-module))
B((other-module))
C1((strip-ansi<br/>7.0.1))
C2((strip-ansi<br/>7.0.1))
D1((ansi-regex<br/>6.0.1))
D2((ansi-regex<br/>6.0.0))
D3((ansi-regex<br/>5.0.1))
R --> A
R --> B
R --> D3
A --> C1
B --> C2
A --> D1
B --> D2
C1 --> D1
C2 --> D2
```

### A corresponding File-System Tree

```text
application
└── node_modules
├── ansi-regex <- @5.0.1
├── other-module
│ └── node_modules
│ ├── ansi-regex <- @6.0.0
│ └── strip-ansi <- @7.0.1
└── some-module
└── node_modules
├── ansi-regex <- @6.0.1
└── strip-ansi <- @7.0.1
```

### The corresponding Module Resolution Graph

```mermaid
graph LR
R((application))
A((some-module))
B((other-module))
C1((strip-ansi<br/>7.0.1))
C2((strip-ansi<br/>7.0.1))
D1((ansi-regex<br/>6.0.1))
D2((ansi-regex<br/>6.0.0))
D3((ansi-regex<br/>5.0.1))
R --> A
R --> B
R --> D3
A --- B
B --- D3
D3 --- A
A --- C1
B --- C2
A --- D1
B --- D2
C1 --- D1
C2 --- D2
C1 --> B
C2 --> A
D1 --> B
D2 --> A
```

### The resulting CycloneDX SBOM

... to be described

### Component De-duplication

NPM does the needed graph de-duplications internally already when it generates the affective module layout in the file system.
See [`npm dedupe` docs](https://github.com/npm/cli/blob/latest/docs/lib/content/commands/npm-dedupe.md).

See also: [Component De-duplication](component_deduplication.md)

----

## Project => `bom.metadata.component`

... to be described

## Package -> `...component`
## Package => `...component`

... to be described

## Bundled dependencies -> `...component.components`
## Bundled dependencies => `...component.components`

Some projects might have [`bundleDependencies`](https://docs.npmjs.com/cli/v8/configuring-npm/package-json#bundledependencies),
which means, that dependencies are part of a package
Expand Down