Skip to content
/ json-merger Public
forked from boschni/json-merger

Merge JSON files and objects with indicators like $import $remove $replace $merge

License

MIT license
0 stars 17 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

pactiotech/json-merger

BranchesTags
 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

120 Commits
__helpers__
__helpers__
 
 
__mocks__
__mocks__
 
 
__tests__
__tests__
 
 
bin
bin
 
 
src
src
 
 
.editorconfig
.editorconfig
 
 
.gitignore
.gitignore
 
 
.prettierignore
.prettierignore
 
 
.prettierrc.json
.prettierrc.json
 
 
LICENSE
LICENSE
 
 
README.md
README.md
 
 
package-lock.json
package-lock.json
 
 
package.json
package.json
 
 
tsconfig.json
tsconfig.json
 
 

Repository files navigation

json-merger

Merge JSON (or YAML) files and objects with operations like $import $remove $replace $merge and more.

Table of Contents:

API

.mergeFile(file: string, config?: Config)

javascript

var jsonMerger = require("json-merger");
var result = jsonMerger.mergeFile("a.json");

a.json:

{
  "$merge": {
    "source": {
      "$import": "b.json"
    },
    "with": {
      "prop1": {
        "$replace": {
          "prop1a": "this will replace b.json's property prop1"
        }
      },
      "prop2": {
        "prop2a": "this will merge with b.json's property prop2"
      }
    }
  }
}

b.json:

{
  "prop1": {
    "prop1b": "will be replaced"
  },
  "prop2": {
    "prop2b": "will be merged"
  }
}

result

{
  "prop1": {
    "prop1a": "this will replace b.json's property prop1"
  },
  "prop2": {
    "prop2a": "this will merge with b.json's property prop2",
    "prop2b": "will be merged"
  }
}

.mergeFiles(files: string[], config?: Config)

javascript

var jsonMerger = require("json-merger");
var result = jsonMerger.mergeFiles(["a.json", "b.json"]);

a.json:

{
  "a": "some value"
}

b.json:

{
  "b": "some other value"
}

result

{
  "a": "some value",
  "b": "some other value"
}

.mergeObject(object: object, config?: Config)

javascript

var jsonMerger = require("json-merger");

var object = {
  a: {
    aa: "some value",
  },
  b: {
    $import: "b.json",
  },
};

var result = jsonMerger.mergeObject(object);

b.json:

{
  "bb": "some other value"
}

result

{
  "a": {
    "aa": "some value"
  },
  "b": {
    "bb": "some other value"
  }
}

.mergeObjects(objects: object[], config?: Config)

javascript

var jsonMerger = require("json-merger");

var object1 = {
  a: [1, 1, 1, 1],
};

var object2 = {
  a: [2, 2],
};

var result = jsonMerger.mergeObjects([object1, object2]);

result

{
  "a": [2, 2, 1, 1]
}

Merger(config?: Config)

The actual Merger class is also exported. The other exports are just shortcut methods.

Using one Merger instance has some performance advantages because it will cache previously loaded and processed files.

javascript

var Merger = require("json-merger").Merger;

var merger = new Merger();

// the first call will load and process "a.json"
var result1 = merger.mergeFile("a.json");

// the second call will return the cached result
var result2 = merger.mergeFile("a.json");

// clear the caches
merger.clearCaches();

Config

interface Config {
  cwd?: string;
  errorOnFileNotFound?: boolean;
  errorOnRefNotFound?: boolean;
  operationPrefix?: string;
  params?: object;
  stringify?: boolean | "pretty";
  defaultArrayMergeOperation: "combine" | "replace" | "concat";
}

cwd: string

The current working directory when importing files. Defaults to process.cwd().

errorOnFileNotFound: boolean

Set this property to false to disable throwing errors when an imported file does not exist.

errorOnRefNotFound: boolean

Set this property to false to disable throwing errors when an JSON pointer or JSON path does not exist.

operationPrefix: string

Use this property to override the prefix to indicate a property is an operation like $import. The default prefix is $but it is possible to change this to for example@to use keywords like@import.

params: object

Object that will be available in $expression operations as $params variable.

stringify: boolean | "pretty"

Set this property to true to stringify the JSON result. Set the property to "pretty" if the output should be pretty printed.

defaultArrayMergeOperation: "combine" | "replace" | "concat"

Set this property to override default merge operation. Default value is set to "combine". Possible values are:

Operations

$import

Use $import to import other JSON or YAML files.

Files imported with $import are processed before the result is returned.

{
  "$import": "a.json"
}

JSON reference syntax is supported. The following example will import the first array item from the someArray property in a.json.

{
  "$import": "a.json#/someArray/0"
}

When defined as an array, $import will process and merge the files in order before returning the result.

{
  "$import": ["a.json", "b.yaml", "c.json"]
}

When importing a file it is also possible to provide a different $params object. Setting this property will override the Config.params property.

{
  "$import": {
    "path": "a.json",
    "params": {
      "prop": "some value that will be available in a.json as $params.prop"
    }
  }
}

The object syntax is also supported in an array.

{
  "$import": [
    {
      "path": "a.json",
      "params": {
        "prop": "value1"
      }
    },
    {
      "path": "a.json",
      "params": {
        "prop": "value2"
      }
    }
  ]
}

Use $include to process a file in the current scope.

$merge

Use the $merge operation to merge objects and arrays.

javascript

var result = jsonMerger.mergeFile("a.json");

a.json

{
  "$merge": {
    "source": {
      "a": {
        "aa": "some value"
      }
    },
    "with": {
      "a": {
        "bb": "some other value"
      }
    }
  }
}

result

{
  "a": {
    "aa": "some value",
    "bb": "some other value"
  }
}

Merging with other files

The $merge operation is often used with the $import operation to merge other files from within the JSON itself.

javascript

var result = jsonMerger.mergeFile("a.json");

a.json

{
  "$merge": {
    "source": {
      "$import": "b.json"
    },
    "with": {
      "a": {
        "bb": "some other value"
      }
    }
  }
}

b.json

{
  "a": {
    "aa": "some value"
  }
}

result

{
  "a": {
    "aa": "some value",
    "bb": "some other value"
  }
}

$remove

Use the $remove operation to remove properties and array items.

Remove object properties

javascript

var result = jsonMerger.mergeFiles(["a.json", "b.json"]);

a.json

{
  "prop1": {
    "prop1a": "some value"
  },
  "prop2": {
    "prop2a": "some other value"
  }
}

b.json

{
  "prop2": {
    "$remove": true
  }
}

result

{
  "prop1": {
    "prop1a": "some value"
  }
}

Remove array items

javascript

var result = jsonMerger.mergeFiles(["a.json", "b.json"]);

a.json

{
  "someArray": [1, 2, 3]
}

b.json

{
  "someArray": [
    {
      "$remove": true
    },
    {
      "$remove": true
    }
  ]
}

result

{
  "someArray": [3]
}

$replace

Use the $replace operation to replace properties and array items.

Replace object properties

javascript

var result = jsonMerger.mergeFiles(["a.json", "b.json"]);

a.json

{
  "prop1": {
    "prop1a": "some value"
  },
  "prop2": {
    "prop2a": "some other value"
  }
}

b.json

{
  "prop2": {
    "$replace": {
      "prop2b": "replaced value"
    }
  }
}

result

{
  "prop1": {
    "prop1a": "some value"
  },
  "prop2": {
    "prop2b": "replaced value"
  }
}

Replace array items

javascript

var result = jsonMerger.mergeFiles(["a.json", "b.json"]);

a.json

{
  "someArray": [
    {
      "a": 1
    },
    {
      "b": 2
    }
  ]
}

b.json

{
  "someArray": [
    {
      "$replace": {
        "c": 3
      }
    }
  ]
}

result

{
  "someArray": [
    {
      "c": 3
    },
    {
      "b": 2
    }
  ]
}

$concat

Use the $concat operation to concatenate two arrays.

javascript

var result = jsonMerger.mergeFiles(["a.json", "b.json"]);

a.json

{
  "someArray": [1]
}

b.json

{
  "someArray": {
    "$concat": [2]
  }
}

result

{
  "someArray": [1, 2]
}

$combine

Use the $combine operation to combine two arrays.

javascript

var result = jsonMerger.mergeFiles(["a.json", "b.json"]);

a.json

{
  "someArray": [1, 2, 3]
}

b.json

{
  "someArray": {
    "$combine": [3, 3]
  }
}

result

{
  "someArray": [3, 3, 3]
}

$append

Use the $append operation to append an item to an array.

javascript

var result = jsonMerger.mergeFiles(["a.json", "b.json"]);

a.json

{
  "someArray": [1, 2, 3]
}

b.json

{
  "someArray": [
    {
      "$append": 4
    }
  ]
}

result

{
  "someArray": [1, 2, 3, 4]
}

$prepend

Use the $prepend operation to prepend an item to an array.

javascript

var result = jsonMerger.mergeFiles(["a.json", "b.json"]);

a.json

{
  "someArray": [1, 2, 3]
}

b.json

{
  "someArray": [
    {
      "$prepend": 4
    }
  ]
}

result

{
  "someArray": [4, 1, 2, 3]
}

$insert

Use the $insert operation to insert an item to an array.

javascript

var result = jsonMerger.mergeFiles(["a.json", "b.json"]);

a.json

{
  "someArray": [1, 2, 3]
}

b.json

{
  "someArray": [
    {
      "$insert": {
        "index": 1,
        "value": 4
      }
    }
  ]
}

result

{
  "someArray": [1, 4, 2, 3]
}

Insert as last item

Set $insert.index to - to insert an item at the end of the array.

javascript

var result = jsonMerger.mergeFiles(["a.json", "b.json"]);

a.json

{
  "someArray": [1, 2, 3]
}

b.json

{
  "someArray": [
    {
      "$insert": {
        "index": "-",
        "value": 4
      }
    }
  ]
}

result

{
  "someArray": [1, 2, 3, 4]
}

Insert before the last item

A negative $insert.index can be used, indicating an offset from the end of the array.

javascript

var result = jsonMerger.mergeFiles(["a.json", "b.json"]);

a.json

{
  "someArray": [1, 2, 3]
}

b.json

{
  "someArray": [
    {
      "$insert": {
        "index": -1,
        "value": 4
      }
    }
  ]
}

result

{
  "someArray": [1, 2, 4, 3]
}

$match

Use the $match operation to search for a specific array item and merge with that item.

Match by index

Use $match.index to match an array item by index.

javascript

var result = jsonMerger.mergeFiles(["a.json", "b.json"]);

a.json

{
  "someArray": [1, 2, 3]
}

b.json

{
  "someArray": [
    {
      "$match": {
        "index": 1,
        "value": 4
      }
    }
  ]
}

result

{
  "someArray": [1, 4, 3]
}

Match by JSON pointer

Use $match.path to match an array item with a JSON pointer.

javascript

var result = jsonMerger.mergeFiles(["a.json", "b.json"]);

a.json

{
  "someArray": [1, 2, 3]
}

b.json

{
  "someArray": [
    {
      "$match": {
        "path": "/1",
        "value": 4
      }
    }
  ]
}

result

{
  "someArray": [1, 4, 3]
}

Match by JSON path query

Use $match.query to match an array item with a JSON path query. The following example will search for an array item containing the value 2 and merge it with the value 4.

javascript

var result = jsonMerger.mergeFiles(["a.json", "b.json"]);

a.json

{
  "someArray": [1, 2, 3]
}

b.json

{
  "someArray": [
    {
      "$match": {
        "query": "$[?(@ == 2)]",
        "value": 4
      }
    }
  ]
}

result

{
  "someArray": [1, 4, 3]
}

$move

Use the $move operation to move an array item.

javascript

var result = jsonMerger.mergeFiles(["a.json", "b.json"]);

a.json

{
  "someArray": [1, 2, 3]
}

b.json

{
  "someArray": [
    {
      "$move": 1
    }
  ]
}

result

{
  "someArray": [2, 1, 3]
}

Move a matched array item

Use the $match operation in conjunction with the $move operation to move a specific array item.

javascript

var result = jsonMerger.mergeFiles(["a.json", "b.json"]);

a.json

{
  "someArray": [1, 2, 3]
}

b.json

{
  "someArray": [
    {
      "$match": {
        "index": 0,
        "value": {
          "$move": 1
        }
      }
    }
  ]
}

result

{
  "someArray": [2, 1, 3]
}

Move a matched array item to the end

Use - as $move.index value to move an array item to the end.

javascript

var result = jsonMerger.mergeFiles(["a.json", "b.json"]);

a.json

{
  "someArray": [1, 2, 3]
}

b.json

{
  "someArray": [
    {
      "$match": {
        "index": 0,
        "value": {
          "$move": "-"
        }
      }
    }
  ]
}

result

{
  "someArray": [2, 3, 1]
}

Move and merge a matched array item

Use $move.value to not only move the item but also merge it with a value.

javascript

var result = jsonMerger.mergeFiles(["a.json", "b.json"]);

a.json

{
  "someArray": [
    {
      "a": 1
    },
    {
      "a": 2
    },
    {
      "a": 3
    }
  ]
}

b.json

{
  "someArray": [
    {
      "$match": {
        "query": "$[?(@.a == 3)]",
        "value": {
          "$move": {
            "index": 0,
            "value": {
              "b": 3
            }
          }
        }
      }
    }
  ]
}

result

{
  "someArray": [
    {
      "a": 3,
      "b": 3
    },
    {
      "a": 1
    },
    {
      "a": 2
    }
  ]
}

$select

Use the $select operation to select one or multiple values.

Be careful not to create an endless loop by selecting a parent property.

Select by JSON pointer

More information about JSON pointers can be found in the JSON pointer specification.

javascript

var result = jsonMerger.mergeFile("a.json");

a.json

{
  "prop": {
    "$select": "/otherProp"
  },
  "otherProp": "Should be the value of prop"
}

result

{
  "prop": "Should be the value of prop",
  "otherProp": "Should be the value of prop"
}

Use $select.query to select by JSON path query

More information about JSON path queries can be found in the JSON path documentation.

javascript

var result = jsonMerger.mergeFile("a.json");

a.json

{
  "prop": {
    "$select": {
      "query": "$.someArray[*]"
    }
  },
  "someArray": [1, 2, 3]
}

result

{
  "prop": 1,
  "someArray": [1, 2, 3]
}

Use $select.multiple to select multiple values

javascript

var result = jsonMerger.mergeFile("a.json");

a.json

{
  "prop": {
    "$select": {
      "query": "$.someArray[?(@ < 3)]",
      "multiple": true
    }
  },
  "someArray": [1, 2, 3]
}

result

{
  "prop": [1, 2]
}

Use $select.from to select from an object

javascript

var result = jsonMerger.mergeFile("a.json");

a.json

{
  "prop": {
    "$select": {
      "from": {
        "$import": "b.json"
      },
      "path": "/someArray/2"
    }
  }
}

b.json

{
  "someArray": [1, 2, 3]
}

result

{
  "prop": 3
}

$repeat

Use the $repeat operation to repeat a value.

Repeat with $repeat.to

operation

{
  "$repeat": {
    "from": 1,
    "to": 4,
    "value": "repeat"
  }
}

result

["repeat", "repeat", "repeat"]

Repeat with $repeat.through

The current value is available on the scope as $repeat.value variable.

operation

{
  "$repeat": {
    "from": 1,
    "through": 4,
    "value": {
      "$expression": "$repeat.value"
    }
  }
}

result

[1, 2, 3, 4]

Repeat with $repeat.step

operation

{
  "$repeat": {
    "from": 0,
    "through": 10,
    "step": 5,
    "value": {
      "$expression": "$repeat.value"
    }
  }
}

result

[0, 5, 10]

Repeat with $repeat.range

operation

{
  "$repeat": {
    "range": "0:-2, 10, 20:30:5",
    "value": {
      "$expression": "$repeat.value"
    }
  }
}

result

[0, -1, -2, 10, 20, 25, 30]

Repeat with $repeat.in as array

operation

{
  "$repeat": {
    "in": ["a", "b"],
    "value": {
      "$expression": "$repeat.value"
    }
  }
}

result

["a", "b"]

Repeat with $repeat.in as object

The current key is available on the scope as $repeat.key variable.

operation

{
  "$repeat": {
    "in": {
      "keyA": "valueA",
      "keyB": "valueB"
    },
    "value": {
      "$expression": "{key: $repeat.key, value: $repeat.value}"
    }
  }
}

result

[
  { "key": "keyA", "value": "valueA" },
  { "key": "keyB", "value": "valueB" }
]

Getting the current index

The current index is available on the scope as $repeat.index variable.

operation

{
  "$repeat": {
    "range": "1:2",
    "value": {
      "$expression": "$repeat.index"
    }
  }
}

result

[0, 1]

Nested repeat

Use $parent to get to the parent scope containing the parent $repeat.

operation

{
  "$repeat": {
    "range": "0:1",
    "value": {
      "$repeat": {
        "range": "0:1",
        "value": {
          "$expression": "$parent.$repeat.index + '.' + $repeat.index"
        }
      }
    }
  }
}

result

["0.0", "0.1", "1.0", "1.1"]

$include

Use $include to load other JSON or YAML files and process them in the current scope.

javascript

var result = jsonMerger.mergeFiles(["a.json", "b.json"]);

a.json

{
  "someArray": [1, 2, 3]
}

b.json

{
  "someArray": [
    {
      "$include": "remove.json"
    }
  ]
}

remove.json

{
  "$remove": true
}

result

{
  "someArray": [2, 3]
}

$expression

Use the $expression operation to calculate a value with the help of a JavaScript expression. The expression has access to the standard built-in JavaScript objects, the current scope and optionally an $input variable.

Calculate a value

javascript

var result = jsonMerger.mergeFile("a.json");

a.json

{
  "prop": {
    "$expression": "1 + 2"
  }
}

result

{
  "prop": 3
}

Calculate a value using $expression.input

javascript

var result = jsonMerger.mergeFile("b.json");

a.json

{
  "add": 2
}

b.json

{
  "prop": {
    "$expression": {
      "expression": "1 + $input",
      "input": {
        "$import": "a.json#/add"
      }
    }
  }
}

result

{
  "prop": 3
}

Calculate a value using the scope $targetProperty

javascript

var result = jsonMerger.mergeFiles(["a.json", "b.json"]);

a.json

{
  "prop": 1
}

b.json

{
  "prop": {
    "$expression": "$targetProperty + 2"
  }
}

result

{
  "prop": 3
}

Calculate a value using the scope $params

javascript

var result = jsonMerger.mergeFile("a.json", {
  params: {
    add: 2,
  },
});

a.json

{
  "prop": {
    "$expression": "1 + $params.add"
  }
}

result

{
  "prop": 3
}

Scopes

Scopes can be created while processing operation properties. If for example a $merge.with is being processed then the merger will create a new scope for the $merge.with property. Or if a $repeat.value property is being processed a new scope is created for the $repeat.value property.

A scope always has a $source property but not necessarily a $target property.

When we are merging object A with object B, then the $target property in the scope of object A is undefined because object A is not merged with anything. It does have a $source property referring to object A itself. Object B on the other hand has the processed object A as $target because object B is being merged with object A. The $source property in the scope of object B refers to object B.

If object B had defined a $merge operation, then the merger would create a new scope for the $merge.source property and a new scope for the $merge.with property.

The $target within the $merge.source scope would be undefined because $merge.source is not merged with anything. The $target within the $merge.with scope is the processed $merge.source because $merge.with is being merged with $merge.source. The result of the $merge operation will eventually be merged with object A.

When in the $merge.source scope it is possible to get to the root (object B) scope using the $root property or to a parent scope using the $parent property.

interface Scope {
  $params?: any; // $params properties in current scope
  $parent?: Scope; // reference to parent scope
  $repeat?: ScopeRepeat; // $repeat properties in current scope
  $root: Scope; // reference to root scope
  $source: any; // reference to the source object
  $target?: any; // reference to the target object
}

Example

javascript

var result = jsonMerger.mergeFiles(["a.json", "b.json"]);

a.json

// this is the root scope
{
  "prop1": {
    "$expression": "$target" // refers to undefined because a.json has no target
  },
  "prop2": {
    "$expression": "$targetProperty" // refers to undefined because a.json has no target
  },
  "prop3": {
    "$expression": "$source" // refers to the unprocessed a.json
  },
  "prop4": {
    "$expression": "$sourceProperty" // refers to the unprocessed a.json#/prop4
  },
  "prop5": {
    "$expression": "$root.$target" // refers to undefined because a.json has no target
  },
  "prop6": {
    "$expression": "$root.$source" // refers to the unprocessed a.json
  },
  "prop7": {
    "$expression": "$parent" // refers to undefined because the current scope has no parent scope
  },
}

b.json

// this is the root scope
{
  "prop1": {
    "$expression": "$target" // refers to the processed a.json
  },
  "prop2": {
    "$expression": "$targetProperty" // refers to the processed a.json#/prop2
  },
  "prop3": {
    "$expression": "$source" // refers to the unprocessed b.json
  },
  "prop4": {
    "$expression": "$sourceProperty" // refers to the unprocessed b.json#/prop4
  },
  "prop5": {
    "$merge": {
      "source": { // $merge.source creates a new scope
        "prop1": {
          "$expression": "$target" // refers to undefined because b.json#/prop5/$merge/source has no target
        },
        "prop2": {
          "$expression": "$targetProperty" // refers to undefined because b.json#/prop5/$merge/source has no target
        },
        "prop3": {
          "$expression": "$source" // refers to the unprocessed b.json#/prop5/$merge/source
        },
        "prop4": {
          "$expression": "$sourceProperty" // refers to the unprocessed b.json#/prop5/$merge/source/prop4
        }
        "prop5": {
          "$expression": "$root.$target" // refers to the processed a.json
        }
        "prop6": {
          "$expression": "$root.$source" // refers to the unprocessed b.json
        },
        "prop7": {
          "$expression": "$parent.$target" // refers to the processed a.json
        }
        "prop8": {
          "$expression": "$parent.$source" // refers to the unprocessed b.json
        }
      },
      "with": { // $merge.with creates a new scope
        "prop1": {
          "$expression": "$target" // refers to the processed b.json#/prop5/$merge/source
        },
        "prop2": {
          "$expression": "$targetProperty" // refers to the processed b.json#/prop5/$merge/source/prop2
        },
        "prop3": {
          "$expression": "$source" // refers to the unprocessed b.json#/prop5/$merge/with
        },
        "prop4": {
          "$expression": "$sourceProperty" // refers to the unprocessed b.json#/prop5/$merge/with/prop4
        },
        "prop5": {
          "$expression": "$root.$target" // refers to the processed a.json
        },
        "prop6": {
          "$expression": "$root.source" // refers to the unprocessed b.json
        },
        "prop7": {
          "$expression": "$parent.$target" // refers to the processed a.json
        },
        "prop8": {
          "$expression": "$parent.source" // refers to the unprocessed b.json
        }
      }
    }
  },
  "prop6": {
    "$repeat": {
      "range": "0:1",
      "value": { // $repeat.value creates a new scope with a $repeat property on it
        "$repeat": {
          "range": "0:1",
          "value": { // $repeat.value creates a new scope with a $repeat property on it
            "$expression": "'This is item ' + $parent.$repeat.index + '.' + $repeat.index"
          }
        }
      }
    }
  }
}

Command line interface json-merger

You can use json-merger as a command line tool:

  Usage: json-merger [options] <file ...>


  Options:

    -V, --version                       output the version number
    -p, --pretty                        pretty-print the output json
    -o, --output [file]                 the output file. Defaults to stdout
    -op, --operation-prefix [prefix]    the operation prefix. Defaults to $
    -am, --default-array-merge-operation [operation]      the default array merge operation. Defaults to combine
    --error-on-file-not-found           throw an error if a file is not found. Defaults to true
    --error-on-ref-not-found            throw an error if a JSON pointer or JSON path is not found. Defaults to true
    -h, --help                          output usage information

Usage:

json-merger a.json > result.json
json-merger --output result.json a.json
json-merger --output result.json --pretty a.json

Install json-merger globally to be able to use the command line interface.

npm install -g json-merger

Roadmap

  • Add configurable file resolvers to import files from different sources.
  • Add configurable (de)serializers to import and export different file formats.

About

Merge JSON files and objects with indicators like $import $remove $replace $merge

Resources

Readme

License

MIT license
Activity
Custom properties

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages

  • JavaScript 62.4%
  • TypeScript 37.6%