From bcdae5444707448fdec9ee405c2e1b6ae3e51502 Mon Sep 17 00:00:00 2001 From: carolxob Date: Mon, 6 Mar 2023 13:05:47 -0500 Subject: [PATCH 01/56] Opening new PR for file. Signed-off-by: carolxob --- .../configuration/processors/mutate-event.md | 286 ++++++++++++++++++ 1 file changed, 286 insertions(+) create mode 100644 _data-prepper/pipelines/configuration/processors/mutate-event.md diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md new file mode 100644 index 00000000000..c4651d779dc --- /dev/null +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -0,0 +1,286 @@ +--- +layout: default +title: Mutate event +parent: Processors +grand_parent: Pipelines +nav_order: 45 +--- + +# Mutate event processors + +The following processors allow you to mutate an event. + +## AddEntry + +The `AddEntry` processor adds entries to an event. + +### Basic usage + +To get started, create the following `pipeline.yaml` file: + +```yaml +pipeline: + source: + file: + path: "/full/path/to/logs_json.log" + record_type: "event" + format: "json" + processor: + - add_entries: + entries: + - key: "newMessage" + value: 3 + overwrite_if_key_exists: true + sink: + - stdout: +``` + +Create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with this filepath. + +```json +{"message": "value"} +``` + +When you run this processor, it parses the message into the following output: + +```json +{"message": "value", "newMessage": 3} +``` + +> If `newMessage` already exists, its existing value is overwritten with a value of `3`. + +### Configuration + +You can configure the `AddEntries` processor with the following options: + +* `entries` (required): A list of entries to add to an event +* `key` (required): The key of the new entry to be added +* `value` (required): The value of the new entry to be added. Strings, booleans, numbers, null, nested objects, and arrays containing the aforementioned data types are valid to use +* `overwrite_if_key_exists` (optional): When set to `true`, if `key` already exists in the event, then the existing value is overwritten. The default value is `false`. + +## copy value + +The `copy value` processor copies values within an event. + +### Basic usage + +To get started, create the following `pipeline.yaml` file: + +```yaml +pipeline: + source: + file: + path: "/full/path/to/logs_json.log" + record_type: "event" + format: "json" + processor: + - copy_values: + entries: + - from_key: "message" + to_key: "newMessage" + overwrite_if_to_key_exists: true + sink: + - stdout: +``` + +Create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with the path of this file: + +```json +{"message": "value"} +``` + +When you run this processor, it parses the message into the following output: + +```json +{"message": "value", "newMessage": "value"} +``` + +> If `newMessage` had already existed, its existing value would have been overwritten with `value` + +### Configuration + +You can configure the `copy value` processor with the following options: + +* `entries` - (required) - A list of entries to be copied in an event + * `from_key` - (required) - The key of the entry to be copied + * `to_key` - (required) - The key of the new entry to be added + * `overwrite_if_to_key_exists` - (optional) - When set to `true`, if `to_key` already exists in the event, then the existing value will be overwritten. The default is `false`. + + +## DeleteEntry + +The `DeleteEntry` processor deletes entries in an event. + +### Basic usage + +To get started, create the following `pipeline.yaml` file: + +```yaml +pipeline: + source: + file: + path: "/full/path/to/logs_json.log" + record_type: "event" + format: "json" + processor: + - delete_entries: + with_keys: ["message"] + sink: + - stdout: +``` + +Create the following file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with the path of this file. + +```json +{"message": "value", "message2": "value2"} +``` + +When you run the `DeleteEntry` processor, it parses the message into the following output: + +```json +{"message2": "value2"} +``` + +> If `message` had not existed in the event, then nothing would have happened + +### Configuration + +You can configure the `DeleteEntry` processor with the following options: + +`with_keys` - (required) - An array of keys of the entries to be deleted. + + +## rename key + +The `rename key` processor renames keys in an event. + +### Basic usage + +To get started, create the following `pipeline.yaml` file: + +```yaml +pipeline: + source: + file: + path: "/full/path/to/logs_json.log" + record_type: "event" + format: "json" + processor: + - rename_keys: + entries: + - from_key: "message" + to_key: "newMessage" + overwrite_if_to_key_exists: true + sink: + - stdout: +``` + +Create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with the path of this file. + +```json +{"message": "value"} +``` + +When run, the processor parses the message into the following output: + +```json +{"newMessage": "value"} +``` + +> If `newMessage` already exists, its existing value is overwritten with `value`. + +### Configuration + +You can configure the `rename key` processor with the following options: + +* `entries` - (required) - A list of entries to rename in an event + * `from_key` - (required) - The key of the entry to be renamed + * `to_key` - (required) - The new key of the entry + * `overwrite_if_to_key_exists` - (optional) - When set to `true`, if `to_key` already exists in the event, then the existing value will be overwritten. The default is `false`. + +### Special considerations + +The renaming operation occurs in a defined order. This means that chaining is implicit with the `RenameKey` processor. See the following `piplines.yaml` file example: + + + +```yaml +pipeline: + source: + file: + path: "/full/path/to/logs_json.log" + record_type: "event" + format: "json" + processor: + - rename_key: + entries: + - from_key: "message" + to_key: "message2" + - from_key: "message2" + to_key: "message3" + sink: + - stdout: +``` + +Add the following contents to the `logs_json.log` file: + +```json +{"message": "value"} +``` + +After the processor runs, the following output appears: + +```json +{"message3": "value"} +``` + +## ConvertEntry + +The `ConvertEntry` processor converts value type associated with the specified key in a message to the specified type. It is a casting processor that changes the types of some fields in the event or message. Some of inputted data may need to be converted to different types, such as an integer or a double, so that it will pass the events through condition-based processors, or to perform conditional routing. + +## Basic usage + +To get started with type conversion processor using Data Prepper, create the following `pipeline.yaml` file: + +```yaml +type-conv-pipeline: + source: + file: + path: "/full/path/to/logs_json.log" + record_type: "event" + format: "json" + processor: + - grok: + match: + message: ['%{IPORHOST:clientip} \[%{HTTPDATE:timestamp}\] %{NUMBER:response_status}'] + - convert_entry_type: + key: "response_status" + type: "integer" + sink: + - stdout: +``` + +Create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with the path of this file. + +```json +{"message": "10.10.10.19 [19/Feb/2015:15:50:36 -0500] 200"} +``` + +When you run the `Grok` processor, it processor parses the message into the following output: + +```json +{"message": "10.10.10.10 [19/Feb/2015:15:50:36 -0500] 200", "clientip":"10.10.10.10", "timestamp": "19/Feb/2015:15:50:36 -0500", "response_status": "200"} +``` + +The type conversion processor changes the output received into the following output, where the type of `response_status` value changes to an integer: + +```json +{"message": "10.10.10.10 [19/Feb/2015:15:50:36 -0500] 200", "clientip":"10.10.10.10", "timestamp": "19/Feb/2015:15:50:36 -0500", "response_status": 200} +``` + +### Configuration + +You can configure the `ConvertEntry` processor with the following options: + +* `key` (required): Keys whose value needs to be converted to a different type +* `type`: Target type for key value. Possible values are `integer`, `double`, `string`, and `boolean`. Default value is `integer`. \ No newline at end of file From 25d1500d60e9115eb88b3b1e5ea4f4af2864a617 Mon Sep 17 00:00:00 2001 From: carolxob Date: Tue, 7 Mar 2023 14:49:03 -0500 Subject: [PATCH 02/56] Minor adjustements. Signed-off-by: carolxob --- .../configuration/processors/mutate-event.md | 94 +++++++++++-------- 1 file changed, 53 insertions(+), 41 deletions(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index c4651d779dc..ef2227df13f 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -8,12 +8,25 @@ nav_order: 45 # Mutate event processors +You can use mutate event processors to add entries to an event, delete entries from an event, rename keys in an event, copy values within an event, and convert value types in an event. + The following processors allow you to mutate an event. ## AddEntry The `AddEntry` processor adds entries to an event. +### Configuration + +You can configure the `AddEntries` processor with the following options: + + + +* `entries` (required): A list of entries to add to an event +* `key` (required): The key of the new entry to be added +* `value` (required): The value of the new entry to be added. Strings, booleans, numbers, null, nested objects, and arrays containing the aforementioned data types are valid to use +* `overwrite_if_key_exists` (optional): When set to `true`, if `key` already exists in the event, then the existing value is overwritten. The default value is `false`. + ### Basic usage To get started, create the following `pipeline.yaml` file: @@ -49,18 +62,20 @@ When you run this processor, it parses the message into the following output: > If `newMessage` already exists, its existing value is overwritten with a value of `3`. -### Configuration +## copy value -You can configure the `AddEntries` processor with the following options: +The `copy value` processor copies values within an event. -* `entries` (required): A list of entries to add to an event -* `key` (required): The key of the new entry to be added -* `value` (required): The value of the new entry to be added. Strings, booleans, numbers, null, nested objects, and arrays containing the aforementioned data types are valid to use -* `overwrite_if_key_exists` (optional): When set to `true`, if `key` already exists in the event, then the existing value is overwritten. The default value is `false`. +### Configuration -## copy value +You can configure the `copy value` processor with the following options: -The `copy value` processor copies values within an event. + + +* `entries` - (required) - A list of entries to be copied in an event + * `from_key` - (required) - The key of the entry to be copied + * `to_key` - (required) - The key of the new entry to be added + * `overwrite_if_to_key_exists` - (optional) - When set to `true`, if `to_key` already exists in the event, then the existing value will be overwritten. The default is `false`. ### Basic usage @@ -97,19 +112,18 @@ When you run this processor, it parses the message into the following output: > If `newMessage` had already existed, its existing value would have been overwritten with `value` -### Configuration -You can configure the `copy value` processor with the following options: +## DeleteEntry -* `entries` - (required) - A list of entries to be copied in an event - * `from_key` - (required) - The key of the entry to be copied - * `to_key` - (required) - The key of the new entry to be added - * `overwrite_if_to_key_exists` - (optional) - When set to `true`, if `to_key` already exists in the event, then the existing value will be overwritten. The default is `false`. +The `DeleteEntry` processor deletes entries in an event. +### Configuration -## DeleteEntry +You can configure the `DeleteEntry` processor with the following options: -The `DeleteEntry` processor deletes entries in an event. + + +`with_keys` - (required) - An array of keys of the entries to be deleted. ### Basic usage @@ -141,18 +155,23 @@ When you run the `DeleteEntry` processor, it parses the message into the followi {"message2": "value2"} ``` -> If `message` had not existed in the event, then nothing would have happened +> If `message` does not exist in the event, then no action occurs. -### Configuration -You can configure the `DeleteEntry` processor with the following options: +## rename key -`with_keys` - (required) - An array of keys of the entries to be deleted. +The `rename key` processor renames keys in an event. +### Configuration -## rename key +You can configure the `rename key` processor with the following options: -The `rename key` processor renames keys in an event. + + +* `entries` - (required) - A list of entries to rename in an event + * `from_key` - (required) - The key of the entry to be renamed + * `to_key` - (required) - The new key of the entry + * `overwrite_if_to_key_exists` - (optional) - When set to `true`, if `to_key` already exists in the event, then the existing value will be overwritten. The default is `false`. ### Basic usage @@ -189,15 +208,6 @@ When run, the processor parses the message into the following output: > If `newMessage` already exists, its existing value is overwritten with `value`. -### Configuration - -You can configure the `rename key` processor with the following options: - -* `entries` - (required) - A list of entries to rename in an event - * `from_key` - (required) - The key of the entry to be renamed - * `to_key` - (required) - The new key of the entry - * `overwrite_if_to_key_exists` - (optional) - When set to `true`, if `to_key` already exists in the event, then the existing value will be overwritten. The default is `false`. - ### Special considerations The renaming operation occurs in a defined order. This means that chaining is implicit with the `RenameKey` processor. See the following `piplines.yaml` file example: @@ -236,9 +246,18 @@ After the processor runs, the following output appears: ## ConvertEntry -The `ConvertEntry` processor converts value type associated with the specified key in a message to the specified type. It is a casting processor that changes the types of some fields in the event or message. Some of inputted data may need to be converted to different types, such as an integer or a double, so that it will pass the events through condition-based processors, or to perform conditional routing. +The `ConvertEntry` processor converts a value type associated with the specified key in a message to the specified type. It is a casting processor that changes the types of some fields in the event or message. Some of inputted data may need to be converted to different types, such as an integer or a double, so that it will pass the events through condition-based processors, or to perform conditional routing. + +### Configuration + +You can configure the `ConvertEntry` processor with the following options: + + -## Basic usage +* `key` (required): Keys whose value needs to be converted to a different type +* `type`: Target type for key value. Possible values are `integer`, `double`, `string`, and `boolean`. Default value is `integer`. + +### Basic usage To get started with type conversion processor using Data Prepper, create the following `pipeline.yaml` file: @@ -276,11 +295,4 @@ The type conversion processor changes the output received into the following out ```json {"message": "10.10.10.10 [19/Feb/2015:15:50:36 -0500] 200", "clientip":"10.10.10.10", "timestamp": "19/Feb/2015:15:50:36 -0500", "response_status": 200} -``` - -### Configuration - -You can configure the `ConvertEntry` processor with the following options: - -* `key` (required): Keys whose value needs to be converted to a different type -* `type`: Target type for key value. Possible values are `integer`, `double`, `string`, and `boolean`. Default value is `integer`. \ No newline at end of file +``` \ No newline at end of file From 8f87f99979ec217bffa4517d18b5cbf69268849f Mon Sep 17 00:00:00 2001 From: carolxob Date: Tue, 7 Mar 2023 15:39:28 -0500 Subject: [PATCH 03/56] Adjustements to tables. Signed-off-by: carolxob --- .../configuration/processors/mutate-event.md | 64 +++++++++++-------- 1 file changed, 39 insertions(+), 25 deletions(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index ef2227df13f..67c24b7445b 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -10,7 +10,17 @@ nav_order: 45 You can use mutate event processors to add entries to an event, delete entries from an event, rename keys in an event, copy values within an event, and convert value types in an event. -The following processors allow you to mutate an event. +The following processors allow you to mutate an event: + + ## AddEntry @@ -22,10 +32,12 @@ You can configure the `AddEntries` processor with the following options: -* `entries` (required): A list of entries to add to an event -* `key` (required): The key of the new entry to be added -* `value` (required): The value of the new entry to be added. Strings, booleans, numbers, null, nested objects, and arrays containing the aforementioned data types are valid to use -* `overwrite_if_key_exists` (optional): When set to `true`, if `key` already exists in the event, then the existing value is overwritten. The default value is `false`. +Option | Required | Description +:--- | :--- | :--- +|`entries` | Yes | A list of entries to add to an event. | +|`key` | Yes | The key of the new entry to be added. | +|`value` | Yes | The value of the new entry to be added. You can use the following data types: strings, booleans, numbers, null, nested objects, and arrays. | +|`overwrite_if_key_exists` | No | When set to `true`, if `key` already exists in the event, then the existing value is overwritten. The default value is `false`. | ### Basic usage @@ -62,7 +74,7 @@ When you run this processor, it parses the message into the following output: > If `newMessage` already exists, its existing value is overwritten with a value of `3`. -## copy value +## CopyValue The `copy value` processor copies values within an event. @@ -70,12 +82,12 @@ The `copy value` processor copies values within an event. You can configure the `copy value` processor with the following options: - - -* `entries` - (required) - A list of entries to be copied in an event - * `from_key` - (required) - The key of the entry to be copied - * `to_key` - (required) - The key of the new entry to be added - * `overwrite_if_to_key_exists` - (optional) - When set to `true`, if `to_key` already exists in the event, then the existing value will be overwritten. The default is `false`. +Option | Required | Description +:--- | :--- | :--- +| `entries` | Yes | A list of entries to be copied in an event. | +| `from_key` | Yes | The key of the entry to be copied. +| `to_key` | Yes | The key of the new entry to be added. +| `overwrite_if_to_key_exists` | No | When set to a value of `true`, if `to_key` already exists in the event, then the existing value will be overwritten. The default value is `false`. | ### Basic usage @@ -121,9 +133,9 @@ The `DeleteEntry` processor deletes entries in an event. You can configure the `DeleteEntry` processor with the following options: - - -`with_keys` - (required) - An array of keys of the entries to be deleted. +Option | Required | Description +:--- | :--- | :--- +| `with_keys` | Yes | An array of keys of the entries to be deleted. | ### Basic usage @@ -158,7 +170,7 @@ When you run the `DeleteEntry` processor, it parses the message into the followi > If `message` does not exist in the event, then no action occurs. -## rename key +## RenameKey The `rename key` processor renames keys in an event. @@ -166,12 +178,12 @@ The `rename key` processor renames keys in an event. You can configure the `rename key` processor with the following options: - - -* `entries` - (required) - A list of entries to rename in an event - * `from_key` - (required) - The key of the entry to be renamed - * `to_key` - (required) - The new key of the entry - * `overwrite_if_to_key_exists` - (optional) - When set to `true`, if `to_key` already exists in the event, then the existing value will be overwritten. The default is `false`. +Option | Required | Description +:--- | :--- | :--- +| `entries` | Yes | A list of entries to rename in an event. | +| `from_key` | Yes | The key of the entry to be renamed. | +| `to_key` | Yes | The new key of the entry. +| `overwrite_if_to_key_exists` | No | When set to a value of`true`, if `to_key` already exists in the event, then the existing value will be overwritten. The default value is `false`. ### Basic usage @@ -252,10 +264,12 @@ The `ConvertEntry` processor converts a value type associated with the specified You can configure the `ConvertEntry` processor with the following options: - + -* `key` (required): Keys whose value needs to be converted to a different type -* `type`: Target type for key value. Possible values are `integer`, `double`, `string`, and `boolean`. Default value is `integer`. +Option | Required | Description +:--- | :--- | :--- +| `key`| Yes | Keys whose value needs to be converted to a different type. | +| `type` | No | Target type for key value. Possible values are `integer`, `double`, `string`, and `boolean`. Default value is `integer`.| ### Basic usage From ae81c0bfce1e0143b7d2580515516b3d6c29b58f Mon Sep 17 00:00:00 2001 From: carolxob Date: Tue, 7 Mar 2023 15:46:14 -0500 Subject: [PATCH 04/56] Removed some markdown comments. Signed-off-by: carolxob --- .../pipelines/configuration/processors/mutate-event.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index 67c24b7445b..321b701632e 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -224,8 +224,6 @@ When run, the processor parses the message into the following output: The renaming operation occurs in a defined order. This means that chaining is implicit with the `RenameKey` processor. See the following `piplines.yaml` file example: - - ```yaml pipeline: source: @@ -264,8 +262,6 @@ The `ConvertEntry` processor converts a value type associated with the specified You can configure the `ConvertEntry` processor with the following options: - - Option | Required | Description :--- | :--- | :--- | `key`| Yes | Keys whose value needs to be converted to a different type. | From 58cf072854f10e3ea4708a2a9733c1be47297da2 Mon Sep 17 00:00:00 2001 From: carolxob Date: Tue, 7 Mar 2023 16:37:42 -0500 Subject: [PATCH 05/56] Minor updates. Signed-off-by: carolxob --- .../configuration/processors/mutate-event.md | 20 +++++++++---------- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index 321b701632e..266ceabb1b3 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -22,15 +22,13 @@ The following processors allow you to mutate an event: ---> -## AddEntry +## AddEntries -The `AddEntry` processor adds entries to an event. +The `AddEntries` processor adds entries to an event. ### Configuration -You can configure the `AddEntries` processor with the following options: - - +You can configure the `AddEntries` processor with the following options. Option | Required | Description :--- | :--- | :--- @@ -74,13 +72,13 @@ When you run this processor, it parses the message into the following output: > If `newMessage` already exists, its existing value is overwritten with a value of `3`. -## CopyValue +## copy value The `copy value` processor copies values within an event. ### Configuration -You can configure the `copy value` processor with the following options: +You can configure the `copy value` processor with the following options. Option | Required | Description :--- | :--- | :--- @@ -122,7 +120,7 @@ When you run this processor, it parses the message into the following output: {"message": "value", "newMessage": "value"} ``` -> If `newMessage` had already existed, its existing value would have been overwritten with `value` +> If `newMessage` had already exists, its current value is overwritten with `value`. ## DeleteEntry @@ -131,7 +129,7 @@ The `DeleteEntry` processor deletes entries in an event. ### Configuration -You can configure the `DeleteEntry` processor with the following options: +You can configure the `DeleteEntry` processor with the following options. Option | Required | Description :--- | :--- | :--- @@ -176,7 +174,7 @@ The `rename key` processor renames keys in an event. ### Configuration -You can configure the `rename key` processor with the following options: +You can configure the `rename key` processor with the following options. Option | Required | Description :--- | :--- | :--- @@ -260,7 +258,7 @@ The `ConvertEntry` processor converts a value type associated with the specified ### Configuration -You can configure the `ConvertEntry` processor with the following options: +You can configure the `ConvertEntry` processor with the following options. Option | Required | Description :--- | :--- | :--- From f18444d329fb50e5c87201e107c271f39365aadb Mon Sep 17 00:00:00 2001 From: carolxob Date: Wed, 8 Mar 2023 09:54:36 -0500 Subject: [PATCH 06/56] Minor edit. Signed-off-by: carolxob --- .../pipelines/configuration/processors/mutate-event.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index 266ceabb1b3..34b3991de4b 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -210,7 +210,7 @@ Create a log file named `logs_json.log` and replace the `path` in the file sourc {"message": "value"} ``` -When run, the processor parses the message into the following output: +When ran, the processor parses the message into the following output: ```json {"newMessage": "value"} From 755c7cf19af249314879fa410450b7822f5bbeeb Mon Sep 17 00:00:00 2001 From: carolxob Date: Wed, 8 Mar 2023 09:56:36 -0500 Subject: [PATCH 07/56] Adjustements to tables. Signed-off-by: carolxob --- .../pipelines/configuration/processors/mutate-event.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index 34b3991de4b..d5823bcff71 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -181,7 +181,7 @@ Option | Required | Description | `entries` | Yes | A list of entries to rename in an event. | | `from_key` | Yes | The key of the entry to be renamed. | | `to_key` | Yes | The new key of the entry. -| `overwrite_if_to_key_exists` | No | When set to a value of`true`, if `to_key` already exists in the event, then the existing value will be overwritten. The default value is `false`. +| `overwrite_if_to_key_exists` | No | When set to a value of`true`, if `to_key` already exists in the event, then the existing value will be overwritten. The default value is `false`. | ### Basic usage @@ -263,7 +263,7 @@ You can configure the `ConvertEntry` processor with the following options. Option | Required | Description :--- | :--- | :--- | `key`| Yes | Keys whose value needs to be converted to a different type. | -| `type` | No | Target type for key value. Possible values are `integer`, `double`, `string`, and `boolean`. Default value is `integer`.| +| `type` | No | Target type for key value. Possible values are `integer`, `double`, `string`, and `boolean`. Default value is `integer`. | ### Basic usage From 3ae7fe58c02cbff7ab7f7071f9d834864586c9ec Mon Sep 17 00:00:00 2001 From: carolxob Date: Wed, 8 Mar 2023 11:17:18 -0500 Subject: [PATCH 08/56] Updates based on doc review feedback. Signed-off-by: carolxob --- .../pipelines/configuration/processors/mutate-event.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index d5823bcff71..bd65f5a9038 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -293,7 +293,7 @@ Create a log file named `logs_json.log` and replace the `path` in the file sourc {"message": "10.10.10.19 [19/Feb/2015:15:50:36 -0500] 200"} ``` -When you run the `Grok` processor, it processor parses the message into the following output: +When you run the `Grok` processor, the processor parses the message into the following output: ```json {"message": "10.10.10.10 [19/Feb/2015:15:50:36 -0500] 200", "clientip":"10.10.10.10", "timestamp": "19/Feb/2015:15:50:36 -0500", "response_status": "200"} From a5ae6bf0f13db958fd7aa86b5ae1bc9039cbd7a8 Mon Sep 17 00:00:00 2001 From: carolxob Date: Thu, 9 Mar 2023 11:00:21 -0500 Subject: [PATCH 09/56] MAde changes based on tech review feedback. Signed-off-by: carolxob --- .../configuration/processors/mutate-event.md | 16 +++------------- 1 file changed, 3 insertions(+), 13 deletions(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index bd65f5a9038..e9fc8714a24 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -8,19 +8,9 @@ nav_order: 45 # Mutate event processors -You can use mutate event processors to add entries to an event, delete entries from an event, rename keys in an event, copy values within an event, and convert value types in an event. +Mutate event processors allow you to modify events in Data Prepper. You can use these processors to add entries to an event, delete entries from an event, rename keys in an event, copy values within an event, and convert value types in an event. -The following processors allow you to mutate an event: - - +The following processors allow you to mutate an event. ## AddEntries @@ -220,7 +210,7 @@ When ran, the processor parses the message into the following output: ### Special considerations -The renaming operation occurs in a defined order. This means that chaining is implicit with the `RenameKey` processor. See the following `piplines.yaml` file example: +Renaming operations occur in the order that the key pair entries are listed in within the `pipelines.yaml` file. This means that chaining (where key pairs are renamed in sequence) is implicit with the `RenameKey` processor. See the following `piplines.yaml` file example: ```yaml pipeline: From 0be3d4aad9af070a847957bb868cc58b6274ad50 Mon Sep 17 00:00:00 2001 From: carolxob Date: Mon, 13 Mar 2023 16:31:23 -0400 Subject: [PATCH 10/56] Updated text for YAML file path locations. Signed-off-by: carolxob --- .../configuration/processors/mutate-event.md | 20 +++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index e9fc8714a24..18583665856 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -27,7 +27,7 @@ Option | Required | Description |`value` | Yes | The value of the new entry to be added. You can use the following data types: strings, booleans, numbers, null, nested objects, and arrays. | |`overwrite_if_key_exists` | No | When set to `true`, if `key` already exists in the event, then the existing value is overwritten. The default value is `false`. | -### Basic usage +### Usage To get started, create the following `pipeline.yaml` file: @@ -48,7 +48,7 @@ pipeline: - stdout: ``` -Create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with this filepath. +Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). ```json {"message": "value"} @@ -77,7 +77,7 @@ Option | Required | Description | `to_key` | Yes | The key of the new entry to be added. | `overwrite_if_to_key_exists` | No | When set to a value of `true`, if `to_key` already exists in the event, then the existing value will be overwritten. The default value is `false`. | -### Basic usage +### Usage To get started, create the following `pipeline.yaml` file: @@ -98,7 +98,7 @@ pipeline: - stdout: ``` -Create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with the path of this file: +Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). ```json {"message": "value"} @@ -125,7 +125,7 @@ Option | Required | Description :--- | :--- | :--- | `with_keys` | Yes | An array of keys of the entries to be deleted. | -### Basic usage +### Usage To get started, create the following `pipeline.yaml` file: @@ -143,7 +143,7 @@ pipeline: - stdout: ``` -Create the following file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with the path of this file. +Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). ```json {"message": "value", "message2": "value2"} @@ -173,7 +173,7 @@ Option | Required | Description | `to_key` | Yes | The new key of the entry. | `overwrite_if_to_key_exists` | No | When set to a value of`true`, if `to_key` already exists in the event, then the existing value will be overwritten. The default value is `false`. | -### Basic usage +### Usage To get started, create the following `pipeline.yaml` file: @@ -194,7 +194,7 @@ pipeline: - stdout: ``` -Create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with the path of this file. +Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). ```json {"message": "value"} @@ -255,7 +255,7 @@ Option | Required | Description | `key`| Yes | Keys whose value needs to be converted to a different type. | | `type` | No | Target type for key value. Possible values are `integer`, `double`, `string`, and `boolean`. Default value is `integer`. | -### Basic usage +### Usage To get started with type conversion processor using Data Prepper, create the following `pipeline.yaml` file: @@ -277,7 +277,7 @@ type-conv-pipeline: - stdout: ``` -Create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with the path of this file. +Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). ```json {"message": "10.10.10.19 [19/Feb/2015:15:50:36 -0500] 200"} From 7b164fd4cf02edefddb3705df0cb3eb9a50c872f Mon Sep 17 00:00:00 2001 From: carolxob Date: Mon, 13 Mar 2023 16:53:57 -0400 Subject: [PATCH 11/56] Edits made based on technical review feedback. Signed-off-by: carolxob --- .../configuration/processors/mutate-event.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index 18583665856..6a31eb42ba5 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -64,13 +64,15 @@ When you run this processor, it parses the message into the following output: ## copy value -The `copy value` processor copies values within an event. +The `copy value` processor copies the values of an existing key within an event to another key. For example, in the following Usage section, the `copy value` processor copies the value of the key "message", which is "value" to another key called "newMessage". The value of the key "newMessage" is also "value". + + ### Configuration You can configure the `copy value` processor with the following options. -Option | Required | Description +Option | Required | Description :--- | :--- | :--- | `entries` | Yes | A list of entries to be copied in an event. | | `from_key` | Yes | The key of the entry to be copied. @@ -115,7 +117,7 @@ When you run this processor, it parses the message into the following output: ## DeleteEntry -The `DeleteEntry` processor deletes entries in an event. +The `DeleteEntry` processor deletes entries in an event, such as key-value pairs. You can define the keys you want to delete in the `with-keys` field following `delete_entries` in the YAML configuration file. Those keys along with their values are deleted. See the example below in the Usage section, where `with_keys: ["message"] exists in the YAML file. This means that the entry {"message": "value"} is deleted. ### Configuration @@ -200,7 +202,7 @@ Next, create a log file named `logs_json.log` and replace the `path` in the file {"message": "value"} ``` -When ran, the processor parses the message into the following output: +When you run the `RenameKey` processor, it parses the message into the following output: ```json {"newMessage": "value"} @@ -244,7 +246,7 @@ After the processor runs, the following output appears: ## ConvertEntry -The `ConvertEntry` processor converts a value type associated with the specified key in a message to the specified type. It is a casting processor that changes the types of some fields in the event or message. Some of inputted data may need to be converted to different types, such as an integer or a double, so that it will pass the events through condition-based processors, or to perform conditional routing. +The `ConvertEntry` processor converts a value type associated with the specified key in a message to the specified type. It is a casting processor that changes the types of some fields in the event or message. Some of inputted data may need to be converted to different types, such as an integer or a double, or a string to an integer, so that it will pass the events through condition-based processors, or to perform conditional routing. ### Configuration From 156ab945da511dacdf7b635ae5c7a6a6f0bc21bd Mon Sep 17 00:00:00 2001 From: carolxob Date: Mon, 13 Mar 2023 16:57:48 -0400 Subject: [PATCH 12/56] Minor edits from tech review feedback. Signed-off-by: carolxob --- .../pipelines/configuration/processors/mutate-event.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index 6a31eb42ba5..5bb761d4bce 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -202,7 +202,7 @@ Next, create a log file named `logs_json.log` and replace the `path` in the file {"message": "value"} ``` -When you run the `RenameKey` processor, it parses the message into the following output: +When you run the `RenameKey` processor, it parses the message into the following output as the "newMessage": ```json {"newMessage": "value"} @@ -210,6 +210,8 @@ When you run the `RenameKey` processor, it parses the message into the following > If `newMessage` already exists, its existing value is overwritten with `value`. + + ### Special considerations Renaming operations occur in the order that the key pair entries are listed in within the `pipelines.yaml` file. This means that chaining (where key pairs are renamed in sequence) is implicit with the `RenameKey` processor. See the following `piplines.yaml` file example: From 3129091a3589a0cec78f13cc02ba11038d6a4111 Mon Sep 17 00:00:00 2001 From: carolxob Date: Mon, 13 Mar 2023 17:00:09 -0400 Subject: [PATCH 13/56] Minor edits. Signed-off-by: carolxob --- .../pipelines/configuration/processors/mutate-event.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index 5bb761d4bce..f360d10af76 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -54,7 +54,7 @@ Next, create a log file named `logs_json.log` and replace the `path` in the file {"message": "value"} ``` -When you run this processor, it parses the message into the following output: +When you run the `AddEntries` processor, it parses the message into the following output: ```json {"message": "value", "newMessage": 3} From 6c1b75f6c1df83bdd9ea901a3525225b584f7867 Mon Sep 17 00:00:00 2001 From: carolxob Date: Tue, 14 Mar 2023 09:52:18 -0400 Subject: [PATCH 14/56] Updates based on tech review feedback and formatting. Signed-off-by: carolxob --- .../pipelines/configuration/processors/mutate-event.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index f360d10af76..c2b0c4d2659 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -12,9 +12,9 @@ Mutate event processors allow you to modify events in Data Prepper. You can use The following processors allow you to mutate an event. -## AddEntries +## add_entries -The `AddEntries` processor adds entries to an event. +The `add_entries` processor adds entries to an event. ### Configuration @@ -62,11 +62,13 @@ When you run the `AddEntries` processor, it parses the message into the followin > If `newMessage` already exists, its existing value is overwritten with a value of `3`. +In the preceding example, the `add_entries` processor adds a new entry `{"newMessage": 3}` to the existing event, `{"message": "value"}` so that the new event contains two entries in the final output, which iss `{"message": "value","newMessage": 3}`. + ## copy value The `copy value` processor copies the values of an existing key within an event to another key. For example, in the following Usage section, the `copy value` processor copies the value of the key "message", which is "value" to another key called "newMessage". The value of the key "newMessage" is also "value". - + ### Configuration From 9a88f1cfa468878aa189790c678cd9b43b7b88c7 Mon Sep 17 00:00:00 2001 From: carolxob Date: Thu, 16 Mar 2023 15:31:55 -0500 Subject: [PATCH 15/56] Minor upate from doc review feedback. Signed-off-by: carolxob --- .../pipelines/configuration/processors/mutate-event.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index c2b0c4d2659..7c3a02c6e8d 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -25,7 +25,7 @@ Option | Required | Description |`entries` | Yes | A list of entries to add to an event. | |`key` | Yes | The key of the new entry to be added. | |`value` | Yes | The value of the new entry to be added. You can use the following data types: strings, booleans, numbers, null, nested objects, and arrays. | -|`overwrite_if_key_exists` | No | When set to `true`, if `key` already exists in the event, then the existing value is overwritten. The default value is `false`. | +|`overwrite_if_key_exists` | No | When set to `true`, if `key` already exists in the event, the existing value is overwritten. The default value is `false`. | ### Usage From b4d07bb9fbff1a8bd61b8cd082a46f6c15faf8dc Mon Sep 17 00:00:00 2001 From: carolxob Date: Thu, 16 Mar 2023 15:42:32 -0500 Subject: [PATCH 16/56] Edits from doc review feedback. Signed-off-by: carolxob --- .../pipelines/configuration/processors/mutate-event.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index 7c3a02c6e8d..a0fdf67bb2e 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -50,6 +50,8 @@ pipeline: Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). +Before you run the `AddEntries` processor, you see the following: + ```json {"message": "value"} ``` @@ -62,7 +64,7 @@ When you run the `AddEntries` processor, it parses the message into the followin > If `newMessage` already exists, its existing value is overwritten with a value of `3`. -In the preceding example, the `add_entries` processor adds a new entry `{"newMessage": 3}` to the existing event, `{"message": "value"}` so that the new event contains two entries in the final output, which iss `{"message": "value","newMessage": 3}`. +In the preceding example, the `add_entries` processor adds a new entry `{"newMessage": 3}` to the existing event `{"message": "value"}` so that the new event contains two entries in the final output: `{"message": "value","newMessage": 3}`. ## copy value From dc7f22c12ba4bfe3d61ee8de66256960596262e5 Mon Sep 17 00:00:00 2001 From: carolxob Date: Fri, 17 Mar 2023 13:30:05 -0500 Subject: [PATCH 17/56] Updates based on doc review feedback. Signed-off-by: carolxob --- .../configuration/processors/mutate-event.md | 38 ++++++++----------- 1 file changed, 15 insertions(+), 23 deletions(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index a0fdf67bb2e..a7e6fed463c 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -52,10 +52,12 @@ Next, create a log file named `logs_json.log` and replace the `path` in the file Before you run the `AddEntries` processor, you see the following: -```json +```json4 {"message": "value"} ``` + + When you run the `AddEntries` processor, it parses the message into the following output: ```json @@ -68,9 +70,7 @@ In the preceding example, the `add_entries` processor adds a new entry `{"newMes ## copy value -The `copy value` processor copies the values of an existing key within an event to another key. For example, in the following Usage section, the `copy value` processor copies the value of the key "message", which is "value" to another key called "newMessage". The value of the key "newMessage" is also "value". - - +The `copy value` processor copies the values of an existing key within an event to another key. ### Configuration @@ -78,10 +78,10 @@ You can configure the `copy value` processor with the following options. Option | Required | Description :--- | :--- | :--- -| `entries` | Yes | A list of entries to be copied in an event. | +| `entries` | Yes | A list of entries to be copied in an event. | `from_key` | Yes | The key of the entry to be copied. | `to_key` | Yes | The key of the new entry to be added. -| `overwrite_if_to_key_exists` | No | When set to a value of `true`, if `to_key` already exists in the event, then the existing value will be overwritten. The default value is `false`. | +|`overwrite_if_key_exists` | No | When set to `true`, if `key` already exists in the event, the existing value is overwritten. The default value is `false`. | ### Usage @@ -121,7 +121,7 @@ When you run this processor, it parses the message into the following output: ## DeleteEntry -The `DeleteEntry` processor deletes entries in an event, such as key-value pairs. You can define the keys you want to delete in the `with-keys` field following `delete_entries` in the YAML configuration file. Those keys along with their values are deleted. See the example below in the Usage section, where `with_keys: ["message"] exists in the YAML file. This means that the entry {"message": "value"} is deleted. +The `DeleteEntry` processor deletes entries in an event, such as key-value pairs. You can define the keys you want to delete in the `with-keys` field following `delete_entries` in the YAML configuration file. Those keys along with their values are deleted. ### Configuration @@ -164,13 +164,13 @@ When you run the `DeleteEntry` processor, it parses the message into the followi > If `message` does not exist in the event, then no action occurs. -## RenameKey +## Rename Key -The `rename key` processor renames keys in an event. +The `Rename Key` processor renames keys in an event. ### Configuration -You can configure the `rename key` processor with the following options. +You can configure the `Rename Key` processor with the following options. Option | Required | Description :--- | :--- | :--- @@ -252,7 +252,7 @@ After the processor runs, the following output appears: ## ConvertEntry -The `ConvertEntry` processor converts a value type associated with the specified key in a message to the specified type. It is a casting processor that changes the types of some fields in the event or message. Some of inputted data may need to be converted to different types, such as an integer or a double, or a string to an integer, so that it will pass the events through condition-based processors, or to perform conditional routing. +The `ConvertEntry` processor converts a value type associated with the specified key in a message to the specified type. It is a casting processor that changes the types of some fields in the event or message. Some of inputted data may need to be converted to different types, such as an integer or a double, or a string to an integer, so that it will pass the events through condition-based processors, or to perform conditional routing. ### Configuration @@ -261,7 +261,7 @@ You can configure the `ConvertEntry` processor with the following options. Option | Required | Description :--- | :--- | :--- | `key`| Yes | Keys whose value needs to be converted to a different type. | -| `type` | No | Target type for key value. Possible values are `integer`, `double`, `string`, and `boolean`. Default value is `integer`. | +| `type` | No | Target type for key value pair. Possible values are `integer`, `double`, `string`, and `boolean`. Default value is `integer`. | ### Usage @@ -275,9 +275,6 @@ type-conv-pipeline: record_type: "event" format: "json" processor: - - grok: - match: - message: ['%{IPORHOST:clientip} \[%{HTTPDATE:timestamp}\] %{NUMBER:response_status}'] - convert_entry_type: key: "response_status" type: "integer" @@ -285,20 +282,15 @@ type-conv-pipeline: - stdout: ``` -Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). - -```json -{"message": "10.10.10.19 [19/Feb/2015:15:50:36 -0500] 200"} -``` +Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). -When you run the `Grok` processor, the processor parses the message into the following output: ```json -{"message": "10.10.10.10 [19/Feb/2015:15:50:36 -0500] 200", "clientip":"10.10.10.10", "timestamp": "19/Feb/2015:15:50:36 -0500", "response_status": "200"} +{"message": "value", "response_status":"200"} ``` The type conversion processor changes the output received into the following output, where the type of `response_status` value changes to an integer: ```json -{"message": "10.10.10.10 [19/Feb/2015:15:50:36 -0500] 200", "clientip":"10.10.10.10", "timestamp": "19/Feb/2015:15:50:36 -0500", "response_status": 200} +{"message":"value","response_status":200} ``` \ No newline at end of file From 5a29f34cb69fc195f0f2886a5dc7ab75ee39894a Mon Sep 17 00:00:00 2001 From: carolxob Date: Fri, 17 Mar 2023 13:54:55 -0500 Subject: [PATCH 18/56] Minor updates. Signed-off-by: carolxob --- .../pipelines/configuration/processors/mutate-event.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index a7e6fed463c..3763e48965c 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -18,7 +18,7 @@ The `add_entries` processor adds entries to an event. ### Configuration -You can configure the `AddEntries` processor with the following options. +You can configure the `add_entries` processor with the following options. Option | Required | Description :--- | :--- | :--- @@ -218,7 +218,7 @@ When you run the `RenameKey` processor, it parses the message into the following ### Special considerations -Renaming operations occur in the order that the key pair entries are listed in within the `pipelines.yaml` file. This means that chaining (where key pairs are renamed in sequence) is implicit with the `RenameKey` processor. See the following `piplines.yaml` file example: +Renaming operations occur in the order in which the key value pair entries are listed in the `pipelines.yaml` file. This means that chaining (where key value pairs are renamed in sequence) is implicit with the `RenameKey` processor. See the following `piplines.yaml` file example: ```yaml pipeline: @@ -243,6 +243,7 @@ Add the following contents to the `logs_json.log` file: ```json {"message": "value"} ``` +{% include copy.html %} After the processor runs, the following output appears: From d9a8dac98e41e9bcb2910e4aac8d2f3ad22665bd Mon Sep 17 00:00:00 2001 From: carolxob Date: Fri, 17 Mar 2023 14:12:43 -0500 Subject: [PATCH 19/56] Minor edits. Signed-off-by: carolxob --- .../pipelines/configuration/processors/mutate-event.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index 3763e48965c..56a5e2dd54b 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -253,7 +253,7 @@ After the processor runs, the following output appears: ## ConvertEntry -The `ConvertEntry` processor converts a value type associated with the specified key in a message to the specified type. It is a casting processor that changes the types of some fields in the event or message. Some of inputted data may need to be converted to different types, such as an integer or a double, or a string to an integer, so that it will pass the events through condition-based processors, or to perform conditional routing. +The `ConvertEntry` processor converts a value type associated with the specified key in a message to the specified type. It is a casting processor that changes the types of some fields in the event or message. Some entered data need to be converted to different types, such as an integer or a double, or a string to an integer, so that it will pass the events through condition-based processors or perform conditional routing. ### Configuration From ba2291e7fecaa79888bd3bc8a2a649a8799f9ee5 Mon Sep 17 00:00:00 2001 From: carolxob Date: Fri, 17 Mar 2023 16:11:56 -0500 Subject: [PATCH 20/56] Minor edits. Signed-off-by: carolxob --- .../pipelines/configuration/processors/mutate-event.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index 56a5e2dd54b..e7de1e8302f 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -47,6 +47,8 @@ pipeline: sink: - stdout: ``` +{% include copy.html %} + Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). @@ -103,6 +105,7 @@ pipeline: sink: - stdout: ``` +{% include copy.html %} Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). @@ -148,6 +151,7 @@ pipeline: sink: - stdout: ``` +{% include copy.html %} Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). From 901b80a96e5d7a24727258807e243475ff833dff Mon Sep 17 00:00:00 2001 From: carolxob Date: Fri, 17 Mar 2023 16:13:21 -0500 Subject: [PATCH 21/56] Added copy button to relevant code blocks. Signed-off-by: carolxob --- .../pipelines/configuration/processors/mutate-event.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index e7de1e8302f..d1277ec35a2 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -58,7 +58,7 @@ Before you run the `AddEntries` processor, you see the following: {"message": "value"} ``` - + When you run the `AddEntries` processor, it parses the message into the following output: @@ -203,6 +203,7 @@ pipeline: sink: - stdout: ``` +{% include copy.html %} Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). @@ -286,6 +287,7 @@ type-conv-pipeline: sink: - stdout: ``` +{% include copy.html %} Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). From 7bacf22fa4604dbe13cc77b8aa65422d29f7e2dd Mon Sep 17 00:00:00 2001 From: carolxob Date: Fri, 17 Mar 2023 16:22:26 -0500 Subject: [PATCH 22/56] Minor edits. Signed-off-by: carolxob --- .../configuration/processors/mutate-event.md | 56 +++++++++---------- 1 file changed, 27 insertions(+), 29 deletions(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index d1277ec35a2..ee5f4914dd3 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -22,10 +22,10 @@ You can configure the `add_entries` processor with the following options. Option | Required | Description :--- | :--- | :--- -|`entries` | Yes | A list of entries to add to an event. | -|`key` | Yes | The key of the new entry to be added. | -|`value` | Yes | The value of the new entry to be added. You can use the following data types: strings, booleans, numbers, null, nested objects, and arrays. | -|`overwrite_if_key_exists` | No | When set to `true`, if `key` already exists in the event, the existing value is overwritten. The default value is `false`. | +`entries` | Yes | A list of entries to add to an event. +`key` | Yes | The key of the new entry to be added. +`value` | Yes | The value of the new entry to be added. You can use the following data types: strings, booleans, numbers, null, nested objects, and arrays. +`overwrite_if_key_exists` | No | When set to `true`, if `key` already exists in the event, the existing value is overwritten. The default value is `false`. ### Usage @@ -55,20 +55,18 @@ Next, create a log file named `logs_json.log` and replace the `path` in the file Before you run the `AddEntries` processor, you see the following: ```json4 -{"message": "value"} +{"message": "hello"} ``` - - When you run the `AddEntries` processor, it parses the message into the following output: ```json -{"message": "value", "newMessage": 3} +{"message": "hello", "newMessage": 3} ``` > If `newMessage` already exists, its existing value is overwritten with a value of `3`. -In the preceding example, the `add_entries` processor adds a new entry `{"newMessage": 3}` to the existing event `{"message": "value"}` so that the new event contains two entries in the final output: `{"message": "value","newMessage": 3}`. +In the preceding example, the `add_entries` processor adds a new entry `{"newMessage": 3}` to the existing event `{"message": "hello"}` so that the new event contains two entries in the final output: `{"message": "hello","newMessage": 3}`. ## copy value @@ -80,10 +78,10 @@ You can configure the `copy value` processor with the following options. Option | Required | Description :--- | :--- | :--- -| `entries` | Yes | A list of entries to be copied in an event. -| `from_key` | Yes | The key of the entry to be copied. -| `to_key` | Yes | The key of the new entry to be added. -|`overwrite_if_key_exists` | No | When set to `true`, if `key` already exists in the event, the existing value is overwritten. The default value is `false`. | +`entries` | Yes | A list of entries to be copied in an event. +`from_key` | Yes | The key of the entry to be copied. +`to_key` | Yes | The key of the new entry to be added. +`overwrite_if_key_exists` | No | When set to `true`, if `key` already exists in the event, the existing value is overwritten. The default value is `false`. ### Usage @@ -110,16 +108,16 @@ pipeline: Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). ```json -{"message": "value"} +{"message": "hello"} ``` When you run this processor, it parses the message into the following output: ```json -{"message": "value", "newMessage": "value"} +{"message": "hello", "newMessage": "hello"} ``` -> If `newMessage` had already exists, its current value is overwritten with `value`. +> If `newMessage` already exists, its existing value is overwritten with `value`. ## DeleteEntry @@ -132,7 +130,7 @@ You can configure the `DeleteEntry` processor with the following options. Option | Required | Description :--- | :--- | :--- -| `with_keys` | Yes | An array of keys of the entries to be deleted. | +`with_keys` | Yes | An array of keys of the entries to be deleted. ### Usage @@ -156,13 +154,13 @@ pipeline: Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). ```json -{"message": "value", "message2": "value2"} +{"message": "hello", "message2": "goodbye"} ``` When you run the `DeleteEntry` processor, it parses the message into the following output: ```json -{"message2": "value2"} +{"message2": "goodbye"} ``` > If `message` does not exist in the event, then no action occurs. @@ -178,10 +176,10 @@ You can configure the `Rename Key` processor with the following options. Option | Required | Description :--- | :--- | :--- -| `entries` | Yes | A list of entries to rename in an event. | -| `from_key` | Yes | The key of the entry to be renamed. | -| `to_key` | Yes | The new key of the entry. -| `overwrite_if_to_key_exists` | No | When set to a value of`true`, if `to_key` already exists in the event, then the existing value will be overwritten. The default value is `false`. | +`entries` | Yes | A list of entries to rename in an event. +`from_key` | Yes | The key of the entry to be renamed. +`to_key` | Yes | The new key of the entry. +`overwrite_if_to_key_exists` | No | When set to a value of`true`, if `to_key` already exists in the event, then the existing value will be overwritten. The default value is `false`. ### Usage @@ -208,13 +206,13 @@ pipeline: Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). ```json -{"message": "value"} +{"message": "hello"} ``` When you run the `RenameKey` processor, it parses the message into the following output as the "newMessage": ```json -{"newMessage": "value"} +{"newMessage": "hello"} ``` > If `newMessage` already exists, its existing value is overwritten with `value`. @@ -246,14 +244,14 @@ pipeline: Add the following contents to the `logs_json.log` file: ```json -{"message": "value"} +{"message": "hello"} ``` {% include copy.html %} After the processor runs, the following output appears: ```json -{"message3": "value"} +{"message3": "hello"} ``` ## ConvertEntry @@ -266,8 +264,8 @@ You can configure the `ConvertEntry` processor with the following options. Option | Required | Description :--- | :--- | :--- -| `key`| Yes | Keys whose value needs to be converted to a different type. | -| `type` | No | Target type for key value pair. Possible values are `integer`, `double`, `string`, and `boolean`. Default value is `integer`. | +`key`| Yes | Keys whose value needs to be converted to a different type. +`type` | No | Target type for key value pair. Possible values are `integer`, `double`, `string`, and `boolean`. Default value is `integer`. ### Usage From d8e5b8aeeb9a2ab6624a703f25c65208c78b9a4a Mon Sep 17 00:00:00 2001 From: carolxob Date: Fri, 17 Mar 2023 16:35:39 -0500 Subject: [PATCH 23/56] Added links. Signed-off-by: carolxob --- .../configuration/processors/mutate-event.md | 26 +++++++++++-------- 1 file changed, 15 insertions(+), 11 deletions(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index ee5f4914dd3..76c43cc02df 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -8,17 +8,21 @@ nav_order: 45 # Mutate event processors -Mutate event processors allow you to modify events in Data Prepper. You can use these processors to add entries to an event, delete entries from an event, rename keys in an event, copy values within an event, and convert value types in an event. +Mutate event processors allow you to modify events in Data Prepper. You can use these processors to add entries to an event, delete entries from an event, rename keys in an event, copy values within an event, and convert value types in an event. The following processors allow you to mutate an event: -The following processors allow you to mutate an event. +* [AddEntries](##AddEntries) +* [CopyValues](##CopyValues) +* [DeleteEntry](##DeleteEntry) +* [RenameKey](##RenameKey) +* [ConvertEntry](##ConvertEntry) -## add_entries +## AddEntries -The `add_entries` processor adds entries to an event. +The `AddEntries` processor adds entries to an event. ### Configuration -You can configure the `add_entries` processor with the following options. +You can configure the `AddEntries` processor with the following options. Option | Required | Description :--- | :--- | :--- @@ -68,13 +72,13 @@ When you run the `AddEntries` processor, it parses the message into the followin In the preceding example, the `add_entries` processor adds a new entry `{"newMessage": 3}` to the existing event `{"message": "hello"}` so that the new event contains two entries in the final output: `{"message": "hello","newMessage": 3}`. -## copy value +## CopyValues -The `copy value` processor copies the values of an existing key within an event to another key. +The `CopyValues` processor copies the values of an existing key within an event to another key. ### Configuration -You can configure the `copy value` processor with the following options. +You can configure the `CopyValues` processor with the following options. Option | Required | Description :--- | :--- | :--- @@ -166,13 +170,13 @@ When you run the `DeleteEntry` processor, it parses the message into the followi > If `message` does not exist in the event, then no action occurs. -## Rename Key +## RenameKey -The `Rename Key` processor renames keys in an event. +The `RenameKey` processor renames keys in an event. ### Configuration -You can configure the `Rename Key` processor with the following options. +You can configure the `RenameKey` processor with the following options. Option | Required | Description :--- | :--- | :--- From 169f6509f4288e6838a2cbfaaefb93c9b1ccfdff Mon Sep 17 00:00:00 2001 From: carolxob Date: Fri, 17 Mar 2023 16:38:51 -0500 Subject: [PATCH 24/56] Minor edits. Signed-off-by: carolxob --- .../pipelines/configuration/processors/mutate-event.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index 76c43cc02df..64dd40e9600 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -10,11 +10,11 @@ nav_order: 45 Mutate event processors allow you to modify events in Data Prepper. You can use these processors to add entries to an event, delete entries from an event, rename keys in an event, copy values within an event, and convert value types in an event. The following processors allow you to mutate an event: -* [AddEntries](##AddEntries) -* [CopyValues](##CopyValues) -* [DeleteEntry](##DeleteEntry) -* [RenameKey](##RenameKey) -* [ConvertEntry](##ConvertEntry) +* [AddEntries](#addentries) +* [CopyValues](#copyvalues) +* [DeleteEntry](#deleteentry) +* [RenameKey](#renamekey) +* [ConvertEntry](#convertentry) ## AddEntries From 54c9011b1340508f709578f57f3f463257058c63 Mon Sep 17 00:00:00 2001 From: carolxob Date: Mon, 20 Mar 2023 11:32:16 -0600 Subject: [PATCH 25/56] Minor edits. Signed-off-by: carolxob --- .../configuration/processors/mutate-event.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index 64dd40e9600..f5445dbac90 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -24,12 +24,12 @@ The `AddEntries` processor adds entries to an event. You can configure the `AddEntries` processor with the following options. -Option | Required | Description +| Option | Required | Description | :--- | :--- | :--- -`entries` | Yes | A list of entries to add to an event. -`key` | Yes | The key of the new entry to be added. -`value` | Yes | The value of the new entry to be added. You can use the following data types: strings, booleans, numbers, null, nested objects, and arrays. -`overwrite_if_key_exists` | No | When set to `true`, if `key` already exists in the event, the existing value is overwritten. The default value is `false`. +| `entries` | Yes | A list of entries to add to an event. | +| `key` | Yes | The key of the new entry to be added. | +| `value` | Yes | The value of the new entry to be added. You can use the following data types: strings, booleans, numbers, null, nested objects, and arrays. | +| `overwrite_if_key_exists` | No | When set to `true`, if `key` already exists in the event, the existing value is overwritten. The default value is `false`. | ### Usage @@ -80,7 +80,7 @@ The `CopyValues` processor copies the values of an existing key within an event You can configure the `CopyValues` processor with the following options. -Option | Required | Description +| Option | Required | Description | :--- | :--- | :--- `entries` | Yes | A list of entries to be copied in an event. `from_key` | Yes | The key of the entry to be copied. From 18fd978378be14a8e360466be676cc7c5a08f0a3 Mon Sep 17 00:00:00 2001 From: carolxob Date: Mon, 20 Mar 2023 11:59:28 -0600 Subject: [PATCH 26/56] Minor updates. Signed-off-by: carolxob --- .../configuration/processors/mutate-event.md | 35 ++++++++++--------- 1 file changed, 18 insertions(+), 17 deletions(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index f5445dbac90..e3ff6d8fe45 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -25,7 +25,7 @@ The `AddEntries` processor adds entries to an event. You can configure the `AddEntries` processor with the following options. | Option | Required | Description | -:--- | :--- | :--- +| :--- | :--- | :--- | | `entries` | Yes | A list of entries to add to an event. | | `key` | Yes | The key of the new entry to be added. | | `value` | Yes | The value of the new entry to be added. You can use the following data types: strings, booleans, numbers, null, nested objects, and arrays. | @@ -56,13 +56,13 @@ pipeline: Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). -Before you run the `AddEntries` processor, you see the following: +For example, before you run the `AddEntries` processor, if the `logs_json.log` file contains the following event record: -```json4 +```json {"message": "hello"} ``` -When you run the `AddEntries` processor, it parses the message into the following output: +Then, when you run the `AddEntries` processor using the previous configuration, it parses the message into the following output: ```json {"message": "hello", "newMessage": 3} @@ -82,10 +82,10 @@ You can configure the `CopyValues` processor with the following options. | Option | Required | Description | :--- | :--- | :--- -`entries` | Yes | A list of entries to be copied in an event. -`from_key` | Yes | The key of the entry to be copied. -`to_key` | Yes | The key of the new entry to be added. -`overwrite_if_key_exists` | No | When set to `true`, if `key` already exists in the event, the existing value is overwritten. The default value is `false`. +| `entries` | Yes | A list of entries to be copied in an event. | +| `from_key` | Yes | The key of the entry to be copied. | +| `to_key` | Yes | The key of the new entry to be added. | +| `overwrite_if_key_exists` | No | When set to `true`, if `key` already exists in the event, the existing value is overwritten. The default value is `false`. | ### Usage @@ -132,9 +132,9 @@ The `DeleteEntry` processor deletes entries in an event, such as key-value pairs You can configure the `DeleteEntry` processor with the following options. -Option | Required | Description +| Option | Required | Description | :--- | :--- | :--- -`with_keys` | Yes | An array of keys of the entries to be deleted. +| `with_keys` | Yes | An array of keys of the entries to be deleted. | ### Usage @@ -178,12 +178,12 @@ The `RenameKey` processor renames keys in an event. You can configure the `RenameKey` processor with the following options. -Option | Required | Description -:--- | :--- | :--- -`entries` | Yes | A list of entries to rename in an event. -`from_key` | Yes | The key of the entry to be renamed. -`to_key` | Yes | The new key of the entry. -`overwrite_if_to_key_exists` | No | When set to a value of`true`, if `to_key` already exists in the event, then the existing value will be overwritten. The default value is `false`. +Option | Required | Description | +| :--- | :--- | :--- | +| `entries` | Yes | A list of entries to rename in an event. | +| `from_key` | Yes | The key of the entry to be renamed. | +| `to_key` | Yes | The new key of the entry. | +| `overwrite_if_to_key_exists` | No | When set to a value of`true`, if `to_key` already exists in the event, then the existing value will be overwritten. The default value is `false`. | ### Usage @@ -207,13 +207,14 @@ pipeline: ``` {% include copy.html %} + Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). ```json {"message": "hello"} ``` -When you run the `RenameKey` processor, it parses the message into the following output as the "newMessage": +When you run the `RenameKey` processor, it parses the message into the following "newMessage" output: ```json {"newMessage": "hello"} From 52e1a740f496658420f7a0bed41c75723313a16c Mon Sep 17 00:00:00 2001 From: carolxob Date: Mon, 20 Mar 2023 12:09:09 -0600 Subject: [PATCH 27/56] Minor edits. Signed-off-by: carolxob --- .../configuration/processors/mutate-event.md | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index e3ff6d8fe45..fffd8d05ebe 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -226,7 +226,7 @@ When you run the `RenameKey` processor, it parses the message into the following ### Special considerations -Renaming operations occur in the order in which the key value pair entries are listed in the `pipelines.yaml` file. This means that chaining (where key value pairs are renamed in sequence) is implicit with the `RenameKey` processor. See the following `piplines.yaml` file example: +Renaming operations occur in the order that the key value pair entries are listed in the `pipeline.yaml` file. This means that chaining (where key value pairs are renamed in sequence) is implicit with the `RenameKey` processor. See the following `pipline.yaml` file example: ```yaml pipeline: @@ -253,7 +253,7 @@ Add the following contents to the `logs_json.log` file: ``` {% include copy.html %} -After the processor runs, the following output appears: +After the `RenameKey` processor runs, the following output appears: ```json {"message3": "hello"} @@ -267,14 +267,14 @@ The `ConvertEntry` processor converts a value type associated with the specified You can configure the `ConvertEntry` processor with the following options. -Option | Required | Description -:--- | :--- | :--- -`key`| Yes | Keys whose value needs to be converted to a different type. -`type` | No | Target type for key value pair. Possible values are `integer`, `double`, `string`, and `boolean`. Default value is `integer`. +| Option | Required | Description | +| :--- | :--- | :--- | +| `key`| Yes | Keys whose value needs to be converted to a different type. | +| `type` | No | Target type for key value pair. Possible values are `integer`, `double`, `string`, and `boolean`. Default value is `integer`. | ### Usage -To get started with type conversion processor using Data Prepper, create the following `pipeline.yaml` file: +To get started, create the following `pipeline.yaml` file: ```yaml type-conv-pipeline: @@ -294,12 +294,11 @@ type-conv-pipeline: Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). - ```json {"message": "value", "response_status":"200"} ``` -The type conversion processor changes the output received into the following output, where the type of `response_status` value changes to an integer: +The `ConvertEntry` processor changes the output received into the following output, where the type of `response_status` value changes from a string to an integer: ```json {"message":"value","response_status":200} From a86522144d14dad9d50989e3c082305ca094dc4e Mon Sep 17 00:00:00 2001 From: carolxob Date: Thu, 23 Mar 2023 14:24:55 -0600 Subject: [PATCH 28/56] Minor updates. Signed-off-by: carolxob --- .../pipelines/configuration/processors/mutate-event.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index fffd8d05ebe..443c84da055 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -62,7 +62,7 @@ For example, before you run the `AddEntries` processor, if the `logs_json.log` f {"message": "hello"} ``` -Then, when you run the `AddEntries` processor using the previous configuration, it parses the message into the following output: +Then, when you run the `AddEntries` processor using the previous configuration, it adds a new entry `{"newMessage": 3}` to the existing event `{"message": "hello"}` so that the new event contains two entries in the final output: ```json {"message": "hello", "newMessage": 3} @@ -70,7 +70,6 @@ Then, when you run the `AddEntries` processor using the previous configuration, > If `newMessage` already exists, its existing value is overwritten with a value of `3`. -In the preceding example, the `add_entries` processor adds a new entry `{"newMessage": 3}` to the existing event `{"message": "hello"}` so that the new event contains two entries in the final output: `{"message": "hello","newMessage": 3}`. ## CopyValues From 7bcaa95479abbc93e147ff89ce7ca541db98cee2 Mon Sep 17 00:00:00 2001 From: carolxob Date: Wed, 29 Mar 2023 11:08:55 -0600 Subject: [PATCH 29/56] Minor edits based on doc review. Signed-off-by: carolxob --- .../configuration/processors/mutate-event.md | 22 +++++++++---------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index 443c84da055..bb15901eb4a 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -8,13 +8,13 @@ nav_order: 45 # Mutate event processors -Mutate event processors allow you to modify events in Data Prepper. You can use these processors to add entries to an event, delete entries from an event, rename keys in an event, copy values within an event, and convert value types in an event. The following processors allow you to mutate an event: +Mutate event processors allow you to modify events in Data Prepper. The following processors are available: -* [AddEntries](#addentries) -* [CopyValues](#copyvalues) -* [DeleteEntry](#deleteentry) -* [RenameKey](#renamekey) -* [ConvertEntry](#convertentry) +* [AddEntries](#addentries): Allows you to add entries to an event. +* [CopyValues](#copyvalues): Allows you to copy values within an event. +* [DeleteEntry](#deleteentry): Allows you to delete entries from an event. +* [RenameKey](#renamekey): Allows you to rename keys in an event. +* [ConvertEntry](#convertentry): Allows you to convert value types in an event. ## AddEntries @@ -24,12 +24,12 @@ The `AddEntries` processor adds entries to an event. You can configure the `AddEntries` processor with the following options. -| Option | Required | Description | +| Option | Required | Description | Example | | :--- | :--- | :--- | -| `entries` | Yes | A list of entries to add to an event. | -| `key` | Yes | The key of the new entry to be added. | -| `value` | Yes | The value of the new entry to be added. You can use the following data types: strings, booleans, numbers, null, nested objects, and arrays. | -| `overwrite_if_key_exists` | No | When set to `true`, if `key` already exists in the event, the existing value is overwritten. The default value is `false`. | +| `entries` | Yes | A list of entries to add to an event. | | +| `key` | Yes | The key of the new entry to be added. | | +| `value` | Yes | The value of the new entry to be added. You can use the following data types: strings, booleans, numbers, null, nested objects, and arrays. | | +| `overwrite_if_key_exists` | No | When set to `true`, if `key` already exists in the event, the existing value is overwritten. The default value is `false`. | | ### Usage From 5167be12002441be494463362b976c1717bd77c4 Mon Sep 17 00:00:00 2001 From: carolxob Date: Wed, 29 Mar 2023 11:12:03 -0600 Subject: [PATCH 30/56] Minor edits based on doc review feedback. Signed-off-by: carolxob --- .../pipelines/configuration/processors/mutate-event.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index bb15901eb4a..4537bca501b 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -29,7 +29,7 @@ You can configure the `AddEntries` processor with the following options. | `entries` | Yes | A list of entries to add to an event. | | | `key` | Yes | The key of the new entry to be added. | | | `value` | Yes | The value of the new entry to be added. You can use the following data types: strings, booleans, numbers, null, nested objects, and arrays. | | -| `overwrite_if_key_exists` | No | When set to `true`, if `key` already exists in the event, the existing value is overwritten. The default value is `false`. | | +| `overwrite_if_key_exists` | No | When set to `true`, the existing value is overwritten if `key` already exists in the event. The default value is `false`. | | ### Usage @@ -84,7 +84,7 @@ You can configure the `CopyValues` processor with the following options. | `entries` | Yes | A list of entries to be copied in an event. | | `from_key` | Yes | The key of the entry to be copied. | | `to_key` | Yes | The key of the new entry to be added. | -| `overwrite_if_key_exists` | No | When set to `true`, if `key` already exists in the event, the existing value is overwritten. The default value is `false`. | +| `overwrite_if_key_exists` | No | When set to `true`, the existing value is overwritten if `key` already exists in the event. The default value is `false`. | ### Usage @@ -182,7 +182,7 @@ Option | Required | Description | | `entries` | Yes | A list of entries to rename in an event. | | `from_key` | Yes | The key of the entry to be renamed. | | `to_key` | Yes | The new key of the entry. | -| `overwrite_if_to_key_exists` | No | When set to a value of`true`, if `to_key` already exists in the event, then the existing value will be overwritten. The default value is `false`. | +| `overwrite_if_to_key_exists` | No | When set to `true`, the existing value is overwritten if `key` already exists in the event. The default value is `false`. | ### Usage From f98259ef5953b219d0fb83f071fec79067568926 Mon Sep 17 00:00:00 2001 From: carolxob Date: Wed, 29 Mar 2023 11:18:12 -0600 Subject: [PATCH 31/56] Edits made based on doc review feedback. Signed-off-by: carolxob --- .../pipelines/configuration/processors/mutate-event.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index 4537bca501b..be69ea4bfc1 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -125,7 +125,7 @@ When you run this processor, it parses the message into the following output: ## DeleteEntry -The `DeleteEntry` processor deletes entries in an event, such as key-value pairs. You can define the keys you want to delete in the `with-keys` field following `delete_entries` in the YAML configuration file. Those keys along with their values are deleted. +The `DeleteEntry` processor deletes entries from an event, such as key-value pairs. You can define the keys you want to delete in the `with-keys` field following `delete_entries` in the YAML configuration file. Those keys along with their values are deleted. ### Configuration @@ -260,7 +260,7 @@ After the `RenameKey` processor runs, the following output appears: ## ConvertEntry -The `ConvertEntry` processor converts a value type associated with the specified key in a message to the specified type. It is a casting processor that changes the types of some fields in the event or message. Some entered data need to be converted to different types, such as an integer or a double, or a string to an integer, so that it will pass the events through condition-based processors or perform conditional routing. +The `ConvertEntry` processor converts a value type associated with the specified key in a message to the specified type. It is a casting processor that changes the types of some fields in the event or message. Some entered data needs to be converted to different types, such as an integer or a double, or a string to an integer, so that it will pass the events through condition-based processors or perform conditional routing. ### Configuration From f1ee5d9dfd2bf1f978732cd70f64f38ecfd0f182 Mon Sep 17 00:00:00 2001 From: carolxob Date: Fri, 31 Mar 2023 09:10:32 -0600 Subject: [PATCH 32/56] Minor update to include key examples. Signed-off-by: carolxob --- .../pipelines/configuration/processors/mutate-event.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index be69ea4bfc1..141243d6b44 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -24,12 +24,12 @@ The `AddEntries` processor adds entries to an event. You can configure the `AddEntries` processor with the following options. -| Option | Required | Description | Example | +| Option | Required | Description | | :--- | :--- | :--- | -| `entries` | Yes | A list of entries to add to an event. | | -| `key` | Yes | The key of the new entry to be added. | | -| `value` | Yes | The value of the new entry to be added. You can use the following data types: strings, booleans, numbers, null, nested objects, and arrays. | | -| `overwrite_if_key_exists` | No | When set to `true`, the existing value is overwritten if `key` already exists in the event. The default value is `false`. | | +| `entries` | Yes | A list of entries to add to an event. | +| `key` | Yes | The key of the new entry to be added. Some examples of keys include: `my_key`, `myKey`, and `object/sub_Key`. | +| `value` | Yes | The value of the new entry to be added. You can use the following data types: strings, booleans, numbers, null, nested objects, and arrays. | +| `overwrite_if_key_exists` | No | When set to `true`, the existing value is overwritten if `key` already exists in the event. The default value is `false`. | ### Usage From 4e34b0263b908a37503ec995acfb2fdf76a8a5a0 Mon Sep 17 00:00:00 2001 From: Caroline <113052567+carolxob@users.noreply.github.com> Date: Fri, 31 Mar 2023 10:41:46 -0600 Subject: [PATCH 33/56] Update _data-prepper/pipelines/configuration/processors/mutate-event.md Co-authored-by: Nathan Bower --- .../pipelines/configuration/processors/mutate-event.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index 141243d6b44..d688dcf2e41 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -10,7 +10,7 @@ nav_order: 45 Mutate event processors allow you to modify events in Data Prepper. The following processors are available: -* [AddEntries](#addentries): Allows you to add entries to an event. +* [AddEntries](#addentries) allows you to add entries to an event. * [CopyValues](#copyvalues): Allows you to copy values within an event. * [DeleteEntry](#deleteentry): Allows you to delete entries from an event. * [RenameKey](#renamekey): Allows you to rename keys in an event. From 4f76945992837e70dfe25b42e4c1320d931c8b8c Mon Sep 17 00:00:00 2001 From: Caroline <113052567+carolxob@users.noreply.github.com> Date: Fri, 31 Mar 2023 10:41:59 -0600 Subject: [PATCH 34/56] Update _data-prepper/pipelines/configuration/processors/mutate-event.md Co-authored-by: Nathan Bower --- .../pipelines/configuration/processors/mutate-event.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index d688dcf2e41..2ce14853d15 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -11,7 +11,7 @@ nav_order: 45 Mutate event processors allow you to modify events in Data Prepper. The following processors are available: * [AddEntries](#addentries) allows you to add entries to an event. -* [CopyValues](#copyvalues): Allows you to copy values within an event. +* [CopyValues](#copyvalues) allows you to copy values within an event. * [DeleteEntry](#deleteentry): Allows you to delete entries from an event. * [RenameKey](#renamekey): Allows you to rename keys in an event. * [ConvertEntry](#convertentry): Allows you to convert value types in an event. From 3bd95360e7337706732c347c4565902715a99daa Mon Sep 17 00:00:00 2001 From: Caroline <113052567+carolxob@users.noreply.github.com> Date: Fri, 31 Mar 2023 10:45:09 -0600 Subject: [PATCH 35/56] Update _data-prepper/pipelines/configuration/processors/mutate-event.md Co-authored-by: Nathan Bower --- .../pipelines/configuration/processors/mutate-event.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index 2ce14853d15..786fab5f2d5 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -12,7 +12,7 @@ Mutate event processors allow you to modify events in Data Prepper. The followin * [AddEntries](#addentries) allows you to add entries to an event. * [CopyValues](#copyvalues) allows you to copy values within an event. -* [DeleteEntry](#deleteentry): Allows you to delete entries from an event. +* [DeleteEntry](#deleteentry) allows you to delete entries from an event. * [RenameKey](#renamekey): Allows you to rename keys in an event. * [ConvertEntry](#convertentry): Allows you to convert value types in an event. From 6154709b73cf5ad4dd6545f98148b82589cd7d42 Mon Sep 17 00:00:00 2001 From: Caroline <113052567+carolxob@users.noreply.github.com> Date: Fri, 31 Mar 2023 10:45:21 -0600 Subject: [PATCH 36/56] Update _data-prepper/pipelines/configuration/processors/mutate-event.md Co-authored-by: Nathan Bower --- .../pipelines/configuration/processors/mutate-event.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index 786fab5f2d5..2aa705c66a0 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -13,7 +13,7 @@ Mutate event processors allow you to modify events in Data Prepper. The followin * [AddEntries](#addentries) allows you to add entries to an event. * [CopyValues](#copyvalues) allows you to copy values within an event. * [DeleteEntry](#deleteentry) allows you to delete entries from an event. -* [RenameKey](#renamekey): Allows you to rename keys in an event. +* [RenameKey](#renamekey) allows you to rename keys in an event. * [ConvertEntry](#convertentry): Allows you to convert value types in an event. ## AddEntries From 94e3fe1a629aebf759fdda5b806a246b46ecd9d5 Mon Sep 17 00:00:00 2001 From: Caroline <113052567+carolxob@users.noreply.github.com> Date: Fri, 31 Mar 2023 10:45:45 -0600 Subject: [PATCH 37/56] Update _data-prepper/pipelines/configuration/processors/mutate-event.md Co-authored-by: Nathan Bower --- .../pipelines/configuration/processors/mutate-event.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index 2aa705c66a0..e30669340bd 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -14,7 +14,7 @@ Mutate event processors allow you to modify events in Data Prepper. The followin * [CopyValues](#copyvalues) allows you to copy values within an event. * [DeleteEntry](#deleteentry) allows you to delete entries from an event. * [RenameKey](#renamekey) allows you to rename keys in an event. -* [ConvertEntry](#convertentry): Allows you to convert value types in an event. +* [ConvertEntry](#convertentry) allows you to convert value types in an event. ## AddEntries From ce7d1e0de09d3cbe940fe6f38b913f701408189d Mon Sep 17 00:00:00 2001 From: Caroline <113052567+carolxob@users.noreply.github.com> Date: Fri, 31 Mar 2023 10:46:49 -0600 Subject: [PATCH 38/56] Update _data-prepper/pipelines/configuration/processors/mutate-event.md Co-authored-by: Nathan Bower --- .../pipelines/configuration/processors/mutate-event.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index e30669340bd..dc9bb2c5fe7 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -27,7 +27,7 @@ You can configure the `AddEntries` processor with the following options. | Option | Required | Description | | :--- | :--- | :--- | | `entries` | Yes | A list of entries to add to an event. | -| `key` | Yes | The key of the new entry to be added. Some examples of keys include: `my_key`, `myKey`, and `object/sub_Key`. | +| `key` | Yes | The key of the new entry to be added. Some examples of keys include `my_key`, `myKey`, and `object/sub_Key`. | | `value` | Yes | The value of the new entry to be added. You can use the following data types: strings, booleans, numbers, null, nested objects, and arrays. | | `overwrite_if_key_exists` | No | When set to `true`, the existing value is overwritten if `key` already exists in the event. The default value is `false`. | From cf16ba6cbd685cb1f124d3fd0dc2fafb3e5e1545 Mon Sep 17 00:00:00 2001 From: Caroline <113052567+carolxob@users.noreply.github.com> Date: Fri, 31 Mar 2023 10:48:34 -0600 Subject: [PATCH 39/56] Update _data-prepper/pipelines/configuration/processors/mutate-event.md Co-authored-by: Nathan Bower --- .../pipelines/configuration/processors/mutate-event.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index dc9bb2c5fe7..351919aa020 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -28,7 +28,7 @@ You can configure the `AddEntries` processor with the following options. | :--- | :--- | :--- | | `entries` | Yes | A list of entries to add to an event. | | `key` | Yes | The key of the new entry to be added. Some examples of keys include `my_key`, `myKey`, and `object/sub_Key`. | -| `value` | Yes | The value of the new entry to be added. You can use the following data types: strings, booleans, numbers, null, nested objects, and arrays. | +| `value` | Yes | The value of the new entry to be added. You can use the following data types: strings, Booleans, numbers, null, nested objects, and arrays. | | `overwrite_if_key_exists` | No | When set to `true`, the existing value is overwritten if `key` already exists in the event. The default value is `false`. | ### Usage From 376295520e62767d94f5059714b01eb88ad299ca Mon Sep 17 00:00:00 2001 From: Caroline <113052567+carolxob@users.noreply.github.com> Date: Fri, 31 Mar 2023 10:51:22 -0600 Subject: [PATCH 40/56] Update _data-prepper/pipelines/configuration/processors/mutate-event.md Co-authored-by: Nathan Bower --- .../pipelines/configuration/processors/mutate-event.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index 351919aa020..5604a49371e 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -62,7 +62,7 @@ For example, before you run the `AddEntries` processor, if the `logs_json.log` f {"message": "hello"} ``` -Then, when you run the `AddEntries` processor using the previous configuration, it adds a new entry `{"newMessage": 3}` to the existing event `{"message": "hello"}` so that the new event contains two entries in the final output: +Then when you run the `AddEntries` processor using the previous configuration, it adds a new entry `{"newMessage": 3}` to the existing event `{"message": "hello"}` so that the new event contains two entries in the final output: ```json {"message": "hello", "newMessage": 3} From 8130321652baea929158293a5d8948dfe988a482 Mon Sep 17 00:00:00 2001 From: Caroline <113052567+carolxob@users.noreply.github.com> Date: Fri, 31 Mar 2023 10:52:20 -0600 Subject: [PATCH 41/56] Update _data-prepper/pipelines/configuration/processors/mutate-event.md Co-authored-by: Nathan Bower --- .../pipelines/configuration/processors/mutate-event.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index 5604a49371e..f4b17a859c8 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -54,7 +54,7 @@ pipeline: {% include copy.html %} -Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). +Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` file with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). For example, before you run the `AddEntries` processor, if the `logs_json.log` file contains the following event record: From ae3c018c7482098fb872aac31cd81902cce56696 Mon Sep 17 00:00:00 2001 From: carolxob Date: Fri, 31 Mar 2023 10:56:39 -0600 Subject: [PATCH 42/56] Fixing commit. Signed-off-by: carolxob --- .../pipelines/configuration/processors/mutate-event.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index f4b17a859c8..606a0ede445 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -54,7 +54,11 @@ pipeline: {% include copy.html %} +<<<<<<< HEAD Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` file with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). +======= +Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with that filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). +>>>>>>> fc2247c2 (Incorporated editorial review feedback.) For example, before you run the `AddEntries` processor, if the `logs_json.log` file contains the following event record: From 3f6402312ba59280703a1e2f417d439d26f57954 Mon Sep 17 00:00:00 2001 From: carolxob Date: Fri, 31 Mar 2023 11:02:47 -0600 Subject: [PATCH 43/56] Minor edits. Signed-off-by: carolxob --- .../pipelines/configuration/processors/mutate-event.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index 606a0ede445..22770b7f923 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -54,11 +54,7 @@ pipeline: {% include copy.html %} -<<<<<<< HEAD -Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` file with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). -======= Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with that filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). ->>>>>>> fc2247c2 (Incorporated editorial review feedback.) For example, before you run the `AddEntries` processor, if the `logs_json.log` file contains the following event record: From 46ce3a733c975b297a2fdc7e8befe7c5f380e35f Mon Sep 17 00:00:00 2001 From: Caroline <113052567+carolxob@users.noreply.github.com> Date: Fri, 31 Mar 2023 11:04:49 -0600 Subject: [PATCH 44/56] Update _data-prepper/pipelines/configuration/processors/mutate-event.md Co-authored-by: Nathan Bower --- .../pipelines/configuration/processors/mutate-event.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index 22770b7f923..0e6c9e06ff4 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -108,7 +108,7 @@ pipeline: ``` {% include copy.html %} -Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). +Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` file with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). ```json {"message": "hello"} From c2828a420c8bd82de788baf5d752b3c5f0071d1d Mon Sep 17 00:00:00 2001 From: Caroline <113052567+carolxob@users.noreply.github.com> Date: Fri, 31 Mar 2023 11:05:45 -0600 Subject: [PATCH 45/56] Update _data-prepper/pipelines/configuration/processors/mutate-event.md Co-authored-by: Nathan Bower --- .../pipelines/configuration/processors/mutate-event.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index 0e6c9e06ff4..e561f5931f6 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -125,7 +125,7 @@ When you run this processor, it parses the message into the following output: ## DeleteEntry -The `DeleteEntry` processor deletes entries from an event, such as key-value pairs. You can define the keys you want to delete in the `with-keys` field following `delete_entries` in the YAML configuration file. Those keys along with their values are deleted. +The `DeleteEntry` processor deletes entries from an event, such as key-value pairs. You can define the keys you want to delete in the `with-keys` field following `delete_entries` in the YAML configuration file. Those keys and their values are deleted. ### Configuration From 6e7e912f9650e48fd73dc65f4144cf3dbd43373a Mon Sep 17 00:00:00 2001 From: Caroline <113052567+carolxob@users.noreply.github.com> Date: Fri, 31 Mar 2023 11:05:57 -0600 Subject: [PATCH 46/56] Update _data-prepper/pipelines/configuration/processors/mutate-event.md Co-authored-by: Nathan Bower --- .../pipelines/configuration/processors/mutate-event.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index e561f5931f6..74eae1204b1 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -269,7 +269,7 @@ You can configure the `ConvertEntry` processor with the following options. | Option | Required | Description | | :--- | :--- | :--- | | `key`| Yes | Keys whose value needs to be converted to a different type. | -| `type` | No | Target type for key value pair. Possible values are `integer`, `double`, `string`, and `boolean`. Default value is `integer`. | +| `type` | No | Target type for the key-value pair. Possible values are `integer`, `double`, `string`, and `boolean`. Default value is `integer`. | ### Usage From 3f12c9f6c858819208df502b2c1a174b76c302ed Mon Sep 17 00:00:00 2001 From: Caroline <113052567+carolxob@users.noreply.github.com> Date: Fri, 31 Mar 2023 11:06:29 -0600 Subject: [PATCH 47/56] Update _data-prepper/pipelines/configuration/processors/mutate-event.md Co-authored-by: Nathan Bower --- .../pipelines/configuration/processors/mutate-event.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index 74eae1204b1..c85efe7a4ee 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -154,7 +154,7 @@ pipeline: ``` {% include copy.html %} -Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). +Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` file with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). ```json {"message": "hello", "message2": "goodbye"} From ca11e2771cea6651f8034c5cc8ece0c39e91caec Mon Sep 17 00:00:00 2001 From: Caroline <113052567+carolxob@users.noreply.github.com> Date: Fri, 31 Mar 2023 11:09:22 -0600 Subject: [PATCH 48/56] Update _data-prepper/pipelines/configuration/processors/mutate-event.md Co-authored-by: Nathan Bower --- .../pipelines/configuration/processors/mutate-event.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index c85efe7a4ee..e3b4467a116 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -291,7 +291,7 @@ type-conv-pipeline: ``` {% include copy.html %} -Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). +Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` file with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). ```json {"message": "value", "response_status":"200"} From a74d6ffca965f1c2a7704cc37ed6925549454d1c Mon Sep 17 00:00:00 2001 From: Caroline <113052567+carolxob@users.noreply.github.com> Date: Fri, 31 Mar 2023 11:09:35 -0600 Subject: [PATCH 49/56] Update _data-prepper/pipelines/configuration/processors/mutate-event.md Co-authored-by: Nathan Bower --- .../pipelines/configuration/processors/mutate-event.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index e3b4467a116..80f763c82d8 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -297,7 +297,7 @@ Next, create a log file named `logs_json.log` and replace the `path` in the file {"message": "value", "response_status":"200"} ``` -The `ConvertEntry` processor changes the output received into the following output, where the type of `response_status` value changes from a string to an integer: +The `ConvertEntry` processor converts the output received to the following output, where the type of `response_status` value changes from a string to an integer: ```json {"message":"value","response_status":200} From 3c9314f29f743ce425cf472fd38853703dda1ca1 Mon Sep 17 00:00:00 2001 From: Caroline <113052567+carolxob@users.noreply.github.com> Date: Fri, 31 Mar 2023 11:15:10 -0600 Subject: [PATCH 50/56] Update _data-prepper/pipelines/configuration/processors/mutate-event.md Co-authored-by: Nathan Bower --- .../pipelines/configuration/processors/mutate-event.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index 80f763c82d8..b59dd9acb7d 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -179,7 +179,7 @@ You can configure the `RenameKey` processor with the following options. Option | Required | Description | | :--- | :--- | :--- | -| `entries` | Yes | A list of entries to rename in an event. | +| `entries` | Yes | A list of event entries to rename. | | `from_key` | Yes | The key of the entry to be renamed. | | `to_key` | Yes | The new key of the entry. | | `overwrite_if_to_key_exists` | No | When set to `true`, the existing value is overwritten if `key` already exists in the event. The default value is `false`. | From 70d7fd3ed9da2a77a4ee06129d8dad73244d76ff Mon Sep 17 00:00:00 2001 From: Caroline <113052567+carolxob@users.noreply.github.com> Date: Fri, 31 Mar 2023 11:15:55 -0600 Subject: [PATCH 51/56] Update _data-prepper/pipelines/configuration/processors/mutate-event.md Co-authored-by: Nathan Bower --- .../pipelines/configuration/processors/mutate-event.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index b59dd9acb7d..c8175e7b038 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -207,7 +207,7 @@ pipeline: {% include copy.html %} -Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). +Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` file with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). ```json {"message": "hello"} From 6e466432731ce57f74ebd7d7362df82bb7644396 Mon Sep 17 00:00:00 2001 From: Caroline <113052567+carolxob@users.noreply.github.com> Date: Fri, 31 Mar 2023 11:17:35 -0600 Subject: [PATCH 52/56] Update _data-prepper/pipelines/configuration/processors/mutate-event.md Co-authored-by: Nathan Bower --- .../pipelines/configuration/processors/mutate-event.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index c8175e7b038..d6c3cd7c30f 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -225,7 +225,7 @@ When you run the `RenameKey` processor, it parses the message into the following ### Special considerations -Renaming operations occur in the order that the key value pair entries are listed in the `pipeline.yaml` file. This means that chaining (where key value pairs are renamed in sequence) is implicit with the `RenameKey` processor. See the following `pipline.yaml` file example: +Renaming operations occur in the order that the key-value pair entries are listed in the `pipeline.yaml` file. This means that chaining (where key-value pairs are renamed in sequence) is implicit in the `RenameKey` processor. See the following example `pipline.yaml` file: ```yaml pipeline: From 5d972d65d6e9f1b112d2894c67d4df1f63f34306 Mon Sep 17 00:00:00 2001 From: Caroline <113052567+carolxob@users.noreply.github.com> Date: Fri, 31 Mar 2023 11:19:41 -0600 Subject: [PATCH 53/56] Update _data-prepper/pipelines/configuration/processors/mutate-event.md Co-authored-by: Nathan Bower --- .../pipelines/configuration/processors/mutate-event.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index d6c3cd7c30f..c7d3b7c3fd0 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -260,7 +260,7 @@ After the `RenameKey` processor runs, the following output appears: ## ConvertEntry -The `ConvertEntry` processor converts a value type associated with the specified key in a message to the specified type. It is a casting processor that changes the types of some fields in the event or message. Some entered data needs to be converted to different types, such as an integer or a double, or a string to an integer, so that it will pass the events through condition-based processors or perform conditional routing. +The `ConvertEntry` processor converts a value type associated with the specified message key to the specified type. It is a casting processor that changes the types of some fields in an event or message. Some entered data needs to be converted to a different type, such as an integer to a double or a string to an integer, so that it will pass the events through condition-based processors or perform conditional routing. ### Configuration From cabfd54dd6bf2e6aa9a4bdee556641e6d25f98ab Mon Sep 17 00:00:00 2001 From: carolxob Date: Fri, 31 Mar 2023 14:37:14 -0600 Subject: [PATCH 54/56] Incorporating editorial review changes. Signed-off-by: carolxob --- .../pipelines/configuration/processors/mutate-event.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index c7d3b7c3fd0..4db3fa5ff84 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -108,7 +108,9 @@ pipeline: ``` {% include copy.html %} -Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` file with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). +Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` file with that filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). + +For example, before you run the `AddEntries` processor, if the `logs_json.log` file contains the following event record: ```json {"message": "hello"} From bc172862e4de5b09b0bf3976ed516848bf7bf3f1 Mon Sep 17 00:00:00 2001 From: carolxob Date: Fri, 31 Mar 2023 15:07:50 -0600 Subject: [PATCH 55/56] Incorporated the rest of editorial feedback received. Signed-off-by: carolxob --- .../configuration/processors/mutate-event.md | 23 ++++++++++++------- 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index 4db3fa5ff84..4c243c819c6 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -54,7 +54,7 @@ pipeline: {% include copy.html %} -Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` with that filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). +Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` file with that filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). For example, before you run the `AddEntries` processor, if the `logs_json.log` file contains the following event record: @@ -110,7 +110,7 @@ pipeline: Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` file with that filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). -For example, before you run the `AddEntries` processor, if the `logs_json.log` file contains the following event record: +For example, before you run the `CopyValues` processor, if the `logs_json.log` file contains the following event record: ```json {"message": "hello"} @@ -127,7 +127,7 @@ When you run this processor, it parses the message into the following output: ## DeleteEntry -The `DeleteEntry` processor deletes entries from an event, such as key-value pairs. You can define the keys you want to delete in the `with-keys` field following `delete_entries` in the YAML configuration file. Those keys and their values are deleted. +The `DeleteEntry` processor deletes entries, such as key-value pairs, from an event. You can define the keys you want to delete in the `with-keys` field following `delete_entries` in the YAML configuration file. Those keys and their values are deleted. ### Configuration @@ -135,7 +135,7 @@ You can configure the `DeleteEntry` processor with the following options. | Option | Required | Description | :--- | :--- | :--- -| `with_keys` | Yes | An array of keys of the entries to be deleted. | +| `with_keys` | Yes | An array of keys for the entries to be deleted. | ### Usage @@ -156,7 +156,9 @@ pipeline: ``` {% include copy.html %} -Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` file with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). +Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` file with that filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). + +For example, before you run the `DeleteEntry` processor, if the `logs_json.log` file contains the following event record: ```json {"message": "hello", "message2": "goodbye"} @@ -209,7 +211,9 @@ pipeline: {% include copy.html %} -Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` file with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). +Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` file with that filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). + +For example, before you run the `RenameKey` processor, if the `logs_json.log` file contains the following event record: ```json {"message": "hello"} @@ -271,7 +275,7 @@ You can configure the `ConvertEntry` processor with the following options. | Option | Required | Description | | :--- | :--- | :--- | | `key`| Yes | Keys whose value needs to be converted to a different type. | -| `type` | No | Target type for the key-value pair. Possible values are `integer`, `double`, `string`, and `boolean`. Default value is `integer`. | +| `type` | No | Target type for the key-value pair. Possible values are `integer`, `double`, `string`, and `Boolean`. Default value is `integer`. | ### Usage @@ -293,7 +297,10 @@ type-conv-pipeline: ``` {% include copy.html %} -Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` file with this filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). +Next, create a log file named `logs_json.log` and replace the `path` in the file source of your `pipeline.yaml` file with that filepath. For more information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper). + +For example, before you run the `ConvertEntry` processor, if the `logs_json.log` file contains the following event record: + ```json {"message": "value", "response_status":"200"} From ada07240d071d7e060d5e11fb72181b867913515 Mon Sep 17 00:00:00 2001 From: carolxob Date: Mon, 3 Apr 2023 12:40:00 -0600 Subject: [PATCH 56/56] Minor updates based on feedback comments. Signed-off-by: carolxob --- .../pipelines/configuration/processors/mutate-event.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index 4c243c819c6..dffb64cb234 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -266,7 +266,7 @@ After the `RenameKey` processor runs, the following output appears: ## ConvertEntry -The `ConvertEntry` processor converts a value type associated with the specified message key to the specified type. It is a casting processor that changes the types of some fields in an event or message. Some entered data needs to be converted to a different type, such as an integer to a double or a string to an integer, so that it will pass the events through condition-based processors or perform conditional routing. +The `ConvertEntry` processor converts a value type associated with the specified key in a event to the specified type. It is a casting processor that changes the types of some fields in events. Some data must be converted to a different type, such as an integer to a double, or a string to an integer, so that it will pass the events through condition-based processors or perform conditional routing. ### Configuration