Update documentation for delete_entries and select_entries processors

_data-prepper/pipelines/configuration/processors/delete-entries.md

@@ -8,7 +8,11 @@
 
 # Delete entries processor
 
-The `delete_entries` processor deletes entries, such as key-value pairs, from an event. You can define the keys you want to delete in the `with-keys` field following `delete_entries` in the YAML configuration file. Those keys and their values are deleted. 
+The `delete_entries` processor removes entries, such as key-value pairs, from an event. Use the `with_keys` field to specify
+the exact keys to delete. To delete keys that match a regular expression pattern, use the `with_keys_regex` field. You can
+prevent deletion of specific events when using regular expressions by configuring the `exclude_from_regex` field.
+
+To use both `with_keys` and `with_keys_regex` in the same configuration, define them under the `entries` field.
 
 ## Configuration
 
@@ -21,9 +25,15 @@
 - source: https://github.com/opensearch-project/data-prepper/blob/c4455a7785bc2da4358067c217be7085e0bc8d0f/data-prepper-plugins/mutate-event-processors/src/main/java/org/opensearch/dataprepper/plugins/processor/mutateevent/DeleteEntryProcessorConfig.java
 -->
 
-| Option | Required | Description |
-:--- | :--- | :---
-| `with_keys` | Yes | An array of keys for the entries to be deleted. |
+| Option | Required | Description                                                                                                                                                                                     |
+:--- |:---------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+| `with_keys` | No       | An array that specifies the keys of the entries to delete.                                                                                                                                      |
+| `with_keys_regex` | No       | An array of regular expression (regex) patterns used to match the keys of entries to delete.                                                                                                    |
+| `exclude_from_delete` | No | A set of entries to exclude from deletion when using the `with_keys_regex` configuration.                                                                                                       |
+| `entries` | No | A list of entries to delete from the event.                                                                                                                                                     |
+| `delete_when` | No | Specifies under what conditions the deletion should be perfomed. (Example: <i>value= "/some_key == null"</i> - Deletion runs only if <i>some_key</i> is null or does not exist.)                |
+|`delete_from_element_when` | No | Specifies the condition needed to delete the key from each element of a list whe using `iterate_on`                                                                                             |
+| `iterate_on` | No | A string that specifies the key of the list of objects to iterate over and delete the keys specified in `with_keys`                                                                             
 
 ## Usage
 
@@ -33,10 +43,13 @@
 pipeline:
   source:
     ...
-  ....  
+  ....
   processor:
     - delete_entries:
-        with_keys: ["message"]
+        entries:
+          - with_keys: [ "message" ]
+          - with_keys_regex: [ "^tes.*" ]
+            exclude_from_delete: [ "test" ]
   sink:
 ```
 {% include copy.html %}
@@ -46,13 +59,13 @@
 For example, before you run the `delete_entries` processor, if the `logs_json.log` file contains the following event record:
 
 ```json
-{"message": "hello", "message2": "goodbye"}
+{"message": "hello", "message2": "goodbye", "test": "friends", "test2": "are", "test3": "kind"}
 ```
 
 When you run the `delete_entries` processor, it parses the message into the following output:
 
 ```json
-{"message2": "goodbye"}
+{"message2": "goodbye","test": "friends"}
 ```
 
-> If `message` does not exist in the event, then no action occurs.
+> If `with_keys`, `with_keys_regex`, or `exclude_from_delete` values do not match any event keys, then no action occurs.

_data-prepper/pipelines/configuration/processors/select-entries.md

@@ -16,9 +16,10 @@ Only the selected entries remain in the processed event and while all other entr
 You can configure the `select_entries` processor using the following options.
 
 | Option | Required | Description |
-| :--- | :--- | :--- |
-| `include_keys` | Yes | A list of keys to be selected from an event. |
-| `select_when` | No | A [conditional expression]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/expression-syntax/), such as `/some-key == "test"'`, that will be evaluated to determine whether the processor will be run on the event. If the condition is not met, then the event continues through the pipeline unmodified with all the original fields present. |
+| :--- |:---------| :--- |
+| `include_keys` | No       | A list of keys to be selected from an event. |
+| `include_keys_regex` | No | A regex pattern that matches the keys to be selected from an event. |
+| `select_when` | No       | A [conditional expression]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/expression-syntax/), such as `/some-key == "test"'`, that will be evaluated to determine whether the processor will be run on the event. If the condition is not met, then the event continues through the pipeline unmodified with all the original fields present. |
 
 ## Usage
 
@@ -32,6 +33,7 @@ pipeline:
   processor:
     - select_entries:
         include_keys: [ "key1", "key2" ]
+        include_keys_regex: ["^ran.*"]
         select_when: '/some_key == "test"'
   sink:
 ```
@@ -41,13 +43,22 @@ pipeline:
 For example, when your source contains the following event record:
 
 ```json
-{"message": "hello", "key1" : "value1", "key2" : "value2", "some_key" : "test"}
+{
+  "message": "hello",
+  "key1" : "value1",
+  "key2" : "value2",
+  "some_key" : "test",
+  "random1": "another",
+  "random2" : "set",
+  "random3": "of",
+  "random4": "values"
+}
 ```
 
-The `select_entries` processor includes only `key1` and `key2` in the processed output:
+The `select_entries` processor will output:
 
 ```json
-{"key1": "value1", "key2": "value2"}
+{"key1": "value1", "key2": "value2", "random1": "another", "random2" : "set", "random3": "of", "random4": "values"}
 ```
 
 ### Accessing nested fields