Skip to content

Commit 3d4c84f

Browse files
authored
ingest: doc: move Dot Expander Processor doc to correct position (#31743)
No changes to the content.
1 parent bc274b2 commit 3d4c84f

File tree

1 file changed

+119
-119
lines changed

1 file changed

+119
-119
lines changed

docs/reference/ingest/ingest-node.asciidoc

Lines changed: 119 additions & 119 deletions
Original file line numberDiff line numberDiff line change
@@ -1049,6 +1049,125 @@ understands this to mean `2016-04-01` as is explained in the <<date-math-index-n
10491049
| `index_name_format` | no | yyyy-MM-dd | The format to be used when printing the parsed date into the index name. An valid Joda pattern is expected here.
10501050
|======
10511051

1052+
[[dot-expand-processor]]
1053+
=== Dot Expander Processor
1054+
1055+
Expands a field with dots into an object field. This processor allows fields
1056+
with dots in the name to be accessible by other processors in the pipeline.
1057+
Otherwise these <<accessing-data-in-pipelines,fields>> can't be accessed by any processor.
1058+
1059+
[[dot-expender-options]]
1060+
.Dot Expand Options
1061+
[options="header"]
1062+
|======
1063+
| Name | Required | Default | Description
1064+
| `field` | yes | - | The field to expand into an object field
1065+
| `path` | no | - | The field that contains the field to expand. Only required if the field to expand is part another object field, because the `field` option can only understand leaf fields.
1066+
|======
1067+
1068+
[source,js]
1069+
--------------------------------------------------
1070+
{
1071+
"dot_expander": {
1072+
"field": "foo.bar"
1073+
}
1074+
}
1075+
--------------------------------------------------
1076+
// NOTCONSOLE
1077+
1078+
For example the dot expand processor would turn this document:
1079+
1080+
[source,js]
1081+
--------------------------------------------------
1082+
{
1083+
"foo.bar" : "value"
1084+
}
1085+
--------------------------------------------------
1086+
// NOTCONSOLE
1087+
1088+
into:
1089+
1090+
[source,js]
1091+
--------------------------------------------------
1092+
{
1093+
"foo" : {
1094+
"bar" : "value"
1095+
}
1096+
}
1097+
--------------------------------------------------
1098+
// NOTCONSOLE
1099+
1100+
If there is already a `bar` field nested under `foo` then
1101+
this processor merges the `foo.bar` field into it. If the field is
1102+
a scalar value then it will turn that field into an array field.
1103+
1104+
For example, the following document:
1105+
1106+
[source,js]
1107+
--------------------------------------------------
1108+
{
1109+
"foo.bar" : "value2",
1110+
"foo" : {
1111+
"bar" : "value1"
1112+
}
1113+
}
1114+
--------------------------------------------------
1115+
// NOTCONSOLE
1116+
1117+
is transformed by the `dot_expander` processor into:
1118+
1119+
[source,js]
1120+
--------------------------------------------------
1121+
{
1122+
"foo" : {
1123+
"bar" : ["value1", "value2"]
1124+
}
1125+
}
1126+
--------------------------------------------------
1127+
// NOTCONSOLE
1128+
1129+
If any field outside of the leaf field conflicts with a pre-existing field of the same name,
1130+
then that field needs to be renamed first.
1131+
1132+
Consider the following document:
1133+
1134+
[source,js]
1135+
--------------------------------------------------
1136+
{
1137+
"foo": "value1",
1138+
"foo.bar": "value2"
1139+
}
1140+
--------------------------------------------------
1141+
// NOTCONSOLE
1142+
1143+
Then the `foo` needs to be renamed first before the `dot_expander`
1144+
processor is applied. So in order for the `foo.bar` field to properly
1145+
be expanded into the `bar` field under the `foo` field the following
1146+
pipeline should be used:
1147+
1148+
[source,js]
1149+
--------------------------------------------------
1150+
{
1151+
"processors" : [
1152+
{
1153+
"rename" : {
1154+
"field" : "foo",
1155+
"target_field" : "foo.bar""
1156+
}
1157+
},
1158+
{
1159+
"dot_expander": {
1160+
"field": "foo.bar"
1161+
}
1162+
}
1163+
]
1164+
}
1165+
--------------------------------------------------
1166+
// NOTCONSOLE
1167+
1168+
The reason for this is that Ingest doesn't know how to automatically cast
1169+
a scalar field to an object field.
1170+
10521171
[[fail-processor]]
10531172
=== Fail Processor
10541173
Raises an exception. This is useful for when
@@ -2058,125 +2177,6 @@ Converts a string to its uppercase equivalent.
20582177
--------------------------------------------------
20592178
// NOTCONSOLE
20602179

2061-
[[dot-expand-processor]]
2062-
=== Dot Expander Processor
2063-
2064-
Expands a field with dots into an object field. This processor allows fields
2065-
with dots in the name to be accessible by other processors in the pipeline.
2066-
Otherwise these <<accessing-data-in-pipelines,fields>> can't be accessed by any processor.
2067-
2068-
[[dot-expender-options]]
2069-
.Dot Expand Options
2070-
[options="header"]
2071-
|======
2072-
| Name | Required | Default | Description
2073-
| `field` | yes | - | The field to expand into an object field
2074-
| `path` | no | - | The field that contains the field to expand. Only required if the field to expand is part another object field, because the `field` option can only understand leaf fields.
2075-
|======
2076-
2077-
[source,js]
2078-
--------------------------------------------------
2079-
{
2080-
"dot_expander": {
2081-
"field": "foo.bar"
2082-
}
2083-
}
2084-
--------------------------------------------------
2085-
// NOTCONSOLE
2086-
2087-
For example the dot expand processor would turn this document:
2088-
2089-
[source,js]
2090-
--------------------------------------------------
2091-
{
2092-
"foo.bar" : "value"
2093-
}
2094-
--------------------------------------------------
2095-
// NOTCONSOLE
2096-
2097-
into:
2098-
2099-
[source,js]
2100-
--------------------------------------------------
2101-
{
2102-
"foo" : {
2103-
"bar" : "value"
2104-
}
2105-
}
2106-
--------------------------------------------------
2107-
// NOTCONSOLE
2108-
2109-
If there is already a `bar` field nested under `foo` then
2110-
this processor merges the `foo.bar` field into it. If the field is
2111-
a scalar value then it will turn that field into an array field.
2112-
2113-
For example, the following document:
2114-
2115-
[source,js]
2116-
--------------------------------------------------
2117-
{
2118-
"foo.bar" : "value2",
2119-
"foo" : {
2120-
"bar" : "value1"
2121-
}
2122-
}
2123-
--------------------------------------------------
2124-
// NOTCONSOLE
2125-
2126-
is transformed by the `dot_expander` processor into:
2127-
2128-
[source,js]
2129-
--------------------------------------------------
2130-
{
2131-
"foo" : {
2132-
"bar" : ["value1", "value2"]
2133-
}
2134-
}
2135-
--------------------------------------------------
2136-
// NOTCONSOLE
2137-
2138-
If any field outside of the leaf field conflicts with a pre-existing field of the same name,
2139-
then that field needs to be renamed first.
2140-
2141-
Consider the following document:
2142-
2143-
[source,js]
2144-
--------------------------------------------------
2145-
{
2146-
"foo": "value1",
2147-
"foo.bar": "value2"
2148-
}
2149-
--------------------------------------------------
2150-
// NOTCONSOLE
2151-
2152-
Then the `foo` needs to be renamed first before the `dot_expander`
2153-
processor is applied. So in order for the `foo.bar` field to properly
2154-
be expanded into the `bar` field under the `foo` field the following
2155-
pipeline should be used:
2156-
2157-
[source,js]
2158-
--------------------------------------------------
2159-
{
2160-
"processors" : [
2161-
{
2162-
"rename" : {
2163-
"field" : "foo",
2164-
"target_field" : "foo.bar""
2165-
}
2166-
},
2167-
{
2168-
"dot_expander": {
2169-
"field": "foo.bar"
2170-
}
2171-
}
2172-
]
2173-
}
2174-
--------------------------------------------------
2175-
// NOTCONSOLE
2176-
2177-
The reason for this is that Ingest doesn't know how to automatically cast
2178-
a scalar field to an object field.
2179-
21802180
[[urldecode-processor]]
21812181
=== URL Decode Processor
21822182
URL-decodes a string

0 commit comments

Comments
 (0)