Skip to content

Commit cd90433

Browse files
jpountzjtibshirani
jpountz
authored and
jtibshirani
committed
Add an include_type_name option to 6.x. (#29453) (#37147)
This adds an `include_type_name` option to the `indices.create`, `indices.get_mapping` and `indices.put_mapping` APIs, which defaults to `true`. When set to `false`, then mappings will be returned directly in the body of the `indices.get_mapping` API, without keying them by the type name, the `indices.create` will expect mappings directly under the `mappings` key, and the `indices.put_mapping` will use `_doc` as a type name and fail if a `type` is provided explicitly. On 5.x indices, get-mapping will fail if the index has multiple mappings, and put-mapping will update or introduce mappings for the `_doc` type instead of updating existing mappings. This oddity is required so that we don't have to introduce a new flag to put-mapping requests to know whether they are actually updating the `_doc` type or performing a typeless call. Relates #15613
1 parent 99eea22 commit cd90433

File tree

22 files changed

+1147
-311
lines changed

22 files changed

+1147
-311
lines changed

‎docs/reference/indices/create-index.asciidoc‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -177,3 +177,28 @@ PUT test?wait_for_active_shards=2
177177

178178
A detailed explanation of `wait_for_active_shards` and its possible values can be found
179179
<<index-wait-for-active-shards,here>>.
180+
181+
[float]
182+
=== Skipping types
183+
184+
Types are being removed from Elasticsearch: in 7.0, the `mappings` element will no
185+
longer take the type name as a top-level key by default. You can already opt in for
186+
this behavior by setting `include_type_name=false` and putting mappings directly under
187+
`mappings` in the index creation call, without specifying a type name.
188+
189+
Here is an example:
190+
191+
[source,js]
192+
--------------------------------------------------
193+
PUT test?include_type_name=false
194+
{
195+
"mappings": {
196+
"properties": {
197+
"foo": {
198+
"type": "keyword"
199+
}
200+
}
201+
}
202+
}
203+
--------------------------------------------------
204+
// CONSOLE

‎docs/reference/indices/get-mapping.asciidoc‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,3 +41,51 @@ GET /_mapping
4141
--------------------------------------------------
4242
// CONSOLE
4343
// TEST[setup:twitter]
44+
45+
[float]
46+
=== Skipping types
47+
48+
Types are being removed from Elasticsearch: in 7.0, the `mappings` element will no
49+
longer return the type name as a top-level key by default. You can already opt in for
50+
this behavior by setting `include_type_name=false` on the request.
51+
52+
NOTE: Such calls will be rejected on indices that have multiple types as it
53+
introduces ambiguity as to which mapping should be returned. Only indices
54+
created by Elasticsearch 5.x may have multiple types.
55+
56+
Here is an example:
57+
58+
[source,js]
59+
--------------------------------------------------
60+
PUT test?include_type_name=false
61+
{
62+
"mappings": {
63+
"properties": {
64+
"foo": {
65+
"type": "keyword"
66+
}
67+
}
68+
}
69+
}
70+
71+
GET test/_mappings?include_type_name=false
72+
--------------------------------------------------
73+
// CONSOLE
74+
75+
which returns
76+
77+
[source,js]
78+
--------------------------------------------------
79+
{
80+
"test": {
81+
"mappings": {
82+
"properties": {
83+
"foo": {
84+
"type": "keyword"
85+
}
86+
}
87+
}
88+
}
89+
}
90+
--------------------------------------------------
91+
// TESTRESPONSE

‎docs/reference/indices/put-mapping.asciidoc‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,3 +114,39 @@ PUT my_index/_mapping/_doc
114114

115115
Each <<mapping-params,mapping parameter>> specifies whether or not its setting
116116
can be updated on an existing field.
117+
118+
[float]
119+
=== Skipping types
120+
121+
Types are being removed from Elasticsearch: in 7.0, the `mappings` element will no
122+
longer take the type name as a top-level key by default. You can already opt in for
123+
this behavior by setting `include_type_name=false` and putting mappings directly under
124+
`mappings` in the index creation call, without specifying a type name.
125+
126+
NOTE: On indices created on Elasticsearch 5.x, such calls will actually
127+
introduce or update mappings for the `_doc` type. It is recommended to avoid
128+
calling the put-mapping API with `include_type_name=false` on 5.x indices.
129+
130+
Here is an example:
131+
132+
[source,js]
133+
-----------------------------------
134+
PUT my_index?include_type_name=false
135+
{
136+
"mappings": {
137+
"properties": {
138+
"name": {
139+
"properties": {
140+
"first": {
141+
"type": "text"
142+
}
143+
}
144+
},
145+
"user_id": {
146+
"type": "keyword"
147+
}
148+
}
149+
}
150+
}
151+
-----------------------------------
152+
// CONSOLE

‎qa/full-cluster-restart/src/test/java/org/elasticsearch/upgrades/FullClusterRestartIT.java‎

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,7 @@
3535
import org.elasticsearch.test.NotEqualMessageBuilder;
3636
import org.elasticsearch.test.rest.ESRestTestCase;
3737
import org.elasticsearch.test.rest.yaml.ObjectPath;
38+
import org.hamcrest.Matchers;
3839
import org.junit.Before;
3940

4041
import java.io.IOException;
@@ -1003,6 +1004,115 @@ public void testSoftDeletes() throws Exception {
10031004
}
10041005
}
10051006

1007+
public void testIncludeTypeNameOnMultiTypesIndices() throws IOException {
1008+
assumeTrue("This test only makes sense if the old cluster runs a version that supports types",
1009+
getOldClusterVersion().before(Version.V_6_0_0));
1010+
if (isRunningAgainstOldCluster()) {
1011+
String mapping = Strings.toString(JsonXContent.contentBuilder()
1012+
.startObject()
1013+
.startObject("mappings")
1014+
.startObject("type1")
1015+
.startObject("properties")
1016+
.startObject("foo")
1017+
.field("type", "keyword")
1018+
.endObject()
1019+
.endObject()
1020+
.endObject()
1021+
.startObject("type2")
1022+
.startObject("properties")
1023+
.startObject("foo")
1024+
.field("type", "keyword")
1025+
.endObject()
1026+
.endObject()
1027+
.endObject()
1028+
.endObject()
1029+
.endObject());
1030+
Request createIndexRequest = new Request("PUT", "/" + index);
1031+
createIndexRequest.setJsonEntity(mapping);
1032+
client().performRequest(createIndexRequest);
1033+
} else {
1034+
// GET mappings
1035+
Request getMappingsRequest = new Request("GET", index + "/_mappings");
1036+
getMappingsRequest.addParameter("include_type_name", "false");
1037+
ResponseException ex = expectThrows(ResponseException.class, () -> client().performRequest(getMappingsRequest));
1038+
assertThat(EntityUtils.toString(ex.getResponse().getEntity()), Matchers.containsString("has multiple mappings"));
1039+
1040+
// PUT mappings
1041+
Request putMappingsRequest = new Request("PUT", index + "/_mappings");
1042+
putMappingsRequest.addParameter("include_type_name", "false");
1043+
String mapping = Strings.toString(JsonXContent.contentBuilder()
1044+
.startObject()
1045+
.startObject("properties")
1046+
.startObject("bar")
1047+
.field("type", "long")
1048+
.endObject()
1049+
.endObject()
1050+
.endObject());
1051+
putMappingsRequest.setJsonEntity(mapping);
1052+
client().performRequest(putMappingsRequest);
1053+
1054+
Request getMappingsRequest2 = new Request("GET", index + "/_mappings");
1055+
Response getMappingsResponse = client().performRequest(getMappingsRequest2);
1056+
String getMappingsResponseAsString = EntityUtils.toString(getMappingsResponse.getEntity());
1057+
// we introduced a _doc mapping
1058+
assertThat(getMappingsResponseAsString, Matchers.containsString("\"_doc\":{"));
1059+
}
1060+
}
1061+
1062+
public void testIncludeTypeNameOnSingleTypeIndices() throws IOException {
1063+
if (isRunningAgainstOldCluster()) {
1064+
String mapping = Strings.toString(JsonXContent.contentBuilder()
1065+
.startObject()
1066+
.startObject("mappings")
1067+
.startObject("my_type")
1068+
.startObject("properties")
1069+
.startObject("foo")
1070+
.field("type", "keyword")
1071+
.endObject()
1072+
.endObject()
1073+
.endObject()
1074+
.endObject()
1075+
.endObject());
1076+
Request request = new Request("PUT", "/" + index);
1077+
request.setJsonEntity(mapping);
1078+
client().performRequest(request);
1079+
} else {
1080+
// PUT mappings
1081+
Request putMappingsRequest = new Request("PUT", index + "/_mappings");
1082+
putMappingsRequest.addParameter("include_type_name", "false");
1083+
String mapping = Strings.toString(JsonXContent.contentBuilder()
1084+
.startObject()
1085+
.startObject("properties")
1086+
.startObject("bar")
1087+
.field("type", "long")
1088+
.endObject()
1089+
.endObject()
1090+
.endObject());
1091+
putMappingsRequest.setJsonEntity(mapping);
1092+
client().performRequest(putMappingsRequest);
1093+
1094+
// GET mappings
1095+
Request getMappingsRequest = new Request("GET", index + "/_mappings");
1096+
getMappingsRequest.addParameter("include_type_name", "false");
1097+
if (getOldClusterVersion().before(Version.V_6_0_0)) {
1098+
ResponseException ex = expectThrows(ResponseException.class, () -> client().performRequest(getMappingsRequest));
1099+
assertThat(EntityUtils.toString(ex.getResponse().getEntity()), Matchers.containsString("has multiple mappings"));
1100+
Request getMappingsRequest2 = new Request("GET", index + "/_mappings");
1101+
Response getMappingsResponse = client().performRequest(getMappingsRequest2);
1102+
String getMappingsResponseAsString = EntityUtils.toString(getMappingsResponse.getEntity());
1103+
// we introduced a _doc mapping
1104+
assertThat(getMappingsResponseAsString, Matchers.containsString("\"_doc\":{"));
1105+
assertThat(getMappingsResponseAsString, Matchers.containsString("\"my_type\":{"));
1106+
} else {
1107+
Response getMappingsResponse = client().performRequest(getMappingsRequest);
1108+
String getMappingsResponseAsString = EntityUtils.toString(getMappingsResponse.getEntity());
1109+
assertThat(getMappingsResponseAsString, Matchers.not(Matchers.containsString("my_type"))); // no type name
1110+
assertThat(getMappingsResponseAsString, Matchers.containsString("\"bar\":{\"type\":\"long\"}")); // new field is here
1111+
assertThat(getMappingsResponseAsString, Matchers.containsString("\"foo\":{\"type\":\"keyword\"}")); // old field is here too
1112+
}
1113+
}
1114+
}
1115+
10061116
private void checkSnapshot(String snapshotName, int count, Version tookOnVersion) throws IOException {
10071117
// Check the snapshot metadata, especially the version
10081118
Request listSnapshotRequest = new Request("GET", "/_snapshot/repo/" + snapshotName);

‎rest-api-spec/src/main/resources/rest-api-spec/api/indices.create.json‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,10 @@
1313
}
1414
},
1515
"params": {
16+
"include_type_name": {
17+
"type" : "boolean",
18+
"description" : "Whether a type should be expected in the body of the mappings."
19+
},
1620
"wait_for_active_shards": {
1721
"type" : "string",
1822
"description" : "Set the number of active shards to wait for before the operation returns."

‎rest-api-spec/src/main/resources/rest-api-spec/api/indices.get_mapping.json‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,10 @@
1616
}
1717
},
1818
"params": {
19+
"include_type_name": {
20+
"type" : "boolean",
21+
"description" : "Whether to add the type name to the response."
22+
},
1923
"ignore_unavailable": {
2024
"type" : "boolean",
2125
"description" : "Whether specified concrete indices should be ignored when unavailable (missing or closed)"

‎rest-api-spec/src/main/resources/rest-api-spec/api/indices.put_mapping.json‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,19 +4,22 @@
44
"methods": ["PUT", "POST"],
55
"url": {
66
"path": "/{index}/{type}/_mapping",
7-
"paths": ["/{index}/{type}/_mapping", "/{index}/_mapping/{type}", "/_mapping/{type}", "/{index}/{type}/_mappings", "/{index}/_mappings/{type}", "/_mappings/{type}"],
7+
"paths": ["/{index}/{type}/_mapping", "/{index}/_mapping/{type}", "/_mapping/{type}", "/{index}/{type}/_mappings", "/{index}/_mappings/{type}", "/_mappings/{type}", "{index}/_mappings", "{index}/_mapping"],
88
"parts": {
99
"index": {
1010
"type" : "list",
1111
"description" : "A comma-separated list of index names the mapping should be added to (supports wildcards); use `_all` or omit to add the mapping on all indices."
1212
},
1313
"type": {
1414
"type" : "string",
15-
"required" : true,
1615
"description" : "The name of the document type"
1716
}
1817
},
1918
"params": {
19+
"include_type_name": {
20+
"type" : "boolean",
21+
"description" : "Whether a type should be expected in the body of the mappings."
22+
},
2023
"timeout": {
2124
"type" : "time",
2225
"description" : "Explicit operation timeout"

0 commit comments

Comments
 (0)