Completion Suggester for certain search text gives IOException ElasticServer 6.4

[nitineman]

Hi,

I am trying to get some used cases up and running in ElasticSearch 6.4 version on my local machine.

I have created an index with a mapping required for the autocomplete(type as you search capabilities) amongst other used cases.

The issue is with only a few text terms but it works as expected for a few selected text terms("kan") search it gives org.elasticsearch.common.xcontent.XContentParseException and resultantly IOException while trying to fetch the search response from the elastic server. This term is part of the whole text fields which is indexed as the "type": "completion" - something like "Ideal kan" . But it still doesn't explain why it will fail only for this text suggester search but for other's with similar characteristics doesn't.

curl -X PUT "localhost:9200/twitter" -H 'Content-Type: application/json' -d'
{
"settings": {
"analysis": {
"analyzer": {
"autocomplete": {
"tokenizer": "autocomplete",
"filter": [
"lowercase"
]
},
"autocomplete_search": {
"tokenizer": "lowercase"
}
},
"tokenizer": {
"autocomplete": {
"type": "edge_ngram",
"min_gram": 3,
"max_gram": 10,
"token_chars": [
"letter",
"digit",
"symbol"
]
}
}
}
}, "mappings": {
"tweet": {
"properties": {
"MongoDocId": {
"fields": {
"keyword": {
"ignore_above": 256,
"type": "keyword"
}
},
"type": "text"
},
"author": {
"fields": {
"keyword": {
"ignore_above": 256,
"type": "keyword"
}
},
"type": "text"
},
"comment": {
"fields": {
"keyword": {
"ignore_above": 2048,
"type": "keyword"
}
},
"type": "text",
"analyzer" : "english"
},
"createdDateTimeStamp": {
"type": "date"
},
"userId": {
"type": "long"
},
"hashTagList": {
"type": "text",
"fields": {
"hashtags": {
"type": "keyword"
},
"completetag": {
"type": "completion"
}
}
},
"docId": {
"fields": {
"keyword": {
"ignore_above": 256,
"type": "keyword"
}
},
"type": "text"
},
"updated": {
"type": "date"
}
}
}
}
}
'

/* Code where I am invoking the java client code for the suggestion based search with certain text terms. */
public void onApplicationElasticSample(RestHighLevelClient elasticClient) throws IOException {
SearchRequest searchRequest = new SearchRequest("twitter");
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();

	CompletionSuggestionBuilder csb = SuggestBuilders.completionSuggestion("hashTagList.completetag").text("kan");

	SuggestBuilder suggestBuilder = new SuggestBuilder();
	suggestBuilder.addSuggestion("suggest_completer", csb);

	searchSourceBuilder.suggest(suggestBuilder);
	searchRequest.source(searchSourceBuilder);

	SearchResponse searchResponse = elasticClient.search(searchRequest, RequestOptions.DEFAULT);

	System.out.println("searchResponse :" + searchResponse);

	if (searchResponse.getSuggest().getSuggestion("suggest_completer").getEntries().get(0).getOptions().size() > 0)
		System.out.println("searchResponse OTHERS:" + searchResponse.getSuggest().getSuggestion("suggest_completer")
				.getEntries().get(0).getOptions().get(0));
}

/** The Exception trace from the application. */

java.lang.IllegalStateException: Failed to execute CommandLineRunner
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:803) ~[spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:784) ~[spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:338) ~[spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at com.mongo.watcher.MongoApplicationWatcher.main(MongoApplicationWatcher.java:93) [classes/:na]
Caused by: java.io.IOException: Unable to parse response body for Response{requestLine=POST /twitter/_search?typed_keys=true&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true&search_type=query_then_fetch&batched_reduce_size=512 HTTP/1.1, host=http://localhost:9200, response=HTTP/1.1 200 OK}
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1275) ~[elasticsearch-rest-high-level-client-6.4.0.jar:6.4.0]
at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1231) ~[elasticsearch-rest-high-level-client-6.4.0.jar:6.4.0]
at org.elasticsearch.client.RestHighLevelClient.search(RestHighLevelClient.java:730) ~[elasticsearch-rest-high-level-client-6.4.0.jar:6.4.0]
at com.mongo.watcher.MongoApplicationWatcher.onApplicationElasticSample(MongoApplicationWatcher.java:166) [classes/:na]
at com.mongo.watcher.MongoApplicationWatcher.run(MongoApplicationWatcher.java:111) [classes/:na]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:800) ~[spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
... 3 common frames omitted
Caused by: org.elasticsearch.common.xcontent.XContentParseException: [1:1920] [CompletionSuggestionEntryParser] failed to parse field [options]
at org.elasticsearch.common.xcontent.ObjectParser.parseValue(ObjectParser.java:316) ~[elasticsearch-x-content-6.4.0.jar:6.4.0]
at org.elasticsearch.common.xcontent.ObjectParser.parseArray(ObjectParser.java:308) ~[elasticsearch-x-content-6.4.0.jar:6.4.0]
at org.elasticsearch.common.xcontent.ObjectParser.parseSub(ObjectParser.java:329) ~[elasticsearch-x-content-6.4.0.jar:6.4.0]
at org.elasticsearch.common.xcontent.ObjectParser.parse(ObjectParser.java:168) ~[elasticsearch-x-content-6.4.0.jar:6.4.0]
at org.elasticsearch.common.xcontent.ObjectParser.apply(ObjectParser.java:182) ~[elasticsearch-x-content-6.4.0.jar:6.4.0]
at org.elasticsearch.search.suggest.completion.CompletionSuggestion$Entry.fromXContent(CompletionSuggestion.java:257) ~[elasticsearch-6.4.0.jar:6.4.0]
at org.elasticsearch.search.suggest.Suggest$Suggestion.parseEntries(Suggest.java:413) ~[elasticsearch-6.4.0.jar:6.4.0]
at org.elasticsearch.search.suggest.completion.CompletionSuggestion.fromXContent(CompletionSuggestion.java:126) ~[elasticsearch-6.4.0.jar:6.4.0]
at org.elasticsearch.client.RestHighLevelClient.lambda$getDefaultNamedXContents$58(RestHighLevelClient.java:1490) ~[elasticsearch-rest-high-level-client-6.4.0.jar:6.4.0]
at org.elasticsearch.common.xcontent.NamedXContentRegistry.parseNamedObject(NamedXContentRegistry.java:141) ~[elasticsearch-x-content-6.4.0.jar:6.4.0]
at org.elasticsearch.common.xcontent.support.AbstractXContentParser.namedObject(AbstractXContentParser.java:433) ~[elasticsearch-x-content-6.4.0.jar:6.4.0]
at org.elasticsearch.common.xcontent.XContentParserUtils.parseTypedKeysObject(XContentParserUtils.java:153) ~[elasticsearch-6.4.0.jar:6.4.0]
at org.elasticsearch.search.suggest.Suggest$Suggestion.fromXContent(Suggest.java:404) ~[elasticsearch-6.4.0.jar:6.4.0]
at org.elasticsearch.search.suggest.Suggest.fromXContent(Suggest.java:187) ~[elasticsearch-6.4.0.jar:6.4.0]
at org.elasticsearch.action.search.SearchResponse.innerFromXContent(SearchResponse.java:291) ~[elasticsearch-6.4.0.jar:6.4.0]
at org.elasticsearch.action.search.SearchResponse.fromXContent(SearchResponse.java:248) ~[elasticsearch-6.4.0.jar:6.4.0]
at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:1406) ~[elasticsearch-rest-high-level-client-6.4.0.jar:6.4.0]
at org.elasticsearch.client.RestHighLevelClient.lambda$performRequestAndParseEntity$9(RestHighLevelClient.java:1232) ~[elasticsearch-rest-high-level-client-6.4.0.jar:6.4.0]
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1273) ~[elasticsearch-rest-high-level-client-6.4.0.jar:6.4.0]
... 8 common frames omitted
Caused by: org.elasticsearch.common.xcontent.XContentParseException: [1:1920] [CompletionOptionParser] _ignored doesn't support values of type: START_ARRAY
at org.elasticsearch.common.xcontent.ObjectParser$FieldParser.assertSupports(ObjectParser.java:373) ~[elasticsearch-x-content-6.4.0.jar:6.4.0]
at org.elasticsearch.common.xcontent.ObjectParser.parse(ObjectParser.java:167) ~[elasticsearch-x-content-6.4.0.jar:6.4.0]
at org.elasticsearch.common.xcontent.ObjectParser.apply(ObjectParser.java:182) ~[elasticsearch-x-content-6.4.0.jar:6.4.0]
at org.elasticsearch.search.suggest.completion.CompletionSuggestion$Entry$Option.fromXContent(CompletionSuggestion.java:356) ~[elasticsearch-6.4.0.jar:6.4.0]
at org.elasticsearch.search.suggest.completion.CompletionSuggestion$Entry.lambda$static$0(CompletionSuggestion.java:253) ~[elasticsearch-6.4.0.jar:6.4.0]
at org.elasticsearch.common.xcontent.AbstractObjectParser.lambda$declareObjectArray$7(AbstractObjectParser.java:184) ~[elasticsearch-x-content-6.4.0.jar:6.4.0]
at org.elasticsearch.common.xcontent.AbstractObjectParser.lambda$declareFieldArray$13(AbstractObjectParser.java:212) ~[elasticsearch-x-content-6.4.0.jar:6.4.0]
at org.elasticsearch.common.xcontent.AbstractObjectParser.parseArray(AbstractObjectParser.java:230) ~[elasticsearch-x-content-6.4.0.jar:6.4.0]
at org.elasticsearch.common.xcontent.AbstractObjectParser.lambda$declareFieldArray$14(AbstractObjectParser.java:212) ~[elasticsearch-x-content-6.4.0.jar:6.4.0]
at org.elasticsearch.common.xcontent.ObjectParser.lambda$declareField$1(ObjectParser.java:213) ~[elasticsearch-x-content-6.4.0.jar:6.4.0]
at org.elasticsearch.common.xcontent.ObjectParser.parseValue(ObjectParser.java:314) ~[elasticsearch-x-content-6.4.0.jar:6.4.0]
... 26 common frames omitted

[cbuescher]

Hi @nitineman, thanks for opening this issue. You say you are using ES 6.4 as the server, but which High Level Rest client version are you using?
The parsing error that causes the above exception looks like there is a new syntax variation added with the "_ignore" field that we previously didn't have, so this issue might not (only) be related to suggesters but more to compatibility issues with the HL client.

[elasticmachine]

Pinging @elastic/es-search-aggs

[elasticmachine]

Pinging @elastic/es-core-infra

[cbuescher]

I'm not sure yet if this is a bug or a version mismatch that should be expected due to our HL client compatibility requirements, but labeling it as "bug" for further investigation for now.

[nitineman]

The ES server and ES REST High Level Client both are of v6.4.0
Below is the jar Im using in the Spring boot project.
elasticsearch-rest-high-level-client-6.4.0.jar

[jimczi]

This is fixed by #32362. @nitineman you need to upgrade the server and client to 6.4.1 which has the fix.

[nitineman]

@jimczi Thanks your inputs. Upgrading both the ES server and REST High Level Client to 6.4.1 has resolved this issue. Closing the ticket