Change in response format for aggregations on date fields #82688
Description
Elasticsearch version (bin/elasticsearch --version):
7.17, 8.0, 8.1
Plugins installed: no plugins
Description of the problem including expected versus actual behavior:
Given the following query:
GET .ent-search-engine-documents-national-parks-demo/_search
{
"aggs": {
"established_at": {
"range": {
"ranges": [
{
"to": "1900-01-01T00:00:00.000Z",
"from": "1800-01-01T00:00:00.000Z",
"key": "1800 to 1900"
},
{
"from": "1900-01-01T00:00:00.000Z",
"to": "2000-12-31T00:00:00.000Z",
"key": "1900 to 2000"
},
{
"from": "2000-01-01T00:00:00.000Z",
"key": "from 2000 "
}
],
"field": "date_established.date"
},
"meta": {
"unnamed_range_names": [],
"respond_with_rfc3339_date": true
}
}
},
"size": 0,
"from": 0,
"timeout": "30000ms"
}
In 7.16.3, the response is:
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 2,
"successful" : 2,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 59,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"established_at" : {
"meta" : {
"unnamed_range_names" : [ ],
"respond_with_rfc3339_date" : true
},
"buckets" : [
{
"key" : "1800 to 1900",
"from" : -5.3646624E12,
"from_as_string" : "1800-01-01T00:00:00.000Z",
"to" : -2.2089888E12,
"to_as_string" : "1900-01-01T00:00:00.000Z",
"doc_count" : 4
},
{
"key" : "1900 to 2000",
"from" : -2.2089888E12,
"from_as_string" : "1900-01-01T00:00:00.000Z",
"to" : 9.782208E11,
"to_as_string" : "2000-12-31T00:00:00.000Z",
"doc_count" : 52
},
{
"key" : "from 2000 ",
"from" : 9.466848E11,
"from_as_string" : "2000-01-01T00:00:00.000Z",
"doc_count" : 4
}
]
}
}
}
In 7.17/8.0/8.1 the response is:
{
"took" : 4,
"timed_out" : false,
"_shards" : {
"total" : 2,
"successful" : 2,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 59,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"established_at" : {
"meta" : {
"unnamed_range_names" : [ ],
"respond_with_rfc3339_date" : true
},
"buckets" : [
{
"key" : "1800 to 1900",
"doc_count" : 4
},
{
"key" : "1900 to 2000",
"doc_count" : 52
},
{
"key" : "from 2000 ",
"doc_count" : 4
}
]
}
}
}
The buckets are missing the from_as_string, to_as_string, from and to fields.
This is a breaking change introduced in 7.17.
Furthermore if we don't provide keys for the buckets, the response is:
GET .ent-search-engine-documents-national-parks-demo/_search
{
"aggs": {
"established_at": {
"range": {
"ranges": [
{
"to": "1900-01-01T00:00:00.000Z",
"from": "1800-01-01T00:00:00.000Z"
},
{
"from": "1900-01-01T00:00:00.000Z",
"to": "2000-12-31T00:00:00.000Z"
},
{
"from": "2000-01-01T00:00:00.000Z"
}
],
"field": "date_established.date"
},
"meta": {
"unnamed_range_names": [],
"respond_with_rfc3339_date": true
}
}
},
"size": 0,
"from": 0,
"timeout": "30000ms"
}
Response:
{
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 2,
"successful" : 2,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 59,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"established_at" : {
"meta" : {
"unnamed_range_names" : [ ],
"respond_with_rfc3339_date" : true
},
"buckets" : [
{
"key" : "*-*",
"doc_count" : 4
},
{
"key" : "*-*",
"doc_count" : 52
},
{
"key" : "*-*",
"doc_count" : 4
}
]
}
}
}
This means we can only rely on the order of the buckets from the response which is not always practical.