- 
                Notifications
    You must be signed in to change notification settings 
- Fork 25.6k
Closed
Labels
:Search/SearchSearch-related issues that do not fall into other categoriesSearch-related issues that do not fall into other categoriesTeam:SearchMeta label for search teamMeta label for search team
Description
Elasticsearch version (bin/elasticsearch --version):
Master
JVM version (java -version):
OS version (uname -a if on a Unix-like system):
Darwin Kernel Version 18.7.0
Description of the problem including expected versus actual behavior:
A query using search_after based on an index using a date_nanos with a custom timestamp format throws an exception
Steps to reproduce:
- Create custom mapping
PUT date_nanos_custom_timestamp
{
    "mappings": {
      "properties": {
        "timestamp": {
          "format": "yyyy-MM-dd HH:mm:ss.SSSSSS",
          "type": "date_nanos"
        }
      }
    },
    "settings": {
      "index": {
        "number_of_replicas": "1",
        "number_of_shards": "1"
      }
    }
}
- Add same sample data
PUT date_nanos_custom_timestamp/_doc/1
{
  "timestamp": "2019-10-21 00:30:04.828740"
}
PUT date_nanos_custom_timestamp/_doc/2
{
  "timestamp": "2019-10-21 08:30:04.828733"
}
- Search
POST date_nanos_custom_timestamp/_search
{
  "size": 5,
  "search_after": [
    "1571617804828740000",
    1
  ],
  "sort": [
    {
      "timestamp": {
        "order": "asc",
        "unmapped_type": "boolean"
      }
    },
    {
      "_doc": {
        "order": "asc",
        "unmapped_type": "boolean"
      }
    }
  ]
  }
}
Now you're getting an exception:
{
  "error" : {
    "root_cause" : [
      {
        "type" : "parse_exception",
        "reason" : "failed to parse date field [1571617804828740000] with format [yyyy-MM-dd HH:mm:ss.SSSSSS]: [failed to parse date field [1571617804828740000] with format [yyyy-MM-dd HH:mm:ss.SSSSSS]]"
      }
    ],
    "type" : "search_phase_execution_exception",
    "reason" : "all shards failed",
    "phase" : "query",
    "grouped" : true,
    "failed_shards" : [
      {
        "shard" : 0,
        "index" : "date_nanos_custom_timestamp",
        "node" : "059iRHeFT-ObiKskYJ_29w",
        "reason" : {
          "type" : "parse_exception",
          "reason" : "failed to parse date field [1571617804828740000] with format [yyyy-MM-dd HH:mm:ss.SSSSSS]: [failed to parse date field [1571617804828740000] with format [yyyy-MM-dd HH:mm:ss.SSSSSS]]",
          "caused_by" : {
            "type" : "illegal_argument_exception",
            "reason" : "failed to parse date field [1571617804828740000] with format [yyyy-MM-dd HH:mm:ss.SSSSSS]",
            "caused_by" : {
              "type" : "date_time_parse_exception",
              "reason" : "Text '1571617804828740000' could not be parsed at index 0"
            }
          }
        }
      }
    ]
  },
  "status" : 400
}
This worked a while ago, I've created a functional test in Kibana
But had to skip it, since it caused the 400
This works when providing the sort params as number
 "search_after": [
    1571617804828740000,
    1
  ],
But that's no a solution due to Browser limitation with BigInt numbers
Metadata
Metadata
Assignees
Labels
:Search/SearchSearch-related issues that do not fall into other categoriesSearch-related issues that do not fall into other categoriesTeam:SearchMeta label for search teamMeta label for search team