Introduce 64-bit unsigned long field type

docs/reference/mapping/types.asciidoc

@@ -9,7 +9,7 @@ document:
 === Core data types
 
 string::         <<text,`text`>>, <<keyword,`keyword`>> and <<wildcard,`wildcard`>>
-<<number>>::     `long`, `integer`, `short`, `byte`, `double`, `float`, `half_float`, `scaled_float`
+<<number>>::     `long`, `integer`, `short`, `byte`, `double`, `float`, `half_float`, `scaled_float`, <<unsigned-long,`unsigned_long`>>
 <<date>>::       `date`
 <<date_nanos>>:: `date_nanos`
 <<boolean>>::    `boolean`
@@ -137,3 +137,5 @@ include::types/shape.asciidoc[]
 include::types/constant-keyword.asciidoc[]
 
 include::types/wildcard.asciidoc[]
+
+include::types/unsigned_long.asciidoc[]

docs/reference/mapping/types/numeric.asciidoc

@@ -15,6 +15,7 @@ The following numeric types are supported:
 `float`::        A single-precision 32-bit IEEE 754 floating point number, restricted to finite values.
 `half_float`::   A half-precision 16-bit IEEE 754 floating point number, restricted to finite values.
 `scaled_float`:: A floating point number that is backed by a `long`, scaled by a fixed `double` scaling factor.
+`unsigned_long`:: An <<unsigned-long,`unsigned 64-bit integer`>> with a minimum value of 0 and a maximum value of +2^64^-1+.
 
 Below is an example of configuring a mapping with numeric fields:
 
@@ -115,7 +116,7 @@ The following parameters are accepted by numeric types:
 <<coerce,`coerce`>>::
 
     Try to convert strings to numbers and truncate fractions for integers.
-    Accepts `true` (default) and `false`.
+    Accepts `true` (default) and `false`. Not applicable for `unsigned_long`.
 
 <<doc-values,`doc_values`>>::
 

docs/reference/mapping/types/unsigned_long.asciidoc

@@ -0,0 +1,156 @@
+[role="xpack"]
+[testenv="basic"]
+
+[[unsigned-long]]
+=== Unsigned long data type
+++++
+<titleabbrev>Unsigned long</titleabbrev>
+++++
+
+Unsigned long is a numeric field type that represents an unsigned 64-bit
+integer with a minimum value of 0 and a maximum value of +2^64^-1+
+(from 0 to 18446744073709551615 inclusive).
+
+[source,console]
+--------------------------------------------------
+PUT my_index
+{
+  "mappings": {
+    "properties": {
+      "my_counter": {
+        "type": "unsigned_long"
+      }
+    }
+  }
+}
+--------------------------------------------------
+
+Unsigned long can be indexed in a numeric or string form,
+representing integer values in the range [0, 18446744073709551615].
+They can't have a decimal part.
+
+[source,console]
+--------------------------------
+POST /my_index/_bulk?refresh
+{"index":{"_id":1}}
+{"my_counter": 0}
+{"index":{"_id":2}}
+{"my_counter": 9223372036854775808}
+{"index":{"_id":3}}
+{"my_counter": 18446744073709551614}
+{"index":{"_id":4}}
+{"my_counter": 18446744073709551615}
+--------------------------------
+//TEST[continued]
+
+Term queries accept any numbers in a numeric or string form.
+
+[source,console]
+--------------------------------
+GET /my_index/_search
+{
+    "query": {
+        "term" : {
+            "my_counter" : 18446744073709551615
+        }
+    }
+}
+--------------------------------
+//TEST[continued]
+
+Range query terms can contain values with decimal parts.
+In this case {es} converts them to integer values:
+`gte` and `gt` terms are converted to the nearest integer up inclusive,
+and `lt` and `lte` ranges are converted to the nearest integer down inclusive.
+
+It is recommended to pass ranges as strings to ensure they are parsed
+without any loss of precision.
+
+[source,console]
+--------------------------------
+GET /my_index/_search
+{
+    "query": {
+        "range" : {
+            "my_counter" : {
+                "gte" : "9223372036854775808.5",
+                "lte" : "18446744073709551615"
+            }
+        }
+    }
+}
+--------------------------------
+//TEST[continued]
+
+
+For queries with sort on an `unsigned_long` field,
+for a particular document {es} returns a sort value of the type `Long`
+if the value of this document is within the range of long values,
+or of the type `BigIntger` if the value exceeds this range.
+
+WARNING: Not all {es} clients can properly handle big integer values.
+
+[source,console]
+--------------------------------
+GET /my_index/_search
+{
+    "query": {
+        "match_all" : {}
+    },
+    "sort" : {"my_counter" : "desc"}
+}
+--------------------------------
+//TEST[continued]
+
+Similarly to sort values, script values of an `unsigned_long` field
+produce `BigInteger` or `Long` values. The same values: `BigInteger` or
+`Long` are returned as keys for `terms` aggregation.
+
+==== Queries with mixed numeric types
+
+Search queries across several numeric types one of which `unsigned_long` are supported,
+except queries with sort. Thus, a sort query across two indexes where the same field
+is `unsigned_long` in one index, and `long` in another, doesn't produce correct results
+and must be avoided. If there is a need for a such kind of sorting, script based
+sorting can be used instead.
+Aggregations across several numeric types one of which `unsigned_long` are supported,
+except a terms aggregation.
+
+
+[[unsigned-long-params]]
+==== Parameters for unsigned long fields
+
+The following parameters are accepted:
+
+[horizontal]
+
+<<doc-values,`doc_values`>>::
+
+    Should the field be stored on disk in a column-stride fashion, so that it
+    can later be used for sorting, aggregations, or scripting? Accepts `true`
+    (default) or `false`.
+
+<<ignore-malformed,`ignore_malformed`>>::
+
+    If `true`, malformed numbers are ignored. If `false` (default), malformed
+    numbers throw an exception and reject the whole document.
+
+<<mapping-index,`index`>>::
+
+    Should the field be searchable? Accepts `true` (default) and `false`.
+
+<<null-value,`null_value`>>::
+
+    Accepts a numeric value of the same `type` as the field which is
+    substituted for any explicit `null` values.  Defaults to `null`, which
+    means the field is treated as missing.
+
+<<mapping-store,`store`>>::
+
+    Whether the field value should be stored and retrievable separately from
+    the <<mapping-source-field,`_source`>> field. Accepts `true` or `false`
+    (default).
+
+<<mapping-field-meta,`meta`>>::
+
+    Metadata about the field.

modules/lang-painless/src/main/java/org/elasticsearch/painless/Def.java

@@ -29,6 +29,7 @@
 import java.lang.invoke.MethodHandle;
 import java.lang.invoke.MethodHandles;
 import java.lang.invoke.MethodType;
+import java.math.BigInteger;
 import java.time.ZonedDateTime;
 import java.util.BitSet;
 import java.util.Collections;
@@ -734,6 +735,8 @@ public static double defTodoubleImplicit(final Object value) {
             return (float)value;
         } else if (value instanceof Double) {
             return (double)value;
+        } else if (value instanceof BigInteger) {
+            return ((BigInteger)value).doubleValue();
         } else {
             throw new ClassCastException("cannot implicitly cast " +
                     "def [" + PainlessLookupUtility.typeToUnboxedType(value.getClass()).getCanonicalName() + "] to " +
@@ -866,7 +869,8 @@ public static double defTodoubleExplicit(final Object value) {
                 value instanceof Integer ||
                 value instanceof Long    ||
                 value instanceof Float   ||
-                value instanceof Double
+                value instanceof Double  ||
+                value instanceof BigInteger
         ) {
             return ((Number)value).doubleValue();
         } else {
@@ -1004,7 +1008,9 @@ public static Double defToDoubleImplicit(final Object value) {
         } else if (value instanceof Float) {
             return (double)(float)value;
         } else if (value instanceof Double) {
-            return (Double)value;
+            return (Double) value;
+        } else if (value instanceof BigInteger) {
+            return ((BigInteger)value).doubleValue();
         } else {
             throw new ClassCastException("cannot implicitly cast " +
                     "def [" + PainlessLookupUtility.typeToUnboxedType(value.getClass()).getCanonicalName() + "] to " +
@@ -1151,7 +1157,8 @@ public static Double defToDoubleExplicit(final Object value) {
                 value instanceof Integer ||
                 value instanceof Long    ||
                 value instanceof Float   ||
-                value instanceof Double
+                value instanceof Double  ||
+                value instanceof BigInteger
         ) {
             return ((Number)value).doubleValue();
         } else {

modules/lang-painless/src/test/java/org/elasticsearch/painless/DefCastTests.java

@@ -166,6 +166,7 @@ public void testdefTodoubleImplicit() {
         assertEquals((double)0, exec("def d = Long.valueOf(0); double b = d; b"));
         assertEquals((double)0, exec("def d = Float.valueOf(0); double b = d; b"));
         assertEquals((double)0, exec("def d = Double.valueOf(0); double b = d; b"));
+        assertEquals((double)0, exec("def d = BigInteger.valueOf(0); double b = d; b"));
         expectScriptThrows(ClassCastException.class, () -> exec("def d = new ArrayList(); double b = d;"));
     }
 
@@ -485,7 +486,7 @@ public void testdefToFloatImplicit() {
         expectScriptThrows(ClassCastException.class, () -> exec("def d = Double.valueOf(0); Float b = d;"));
         expectScriptThrows(ClassCastException.class, () -> exec("def d = new ArrayList(); Float b = d;"));
     }
-    
+
     public void testdefToDoubleImplicit() {
         expectScriptThrows(ClassCastException.class, () -> exec("def d = 'string'; Double b = d;"));
         expectScriptThrows(ClassCastException.class, () -> exec("def d = true; Double b = d;"));

server/src/main/java/org/elasticsearch/common/io/stream/StreamInput.java

@@ -51,6 +51,7 @@
 import java.io.FilterInputStream;
 import java.io.IOException;
 import java.io.InputStream;
+import java.math.BigInteger;
 import java.nio.file.AccessDeniedException;
 import java.nio.file.AtomicMoveNotSupportedException;
 import java.nio.file.DirectoryNotEmptyException;
@@ -348,6 +349,11 @@ public Long readOptionalLong() throws IOException {
         return null;
     }
 
+    public BigInteger readBigInteger() throws IOException {
+        return new BigInteger(readString());
+    }
+
+
     @Nullable
     public Text readOptionalText() throws IOException {
         int length = readInt();
@@ -760,6 +766,8 @@ public Object readGenericValue() throws IOException {
                 return readCollection(StreamInput::readGenericValue, LinkedHashSet::new, Collections.emptySet());
             case 25:
                 return readCollection(StreamInput::readGenericValue, HashSet::new, Collections.emptySet());
+            case 26:
+                return readBigInteger();
             default:
                 throw new IOException("Can't read unknown type [" + type + "]");
         }

server/src/main/java/org/elasticsearch/common/io/stream/StreamOutput.java

@@ -49,6 +49,7 @@
 import java.io.FileNotFoundException;
 import java.io.IOException;
 import java.io.OutputStream;
+import java.math.BigInteger;
 import java.nio.file.AccessDeniedException;
 import java.nio.file.AtomicMoveNotSupportedException;
 import java.nio.file.DirectoryNotEmptyException;
@@ -826,7 +827,14 @@ public final void writeOptionalInstant(@Nullable Instant instant) throws IOExcep
                             o.writeByte((byte) 25);
                         }
                         o.writeCollection((Set<?>) v, StreamOutput::writeGenericValue);
-                    }
+                    }),
+            entry(
+                // TODO: improve serialization of BigInteger
+                BigInteger.class,
+                (o, v) -> {
+                    o.writeByte((byte) 26);
+                    o.writeString(v.toString());
+                }
             ));
 
     private static Class<?> getGenericType(Object value) {

server/src/main/java/org/elasticsearch/common/lucene/Lucene.java

@@ -95,6 +95,7 @@
 import org.elasticsearch.index.fielddata.IndexFieldData;
 
 import java.io.IOException;
+import java.math.BigInteger;
 import java.text.ParseException;
 import java.util.ArrayList;
 import java.util.Arrays;
@@ -366,6 +367,8 @@ public static FieldDoc readFieldDoc(StreamInput in) throws IOException {
                 cFields[j] = in.readBoolean();
             } else if (type == 9) {
                 cFields[j] = in.readBytesRef();
+            } else if (type == 10) {
+                cFields[j] = new BigInteger(in.readString());
             } else {
                 throw new IOException("Can't match type [" + type + "]");
             }
@@ -510,6 +513,10 @@ public static void writeSortValue(StreamOutput out, Object field) throws IOExcep
             } else if (type == BytesRef.class) {
                 out.writeByte((byte) 9);
                 out.writeBytesRef((BytesRef) field);
+            } else if (type == BigInteger.class) {
+                //TODO: improve serialization of BigInteger
+                out.writeByte((byte) 10);
+                out.writeString(field.toString());
             } else {
                 throw new IOException("Can't handle sort field value of type [" + type + "]");
             }