Fix false positive date detection with trailing dot

[felixbarny]

Date detection is not supposed to kick in with numeric values:

elasticsearch/server/src/main/java/org/elasticsearch/index/mapper/DynamicFieldsBuilder.java

Lines 83 to 100 in 366fa74

	} else if (parseableAsLong == false && parseableAsDouble == false && context.root().dateDetection()) {
	// We refuse to match pure numbers, which are too likely to be
	// false positives with date formats that include eg.
	// `epoch_millis` or `YYYY`
	for (DateFormatter dateTimeFormatter : context.root().dynamicDateTimeFormatters()) {
	try {
	dateTimeFormatter.parseMillis(text);
	} catch (DateTimeException | ElasticsearchParseException | IllegalArgumentException e) {
	// failure to parse this, continue
	continue;
	}
	return createDynamicDateField(
	context,
	name,
	dateTimeFormatter,
	() -> strategy.newDynamicDateField(context, name, dateTimeFormatter)
	);
	}

The way this is ensured is that string values that can be parsed as a number are not subject to date detection.

However, there's a case where a string can't be parsed as a number but is still considered a date that can be parsed by the default epoch_millis formatter.

That's because the epoch_millis format supports parsing numbers with a trailing dot, such as 12345., which is equivalent to 12345.0 or 12345. This was made to retain compatibility with Joda time.

The issue is that the MILLISECONDS_FORMATTER2, built to support trailing dots, is implemented as extension of MILLISECONDS_FORMATTER1, which already supports an optional fractional. The result is that MILLISECONDS_FORMATTER2 supports a fractional and a trailing dot.

The bug got introduced in this PR: #36914.

A potential alternative to this approach would be to remove epoch_millis from the default formatters used in date_detection:

elasticsearch/server/src/main/java/org/elasticsearch/index/mapper/RootObjectMapper.java

Lines 61 to 66 in d6cc86a

	public static final Explicit<DateFormatter[]> DYNAMIC_DATE_TIME_FORMATTERS = new Explicit<>(
	new DateFormatter[] {
	DateFieldMapper.DEFAULT_DATE_TIME_FORMATTER,
	DateFormatter.forPattern("yyyy/MM/dd HH:mm:ss||yyyy/MM/dd||epoch_millis") },
	false
	);

However, the consequence of this is that it will use a non-default formatter for dynamically detected date fields (strict_date_optional_time instead of strict_date_optional_time||epoch_millis). That means the get mapping API will return "date": { "type": "date", "format": "strict_date_optional_time" } for dynamically mapped dates instead of just "date": { "type": "date" }.

That's why I've opted for a solution that has a lower chance of unintended side effects.

Fixes #116946

[elasticsearchmachine]

Hi @felixbarny, I've created a changelog YAML for you.

[elasticsearchmachine]

Pinging @elastic/es-search-foundations (Team:Search Foundations)

[ldematte]

Change looks good to me, but you might want to have one approval from the search team to ensure this has no unintended side effects.

[elasticsearchmachine]

💚 Backport successful

Status	Branch	Result
✅	8.17
✅	8.x