ES|QL: fix sentinel values leaking from SearchContextStats min/max#142726
ES|QL: fix sentinel values leaking from SearchContextStats min/max#142726salvatore-campagna wants to merge 4 commits intoelastic:mainfrom
Conversation
Filter out Lucene sentinel values (Long.MIN_VALUE/Long.MAX_VALUE) in SearchContextStats.min() and max() to prevent Rounding.prepare() from overflowing on wide index patterns mixing TSDB and non-TSDB indices.
fb733ce to
0e9800f
Compare
|
Pinging @elastic/es-storage-engine (Team:StorageEngine) |
|
@fang-xing-esql FYI. I targeted release versions |
Thank you for taking care of this @salvatore-campagna. If this is related to replacing Fixing it in 9.4 is good, but I'm not quite sure about the urgency of backport, perhaps it is ok to wait for the next branch if the current ones on 9.2 and 9.3 are frozen. |
|
Note: today I investigated a bit more this is and I think this fix is more a workaround. There is a better way to fix this. The root cause is that
As a result, these require different Lucene APIs to read min/max (DocValuesSkipper.globalMinValue vs PointValues.getMinPackedValue). Calling the wrong one doesn't error because it silently returns sentinel values (Long.MIN_VALUE / Long.MAX_VALUE) or null. The
This would require either passing the I am working on another fix to do this properly. |
|
Closing this in favor of: #142752 |
SearchContextStats.min()/max()can return sentinel values (Long.MIN_VALUE/Long.MAX_VALUE) instead of real date/timestamp values. This causesRounding.prepare(min, max)to overflow, failing ES|QL queries that access date fields on wide index patterns mixing TSDB and non-TSDB indices (e.g.apm-*,logs-*,...).Two code paths are affected:
PointValues path: when a date field is mapped but no documents contain date values,
PointValues.getMinPackedValue()returns null andminValuestays at itsLong.MAX_VALUEinitializer. The old comparisonminValue <= min[0]evaluates toLong.MAX_VALUE <= Long.MAX_VALUEwhich is true, so the sentinel leaks as a real value and is potentially used by the rounding logic.DocValuesSkipper path (mixed TSDB): when the first matched index is TSDB,
hasDocValueSkipperis set to true no matter if all segments have a skipper or not. Segment readers from non-TSDB indices have the date field indexed (via PointValues) but no skipper, soDocValuesSkipper.globalMinValue()returnsLong.MIN_VALUE. As a result, a sentinel value leaks as a real value and is potentially used by the rounding logic.The fix adds a
hasMin/hasMaxboolean that tracks whether each code path produces a real value (notLong.MIN_VALUEorLong.MAX_VALUE) before updating the accumulator. In the PointValues path, it is set to true only whengetMinPackedValue()returns non-null. In the DocValuesSkipper path, sentinel values are filtered explicitly.Closes #142725