Make TimeSeriesAggregate TimestampAware#140270
Merged
felixbarny merged 12 commits intoelastic:mainfrom Jan 9, 2026
Merged
Conversation
This simplifies the access to the timestamp field during optimization
felixbarny
added
>non-issue
:StorageEngine/ES|QL
elasticsearchmachine
added
Team:StorageEngine
v9.4.0
external-contributor
Collaborator
|
Pinging @elastic/es-storage-engine (Team:StorageEngine) |
felixbarny
commented
Jan 7, 2026
| 0.05898 | 2024-05-10T00:00:00.000Z | ||
| ; | ||
|
|
||
| Rename timestamp when aggs don't require timestamp |
Member
Author
There was a problem hiding this comment.
Note to reviewer: this now throws an error as the TS|STATS command itself requires the timestamp, even though max and max_over_time don't require the timestamp. I think the new behavior makes more sense and is more predictable.
felixbarny
commented
Jan 7, 2026
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/analysis/Analyzer.java
Outdated
Show resolved
Hide resolved
dnhatn
reviewed
Jan 7, 2026
Member
dnhatn
left a comment
dnhatn
left a comment
There was a problem hiding this comment.
The approach makes sense to me. It looks like some spec tests are failing, for example:
./gradlew ":x-pack:plugin:esql:qa:server:single-node:javaRestTest" --tests "org.elasticsearch.xpack.esql.qa.single_node.EsqlSpecIT" -Dtests.method="test {csv-spec:union_types.ImplicitCastingMultiTypedFieldsStatsByNumeric}"
Can you take a look? Thanks Felix!
...plugin/esql/src/main/java/org/elasticsearch/xpack/esql/plan/logical/TimeSeriesAggregate.java
Outdated
Show resolved
Hide resolved
…ssions This avoids problems in local execution planning that got through EsqlSpecIT
…' into ts-aggregate-timestamp-aware
dnhatn
approved these changes
Jan 8, 2026
Member
dnhatn
left a comment
dnhatn
left a comment
There was a problem hiding this comment.
One comment, but this looks great. Thanks Felix!
...plugin/esql/src/main/java/org/elasticsearch/xpack/esql/plan/logical/TimeSeriesAggregate.java
Outdated
Show resolved
Hide resolved
szybia
added a commit
to szybia/elasticsearch
that referenced
this pull request
Jan 9, 2026
* upstream/main: (76 commits) [Inference API] Get _services skips EIS authorization call if CCM is not configured (elastic#139964) Improve TSDB codec benchmarks with full encoder and compression metrics (elastic#140299) ESQL: Consolidate test `BlockLoaderContext`s (elastic#140403) ESQL: Improve Lookup Join performance with CachedDirectoryReader (elastic#139314) ES|QL: Add more examples for the match operator (elastic#139815) ESQL: Add timezone to add and sub operators, and ConfigurationAware planning support (elastic#140101) ESQL: Updated ToIp tests and generated documentation for map parameters (elastic#139994) Disable _delete_by_query and _update_by_query for CCS/stateful (elastic#140301) Remove unused method ElasticInferenceService.translateToChunkedResults (elastic#140442) logging hot threads on large queue of the management threadpool (elastic#140251) Search functions docs cleanup (elastic#140435) Unmute 350_point_in_time/point-in-time with index filter (elastic#140443) Remove unused methods (elastic#140222) Add CPS and `project_routing` support for `_mvt` (elastic#140053) Streamline `ShardDeleteResults` collection (elastic#140363) Fix Docker build to use --load for single-platform images (elastic#140402) Parametrize + test VectorScorerOSQBenchmark (elastic#140354) `RecyclerBytesStreamOutput` using absolute offsets (elastic#140303) Define bulk float native methods for vector scoring (elastic#139885) Make `TimeSeriesAggregate` `TimestampAware` (elastic#140270) ...
jimczi
pushed a commit
to jimczi/elasticsearch
that referenced
this pull request
Jan 12, 2026
elasticsearchmachine
pushed a commit
that referenced
this pull request
Jan 13, 2026
#140526) * ESQL: introduce support for mapping-unavailable fields (#140463) This introduces support for mapping-unavailable fields (present and not mapped or just missing). The behaviour is controlled through a new SET setting unmapped_fields, which can take the values "FAIL", "NULLIFY", "LOAD". An optional field behaves just like a "normal", mapped field, with regards to how it flows through the commands chain: it can be simply used in the commands, as if present in the source, but can no longer be referenced once dropped - explicitly, with DROP, or not selected by a KEEP, or RENAME that doesn't reference it -, or past a STATS reduction. However, unlike a mapped field, if it's not reference at all, it won't show up in the output of a simple FROM index. Currently, the schema difference between nullified fields and the loaded ones is in the type: nullified ones are of data type NULL, while the loaded ones are KEYWORD. The implementation difference w.r.t. logical plan building is that the nullified fields are created as null value aliasing on top of the data source, while the loaded one are pushed as extractors into the source (this leverages the INSIST work). The partially mapped fields are also covered: when the setting is "load", these fields will be extracted from those indices that have the field, but isn't mapped. In case there's a conflict between the loaded KEYWORD field and the mapped type in the fields that have this field mapped, an explicit conversion is needed, just like with union types. Related: #138888 (cherry picked from commit ff745c0) # Conflicts: # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/analysis/Analyzer.java # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/analysis/AnalyzerContext.java # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/plan/QuerySettings.java # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/session/EsqlSession.java # x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/optimizer/promql/PromqlLogicalPlanOptimizerTests.java # x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/parser/SetParserTests.java * Revert transition EsqlProject->Project That refactoring was not backported to 9.3. * Spotless * Fix compilation error The analyzer code was adapted for TimeSeriesAggregate becoming TimestampAware in #140270, but that was not backported to 9.3. Adapt to the old version of the code. * Fix tests --------- Co-authored-by: Gal Lalouche <gal.lalouche@elastic.co>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This simplifies the access to the timestamp field during optimization.
Came up in this PR review: #140135 (comment)