CSV IT spec by idegtiarenko · Pull Request #142585 · elastic/elasticsearch

idegtiarenko · 2026-02-17T09:08:28Z

This change aims to replace CsvTests (that uses query engine with lot of stubs) with CsvIT (build on top of single node InternalTestCluster).

This allows:

reduce amount of test stubs and rely on actual production code when executing tests
reduce amount of assumptions: existing test skips ~1.6k or ~31% of scenarios. The new one supports much wider set of features (including views, subqueries and lookup join) and will likely only need to skip infference related spec (5-10%)
debug wider set of tests from ide
speedup test execution (~1m20s vs ~3m15s): the new test is faster as it only indexes data ones opposed to existing test that has to read data from resource files every time it is queried.

alex-spies

What's the difference between CsvIT and the single-node EsqlSpecIT?

Does the InternalTestCluster run on the same jvm as the test suite and thus we can debug easier without starting Debug Elasticsearch from within IntelliJ, or something like that?

x-pack/plugin/esql/src/internalClusterTest/java/org/elasticsearch/xpack/esql/CsvIT.java

idegtiarenko · 2026-02-18T09:42:41Z

What's the difference between CsvIT and the single-node EsqlSpecIT?

EsqlSpecIT is blackbox, it runs in separate jvm and we query it via rest only. CsvIT runs in in the same jvm, we query transport dirrectly.

alex-spies · 2026-02-18T09:46:08Z

Can confirm these run faster than the regular csv tests, btw (twice as fast!), so for that alone this change is quite nice already.

CsvTests:

CsvIT:

alex-spies

Can't provide a deep review, but agree with the approach here. As mentioned above, I can see that this is a big improvement.

alex-spies · 2026-02-18T09:54:32Z

x-pack/plugin/esql/qa/testFixtures/src/main/resources/k8s-timeseries-count-over-time.csv-spec

 ;

-count_over_time_of_date_nanos_promql
+count_over_time_of_date_nanos_promql-Ignore


Why are we disabling this?

The test is implicitly skipped by relying on a non existing capability promql_date_nanos_support_v0.
I believe it is more obvious to mark it Ignore in order to skip.

luigidellaquila

Thanks @idegtiarenko, LGTM

I left just one minor comment.

I think it could still make sense to keep the old CsvTests for now, just because they are not too expensive in general, compared to the total CI execution time, and because they could give us some flexibility for some specific tests (e.g. they already have some manipulation of the optimization rules in there, and we could extend that to randomize the optimizations in the future)

luigidellaquila · 2026-02-19T15:13:06Z

...ugin/esql/qa/testFixtures/src/main/java/org/elasticsearch/xpack/esql/CsvTestsDataLoader.java

        EMPLOYEES_NOT_REHIRED
    );

+    public static final List<InferenceConfig> INFERENCE_CONFIGS = List.of(


I see these are only used in findInferenceConfigByName(), but not at creation time (eg. line 882). Is it on purpose? Can't we iterate on these in createInferenceEndpoints?

# Conflicts: # x-pack/plugin/esql/qa/server/src/main/java/org/elasticsearch/xpack/esql/qa/rest/EsqlSpecTestCase.java # x-pack/plugin/esql/qa/server/src/main/java/org/elasticsearch/xpack/esql/qa/rest/generative/GenerativeRestTest.java # x-pack/plugin/esql/qa/testFixtures/src/main/java/org/elasticsearch/xpack/esql/CsvAssert.java # x-pack/plugin/esql/qa/testFixtures/src/main/java/org/elasticsearch/xpack/esql/CsvTestUtils.java # x-pack/plugin/esql/qa/testFixtures/src/main/java/org/elasticsearch/xpack/esql/CsvTestsDataLoader.java

elasticsearchmachine · 2026-02-20T15:32:11Z

Pinging @elastic/es-analytical-engine (Team:Analytics)

astefan

Please, consider a consistent and comprehensive javadoc for CsvIT class. We already have people confused about how those csv-spec files are used in multiple unit tests/integration tests and what have you. Look at the example in CsvTests for inspiration. Thank you.

astefan · 2026-02-23T10:07:38Z

x-pack/plugin/esql/src/internalClusterTest/java/org/elasticsearch/xpack/esql/CsvIT.java

+import static org.hamcrest.Matchers.greaterThan;
+import static org.hamcrest.Matchers.hasSize;
+
+public class CsvIT extends ESTestCase {


This class needs a different name and a very good Javadoc to explain the difference between it and CsvTests. This will confuse a lot of us on the minor difference between these two and which one is doing what

astefan · 2026-02-23T10:08:15Z

...ugin/esql/qa/testFixtures/src/main/java/org/elasticsearch/xpack/esql/CsvTestsDataLoader.java

        new ViewConfig("employees_rehired"),
        new ViewConfig("employees_not_rehired")
-    );
+    ).collect(toMap(ViewConfig::name, Function.identity()));;


Double semi-colon

astefan · 2026-02-23T10:09:50Z

x-pack/plugin/esql/src/internalClusterTest/java/org/elasticsearch/xpack/esql/CsvIT.java

+
+    private static void loadEnrichPolicy(EnrichPolicyResolver.LookupRequest request) {
+        for (var name : request.policyNames) {
+            enrich.maybeLoad(CsvTestsDataLoader.ENRICH_POLICIES.get(name));


Here could you adjust for the possibility of .get(name) returning null?

astefan · 2026-02-23T10:10:27Z

x-pack/plugin/esql/src/internalClusterTest/java/org/elasticsearch/xpack/esql/CsvIT.java

+    }
+
+    private static void loadInference(GetInferenceModelAction.Request request) {
+        inference.maybeLoad(INFERENCE_CONFIGS.get(request.getInferenceEntityId()));


Here could you adjust for the possibility of .get(request.....) returning null?

astefan · 2026-02-23T10:11:42Z

x-pack/plugin/esql/qa/testFixtures/src/main/resources/views.csv-spec

 | STATS foo = MAX(num), count = COUNT(num) BY country
 | WHERE country IS NULL
 ;
-warning:Line 1:23 (in view [employees_not_rehired]): evaluation of [is_rehired == false] failed, treating result as null. Only first 20 failures recorded.


Why did you remove the line:column values? Imho, these are an essential part of the UX when writing ESQL queries.

astefan · 2026-02-23T10:13:05Z

x-pack/plugin/esql/src/internalClusterTest/java/org/elasticsearch/xpack/esql/CsvIT.java

+        Stream.of(request.indices()).flatMap(pattern -> {
+            assert pattern.contains("<") == false : "Date-math is not supported in test";
+            if (pattern.contains("*")) {
+                assert pattern.endsWith("*") : "Only suffix patterns are supported in test";


Why this restriction? Can you add a comment in code for this, please?

…cations * upstream/main: (35 commits) Create ARM bulk sqrI8 implementation (elastic#142461) Rework get-snapshots predicates (elastic#143161) Refactor downsampling fetchers and producers (elastic#140357) ESQL: Unmute test and add extra logging to generative test validation (elastic#143168) Fix metadata fields being nullified/loaded by unmapped_fields setting (elastic#143155) Determine remote cluster version (elastic#142494) Populate failure message for aborted clones (elastic#143206) Allow kibana_system role to read and manage logs streams (elastic#143053) Mute org.elasticsearch.xpack.esql.CsvIT test {csv-spec:eval.DocsLength} elastic#143224 Mute org.elasticsearch.xpack.esql.CsvIT test {csv-spec:eval.DocsByteLength} elastic#143223 Mute org.elasticsearch.xpack.esql.CsvIT test {csv-spec:docs.DocsBitLength} elastic#143222 Fix FloatVectorScorerSupplier bulkScore bug (elastic#143211) ESQL: Add data node execution for external sources (elastic#143209) [ESQL] Cleanup commands docs (elastic#143058) [ML]Fix latest transforms disregarding updates when sort and sync fields are non-monotonic (elastic#142856) Mute org.elasticsearch.index.mapper.IpFieldMapperTests testSyntheticSourceInObject elastic#143212 Tests: Fix StoreDirectoryMetricsIT (elastic#143084) ESQL: Add distribution strategy for external sources (elastic#143194) CSV IT spec (elastic#142585) Fix VectorScorerOSQBenchmark.score to read corrections properly (elastic#143137) ...

CSV IT spec

073afe0

idegtiarenko requested review from alex-spies, ivancea, luigidellaquila and swallez February 17, 2026 09:08

idegtiarenko added >test Issues or PRs that are addressing/adding tests Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) :Analytics/ES|QL AKA ESQL v9.4.0 labels Feb 17, 2026

elasticsearchmachine and others added 10 commits February 17, 2026 09:15

[CI] Auto commit changes from spotless

c8a026d

fix testing conversion

d108057

apply mapping overrides

fe115f3

fix several tests

3dc6c45

fix converting for the value assertion

ba290fd

[CI] Auto commit changes from spotless

29c3ad6

ignore test with non-existing capability

c8e740b

lazy load indices

d4cb843

[CI] Auto commit changes from spotless

9a8c1c7

lazy load views

9afd0ca

alex-spies reviewed Feb 18, 2026

View reviewed changes

x-pack/plugin/esql/src/internalClusterTest/java/org/elasticsearch/xpack/esql/CsvIT.java Show resolved Hide resolved

idegtiarenko and others added 2 commits February 18, 2026 10:32

narrow down pattern resolution in test

6c47c5c

[CI] Auto commit changes from spotless

286448f

alex-spies approved these changes Feb 18, 2026

View reviewed changes

alex-spies reviewed Feb 18, 2026

View reviewed changes

idegtiarenko added 2 commits February 18, 2026 12:17

verify warnings

53face7

fix style

5b343b9

astefan self-requested a review February 18, 2026 14:02

support enrich

e2e4643

luigidellaquila approved these changes Feb 19, 2026

View reviewed changes

luigidellaquila mentioned this pull request Feb 20, 2026

Simplify CsvTestsDataLoader #142717

Merged

idegtiarenko and others added 4 commits February 20, 2026 16:07

[CI] Auto commit changes from spotless

81351e1

map enrich policies

567c850

map views

e9b0855

idegtiarenko marked this pull request as ready for review February 20, 2026 15:31

idegtiarenko added 3 commits February 20, 2026 16:42

fix compilation

51ff02d

Merge branch 'main' into csv_it

67c7131

Merge branch 'main' into csv_it

58183b1

astefan reviewed Feb 23, 2026

View reviewed changes

idegtiarenko added 9 commits February 23, 2026 13:27

upd

ac72827

upd

e24c2a0

Merge branch 'main' into csv_it

529b565

Merge branch 'main' into csv_it

c6474d7

add javadoc

016d753

Merge branch 'main' into csv_it

5a799b9

Merge branch 'main' into csv_it

1cd134c

fix merge

434d357

Merge branch 'main' into csv_it

fe22bd4

idegtiarenko merged commit 4afc895 into elastic:main Feb 27, 2026
35 checks passed

idegtiarenko deleted the csv_it branch February 27, 2026 07:42

PeteGillinElastic pushed a commit to PeteGillinElastic/elasticsearch that referenced this pull request Feb 27, 2026

CSV IT spec (elastic#142585)

980b02d

prwhelan mentioned this pull request Feb 27, 2026

[Transform] Clean up internal tests #143246

Merged

kkrik-es mentioned this pull request Mar 2, 2026

[CI] org.elasticsearch.xpack.esql.CsvIT test {csv-spec:k8s-timeseries-irate.Irate_of_* failing #143368

Closed

tballison pushed a commit to tballison/elasticsearch that referenced this pull request Mar 3, 2026

CSV IT spec (elastic#142585)

ebdda76

DiannaHohensee mentioned this pull request Mar 11, 2026

[CI] CsvIT test {csv-spec:approximation.Approximate stats with where on multi-valued data} failing #144066

Closed

BrianRothermich mentioned this pull request Mar 11, 2026

[CI] CsvIT test {csv-spec:approximation.Approximate stats with stats where} failing #144051

Closed

Conversation

idegtiarenko commented Feb 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alex-spies left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

idegtiarenko commented Feb 18, 2026

Uh oh!

alex-spies commented Feb 18, 2026

Uh oh!

alex-spies left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

luigidellaquila left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

elasticsearchmachine commented Feb 20, 2026

Uh oh!

astefan left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

idegtiarenko commented Feb 17, 2026 •

edited

Loading