Showcase support for reading wildcard tables in Elasticsearch & OpenSearch#25098
Conversation
|
If this is supported and syntax is approved by @martint we should definitely document in this connector docs. Probably also in BigQuery.. and maybe OpenSearch connector as well? Although arguably its kinda weird that we support that syntax .. |
...lasticsearch/src/test/java/io/trino/plugin/elasticsearch/BaseElasticsearchConnectorTest.java
Outdated
Show resolved
Hide resolved
...lasticsearch/src/test/java/io/trino/plugin/elasticsearch/BaseElasticsearchConnectorTest.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Can we add another test with different index definition? e.g. long vs text, 2 columns vs 1 column
There was a problem hiding this comment.
I tried both scenarios locally and the results are unpredictable - it basically depends which index is used to retrieve the table schema.
This syntax is thought specifically for tables with the same schema.
There was a problem hiding this comment.
Its worth adding a comment that querying wild card tables work only for the same schema tables.
There was a problem hiding this comment.
No. I see this test more of a "demo" and nothing else.
Bending the functionality as Yuya suggested or speaking about the limitations, is just asking for eventual changes in the future if the functionality slightly changes on the underlying data source.
37c4837 to
42b367d
Compare
...lasticsearch/src/test/java/io/trino/plugin/elasticsearch/BaseElasticsearchConnectorTest.java
Outdated
Show resolved
Hide resolved
...n/trino-opensearch/src/test/java/io/trino/plugin/opensearch/BaseOpenSearchConnectorTest.java
Outdated
Show resolved
Hide resolved
...lasticsearch/src/test/java/io/trino/plugin/elasticsearch/BaseElasticsearchConnectorTest.java
Outdated
Show resolved
Hide resolved
42b367d to
a47d9aa
Compare
a47d9aa to
e28427f
Compare
Description
This contribution documents in the code that the
trino-elasticsearchconnector supportsquerying wildcard tables through the
trino-elasticsearchconnector.Additional context and related issues
Test was inspired from #12428
trino-bigqueryeffort.Querying wildcard tables is a feature supported by ES to allow searching multiple indexes at the same time (a common pattern where people shard their data sets across indexes for retention, etc): https://www.elastic.co/guide/en/elasticsearch/reference/current/search-multiple-indices.html
Release notes
(x) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text: