You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It is also possible to create a table backed by files or remote locations via
98
-
`CREATE EXTERNAL TABLE` as shown below. Note that wildcards (e.g. `*`) are also
99
-
supported
98
+
`CREATE EXTERNAL TABLE` as shown below. Note that DataFusion does not support wildcards (e.g. `*`) in file paths; instead, specify the directory path directly to read all compatible files in that directory.
100
99
101
100
For example, to create a table `hits` backed by a local parquet file, use:
102
101
@@ -126,6 +125,32 @@ select count(*) from hits;
126
125
1 row in set. Query took 0.344 seconds.
127
126
```
128
127
128
+
**Why Wildcards Are Not Supported**
129
+
130
+
Although wildcards (e.g., _.parquet or \*\*/_.parquet) may work forlocal filesystemsin some cases, they are not officially supported by DataFusion. This is because wildcards are not universally applicable across all storage backends (e.g., S3, GCS). Instead, DataFusion expects the user to specify the directory path, and it will automatically read all compatible files within that directory.
131
+
132
+
For example, the following usage is not supported:
133
+
134
+
```sql
135
+
CREATE EXTERNAL TABLE test (
136
+
message TEXT,
137
+
day DATE
138
+
)
139
+
STORED AS PARQUET
140
+
LOCATION 'gs://bucket/*.parquet';
141
+
```
142
+
143
+
Instead, you should use:
144
+
145
+
```sql
146
+
CREATE EXTERNAL TABLE test (
147
+
message TEXT,
148
+
day DATE
149
+
)
150
+
STORED AS PARQUET
151
+
LOCATION 'gs://bucket/my_table';
152
+
```
153
+
129
154
# Formats
130
155
131
156
## Parquet
@@ -149,14 +174,6 @@ STORED AS PARQUET
149
174
LOCATION '/mnt/nyctaxi/';
150
175
```
151
176
152
-
Register a single folder parquet datasource by specifying a wildcard for files to read
153
-
154
-
```sql
155
-
CREATE EXTERNAL TABLE taxi
156
-
STORED AS PARQUET
157
-
LOCATION '/mnt/nyctaxi/*.parquet';
158
-
```
159
-
160
177
## CSV
161
178
162
179
DataFusion will infer the CSV schema automatically or you can provide it explicitly.
0 commit comments