-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Description
Is your feature request related to a problem? Please describe.
When using DuckDB as an offline store with S3-compatible storage, authentication can fail if the S3 URL style is not set correctly. DuckDB requires the configuration s3_url_style='path'
for certain S3 providers, but this setting is not currently supported via environment variables: https://duckdb.org/docs/stable/core_extensions/httpfs/s3api_legacy_authentication.html#configuration
Describe the solution you'd like
Enable Feast to pass the s3_url_style='path'
configuration to DuckDB during initialization. This would allow users to work with S3-compatible storage providers that require path-style URLs.
Describe alternatives you've considered
A temporary workaround involves manually adding the following code line:
self.con.sql("SET s3_url_style='path';")
at line 415 in the file:
AppData\Local\Programs\Python\Python312\Lib\site-packages\ibis\backends\duckdb\__init__.py
However, this is not a sustainable or clean solution, especially for production environments.