Skip to content

Support for s3_url_style='path' Configuration in DuckDB Offline Store #5597

@Marcrb2

Description

@Marcrb2

Is your feature request related to a problem? Please describe.
When using DuckDB as an offline store with S3-compatible storage, authentication can fail if the S3 URL style is not set correctly. DuckDB requires the configuration s3_url_style='path' for certain S3 providers, but this setting is not currently supported via environment variables: https://duckdb.org/docs/stable/core_extensions/httpfs/s3api_legacy_authentication.html#configuration

Describe the solution you'd like
Enable Feast to pass the s3_url_style='path' configuration to DuckDB during initialization. This would allow users to work with S3-compatible storage providers that require path-style URLs.

Describe alternatives you've considered
A temporary workaround involves manually adding the following code line:
self.con.sql("SET s3_url_style='path';")
at line 415 in the file:
AppData\Local\Programs\Python\Python312\Lib\site-packages\ibis\backends\duckdb\__init__.py
However, this is not a sustainable or clean solution, especially for production environments.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions