Skip to content

Commit

Permalink
[Improvement] feat(iceberg): Enable setting of additional configurati…
Browse files Browse the repository at this point in the history
…on parameters through environment variables (#6458)

### What changes were proposed in this pull request?

Add the environment variables `GRAVITINO_JDBC_USER`,
`GRAVITINO_JDBC_PASSWORD`, and `GRAVITINO_S3_ENDPOINT` to the
`rewrite_config.py`, so database user and password can be set by the
user in a container environment as well as a custom S3 endpoint.

### Why are the changes needed?

Currently, JDBC user and PW are hardcoded to `iceberg` and cannot be
changed when using the container images. One workaround is including the
values in the JDBC connection string, but this neither elegant nor in
line with how containers commonly handle this issue.

Additionally, it wasn't possible to set the value of
`gravitino.iceberg-rest.s3-endpoint` in a container environment, which
can be useful in a variety of situations when using a provider other
than AWS.

### Does this PR introduce _any_ user-facing change?

Added the three aforementioned two environment variables that can be
used by the user.

### How was this patch tested?

Was tested locally only, since this is a minimal change.
  • Loading branch information
this-user authored Feb 19, 2025
1 parent ca1c3c3 commit e2947e1
Show file tree
Hide file tree
Showing 2 changed files with 8 additions and 0 deletions.
4 changes: 4 additions & 0 deletions dev/docker/iceberg-rest-server/rewrite_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,17 @@
env_map = {
"GRAVITINO_IO_IMPL" : "io-impl",
"GRAVITINO_URI" : "uri",
"GRAVITINO_JDBC_DRIVER": "jdbc-driver",
"GRAVITINO_JDBC_USER": "jdbc-user",
"GRAVITINO_JDBC_PASSWORD": "jdbc-password",
"GRAVITINO_WAREHOUSE" : "warehouse",
"GRAVITINO_CREDENTIAL_PROVIDER_TYPE" : "credential-providers",
"GRAVITINO_CREDENTIAL_PROVIDERS" : "credential-providers",
"GRAVITINO_GCS_CREDENTIAL_FILE_PATH" : "gcs-service-account-file",
"GRAVITINO_GCS_SERVICE_ACCOUNT_FILE" : "gcs-service-account-file",
"GRAVITINO_S3_ACCESS_KEY" : "s3-access-key-id",
"GRAVITINO_S3_SECRET_KEY" : "s3-secret-access-key",
"GRAVITINO_S3_ENDPOINT" : "s3-endpoint",
"GRAVITINO_S3_REGION" : "s3-region",
"GRAVITINO_S3_ROLE_ARN" : "s3-role-arn",
"GRAVITINO_S3_EXTERNAL_ID" : "s3-external-id",
Expand Down
4 changes: 4 additions & 0 deletions docs/iceberg-rest-service.md
Original file line number Diff line number Diff line change
Expand Up @@ -428,11 +428,15 @@ Gravitino Iceberg REST server in docker image could access local storage by defa
|----------------------------------------|-----------------------------------------------------|------------------|
| `GRAVITINO_IO_IMPL` | `gravitino.iceberg-rest.io-impl` | 0.7.0-incubating |
| `GRAVITINO_URI` | `gravitino.iceberg-rest.uri` | 0.7.0-incubating |
| `GRAVITINO_JDBC_DRIVER` | `gravitino.iceberg-rest.jdbc-driver` | 0.9.0-incubating |
| `GRAVITINO_JDBC_USER` | `gravitino.iceberg-rest.jdbc-user` | 0.9.0-incubating |
| `GRAVITINO_JDBC_PASSWORD` | `gravitino.iceberg-rest.jdbc-password` | 0.9.0-incubating |
| `GRAVITINO_WAREHOUSE` | `gravitino.iceberg-rest.warehouse` | 0.7.0-incubating |
| `GRAVITINO_CREDENTIAL_PROVIDERS` | `gravitino.iceberg-rest.credential-providers` | 0.8.0-incubating |
| `GRAVITINO_GCS_SERVICE_ACCOUNT_FILE` | `gravitino.iceberg-rest.gcs-service-account-file` | 0.8.0-incubating |
| `GRAVITINO_S3_ACCESS_KEY` | `gravitino.iceberg-rest.s3-access-key-id` | 0.7.0-incubating |
| `GRAVITINO_S3_SECRET_KEY` | `gravitino.iceberg-rest.s3-secret-access-key` | 0.7.0-incubating |
| `GRAVITINO_S3_ENDPOINT` | `gravitino.iceberg-rest.s3-endpoint` | 0.9.0-incubating |
| `GRAVITINO_S3_REGION` | `gravitino.iceberg-rest.s3-region` | 0.7.0-incubating |
| `GRAVITINO_S3_ROLE_ARN` | `gravitino.iceberg-rest.s3-role-arn` | 0.7.0-incubating |
| `GRAVITINO_S3_EXTERNAL_ID` | `gravitino.iceberg-rest.s3-external-id` | 0.7.0-incubating |
Expand Down

0 comments on commit e2947e1

Please sign in to comment.