(feat) Add example for rustfs integration by MonkeyCanCode · Pull Request #3482 · apache/polaris

MonkeyCanCode · 2026-01-20T00:41:41Z

Per discussion in https://lists.apache.org/thread/8o31ly7cd8ov70opjbtg630qlhrfl5yh, we would want to add support for RustFS integration. I follow the same layout as MinIO example for easy readability. If we do want to move forward with RustFS, I can then add more advance features such as assume role integration.

Checklist

🛡️ Don't disclose security issues! (contact security@apache.org)
🔗 Clearly explained why the changes are needed, or linked related issues: Fixes #
🧪 Added/updated tests with good coverage, or manually tested (and explained how)
💡 Added comments for complex logic
🧾 Updated CHANGELOG.md (if needed)
📚 Updated documentation in site/content/in-dev/unreleased (if needed)

getting-started/rustfs/docker-compose.yml

snazy · 2026-01-20T11:12:54Z

getting-started/rustfs/docker-compose.yml

+      start_period: 10s
+
+  setup_bucket:
+    image: quay.io/minio/mc:latest


Wonder why mc is being used here. Doesn't Rustfs provide a tool for this as well?

So rustfs itself does have a lib for cli (https://github.com/rustfs/cli), however, there is no published image with only the tool (https://hub.docker.com/u/rustfs). Also, rustfs cli repo is also using mc as the official rustfs-init for creating bucket (https://github.com/rustfs/cli/blob/main/docker/docker-compose.yml). However, if we preferred (as mc and minio server won't get any new update), we can build our own client image with rustfs cli (rc in this case). What do you think?

Nah - I'm okay leaving mc for now. Once they publish their rc as a container image, we can always switch to that one. This is rather a "demonstration".
Might be worth to add a comment to the compose file that mc is from Minio and won't receive any updates and rc should be used in production setups.

Sounds good. Will add a note around this.

aws CLI should work too, I assume, but I'm fine with mc. I also saw it in docs related to RustFS.

That is correct with couple small syntax change where mc and rc are really for interacting with the s3-compatible storage while aws can do a lot more. Other than that, we will just need to set the endpoint-url for aws cli to localhost:9000 then it should all works. But what is more interesting is how come there is no official image for rc and there are not much activities on that repo as well...even the rc repo is using mc for setting up buckets instead of using rc itself (but I did tested rc while I was working and this and it is simply drop-in replacement with change tool from mc to rc). While searching for this, I noticed there is a similar PR from Apache Iceberg for moving away from MinIO and replace with RustFS but they also ended up using mc for s3 interaction (apache/iceberg#14928). Assuming rc will be more mature over time, it may be worth to keep mc for now? Or we can do docker-compose build or switch to aws cli is that is more preferred and reuse-able.

getting-started/rustfs/README.md

snazy · 2026-01-20T16:33:53Z

getting-started/rustfs/docker-compose.yml

+      start_period: 10s
+
+  setup_bucket:
+    image: quay.io/minio/mc:latest


Nah - I'm okay leaving mc for now. Once they publish their rc as a container image, we can always switch to that one. This is rather a "demonstration".
Might be worth to add a comment to the compose file that mc is from Minio and won't receive any updates and rc should be used in production setups.

dimas-b

LGTM assuming current comment threads are resolved :) Thanks, @MonkeyCanCode !

MonkeyCanCode · 2026-01-20T19:43:59Z

LGTM assuming current comment threads are resolved :) Thanks, @MonkeyCanCode !

Thanks for the quick review @dimas-b and @snazy. I had addressed all of the comments. i would like to have this up for couple more days so more people can review. I do see RustFS being a viable replacement for MinIO. However, there are couple concerns on my end as well as called out by community from the mailing list:

Will RustFS be the next "MinIO" in couple of months/years?
There are a bit out of documents for RustFS for certain areas (such as STS...but it doesn't appear they support it but would need more testing for validation if we want to move forward with it)

dimas-b · 2026-01-20T20:17:47Z

There are a bit out of documents for RustFS for certain areas (such as STS...but it doesn't appear they support it [...]

This guide uses vended-credentials, so I assume RustFS supports STS... otherwise how does the example work?

MonkeyCanCode · 2026-01-20T20:50:17Z

There are a bit out of documents for RustFS for certain areas (such as STS...but it doesn't appear they support it [...]

This guide uses vended-credentials, so I assume RustFS supports STS... otherwise how does the example work?

Yes, this is fixed recently (rustfs/rustfs#1337) but no where I can find the doc around this when I was using older image a while back and found this issue and switched to the latest patched image as part of this PR.

flyrain · 2026-01-20T23:58:21Z

Looks like RustFS is actively maintained, a lot of commits in last a few weeks https://github.com/rustfs/rustfs/commits/main/?after=ae50760fcc5703a1e4e1c05ab656fc0fff366f34+104. It does seem a good candidate of MinIO alternatives.

Will RustFS be the next "MinIO" in couple of months/years?

I assume it's really hard to answer it. This blog provides a hint on their vision, https://rustfs.dev/hello-world-rustfs-and-our-journey-to-democratize-storage/, which seems promising on OSS.

flyrain

Thanks @MonkeyCanCode !

dimas-b

LGTM 👍 Thanks, @MonkeyCanCode !

* Add example for rustfs * Make it podman compatible and used fixed version for images * Add disclaimer for mc and set polaris tag to be latest

* (feat) Add example for rustfs integration (apache#3482) * Add example for rustfs * Make it podman compatible and used fixed version for images * Add disclaimer for mc and set polaris tag to be latest * Avoid latest image tage for all other images except polaris (apache#3497) * Use archive.apache.org for previous releases. (apache#3474) Only the last release should be on dist.apache.org. * Site: Update Slack invite (apache#3487) * Updated cli doc with new options introduced in catalog create (apache#3502) * NoSQL: Retain only the most recent state in metastore maintenance (apache#3483) * Extract root Principal setup code into a common util class (apache#3446) * Add `AuthBootstrapUtil` to contain the canonical setup code for the "root" Polaris Principal and default roles. * This class is not meant for reuse outside the Polaris codebase. * Use types config for `POLARIS_TASK_TIMEOUT_MILLIS` (apache#3477) Add a typed `FeatureConfiguration` with the same config name and default value. * Use typed config for `ALLOW_WILDCARD_LOCATION` (apache#3479) Add a typed `FeatureConfiguration` with the same config name and default value. * Last merged commit bf25373 --------- Co-authored-by: Yong Zheng <yongzheng0809@gmail.com> Co-authored-by: JB Onofré <jbonofre@apache.org> Co-authored-by: Yufei Gu <yufei@apache.org> Co-authored-by: Dmitri Bourlatchkov <dmitri.bourlatchkov@gmail.com>

Add example for rustfs

39611d4

github-project-automation bot added this to Basic Kanban Board Jan 20, 2026

github-project-automation bot moved this to PRs In Progress in Basic Kanban Board Jan 20, 2026

MonkeyCanCode requested review from dimas-b, flyrain, jbonofre and snazy January 20, 2026 00:42

snazy reviewed Jan 20, 2026

View reviewed changes

Make it podman compatible and used fixed version for images

1c72d9a

MonkeyCanCode requested a review from snazy January 20, 2026 14:18

snazy reviewed Jan 20, 2026

View reviewed changes

dimas-b reviewed Jan 20, 2026

View reviewed changes

Add disclaimer for mc and set polaris tag to be latest

82ec0bd

flyrain approved these changes Jan 20, 2026

View reviewed changes

github-project-automation bot moved this from PRs In Progress to Ready to merge in Basic Kanban Board Jan 20, 2026

MonkeyCanCode requested review from dimas-b and snazy January 21, 2026 03:46

dimas-b approved these changes Jan 21, 2026

View reviewed changes

MonkeyCanCode merged commit a88d2a1 into apache:main Jan 21, 2026
15 checks passed

github-project-automation bot moved this from Ready to merge to Done in Basic Kanban Board Jan 21, 2026

Comments

Conversation

MonkeyCanCode commented Jan 20, 2026

Checklist

Uh oh!

Uh oh!

snazy Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

MonkeyCanCode Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

snazy Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

MonkeyCanCode Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

dimas-b Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

MonkeyCanCode Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

snazy Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

dimas-b left a comment

Choose a reason for hiding this comment

Uh oh!

MonkeyCanCode commented Jan 20, 2026

Uh oh!

dimas-b commented Jan 20, 2026

Uh oh!

MonkeyCanCode commented Jan 20, 2026

Uh oh!

flyrain commented Jan 20, 2026

Uh oh!

flyrain left a comment

Choose a reason for hiding this comment

Uh oh!

dimas-b left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants