Mountpoint for Amazon S3 is a simple, high-throughput file client for mounting an Amazon S3 bucket as a local file system. With Mountpoint for Amazon S3, your applications can access objects stored in Amazon S3 through file operations like open and read. Mountpoint for Amazon S3 automatically translates these operations into S3 object API calls, giving your applications access to the elastic storage and throughput of Amazon S3 through a file interface.
Mountpoint for Amazon S3 is optimized for read-heavy workloads that need high throughput, and does not implement the full POSIX specification for file systems. For example, Mountpoint for Amazon S3 only supports write operations to new files, and writes must be made to the file sequentially. See SEMANTICS.md for a detailed description of Mountpoint for Amazon S3's behavior and POSIX support and how they could affect your application. For workloads that need full POSIX support, we recommend Amazon FSx for Lustre and its support for linking S3 buckets.
Mountpoint for Amazon S3 is currently an alpha release and should not be used in production. We're tracking its production readiness and future features on the Mountpoint for Amazon S3 public roadmap. We're especially interested in early feedback on features, performance, and compatibility. Please send feedback by opening a GitHub issue.
This release has some notable restrictions:
- The only way to install the alpha release of Mountpoint for Amazon S3 is by compiling from source (see Installation below). This will change in a future release.
- Manual endpoint configuration might be required for some S3 customers (see Configuration and usage below).
- Mountpoint for Amazon S3 does not currently support deleting objects.
- Objects written with Mountpoint for Amazon S3 are always stored in the S3 Standard storage class.
The alpha release of Mountpoint for Amazon S3 is only available by building from source, and only supports Linux.
First, install the necessary dependencies. For RPM-based distributions (Amazon Linux 2 and 2023, Fedora, CentOS, etc):
sudo yum install fuse fuse-devel cmake3 clang git pkg-config
or for apt-based distributions (Debian, Ubuntu, etc):
sudo apt install fuse libfuse-dev cmake clang git pkg-config
Second, install the Rust compiler via rustup:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source "$HOME/.cargo/env"
Now clone this repository and its submodules:
git clone --recurse-submodules https://github.com/awslabs/mountpoint-s3.git
Finally, compile the client:
cd mountpoint-s3
cargo build --release
The final binary will be at target/release/mount-s3.
The following instructions assume you have the binary on your path.
Add it to a location on your path now, or prefix mount-s3 commands with the path to the binary.
To use Mountpoint for Amazon S3, your host needs access to valid AWS credentials. For example, you could create a new IAM user and add its credentials to the ~/.aws/credentials file, or associate your EC2 instance with an IAM role.
Launch Mountpoint for Amazon S3 by running the mount-s3 command.
Replace DOC-EXAMPLE-BUCKET with the name of your bucket,
and specify the directory in which your S3 bucket should be mounted (here, ~/mnt):
mkdir ~/mnt
mount-s3 DOC-EXAMPLE-BUCKET ~/mnt
The client will run in the background by default, and the ~/mnt directory now gives access to the objects in your S3 bucket.
To unmount your S3 bucket and exit Mountpoint for Amazon S3, run the umount command:
umount ~/mnt
Mountpoint for Amazon S3 supports the same configuration options for AWS credentials as the AWS CLI. For example, to use a different profile from your ~/.aws/config configuration file, set the AWS_PROFILE environment variable or pass the --profile argument to mount-s3.
To use Mountpoint for Amazon S3 with S3 Access Points, use the bucket-style access point alias of the access point as the bucket name to mount.
For the alpha release, additional configuration is required to use Mountpoint for Amazon S3 in some circumstances:
- AWS GovCloud, China (Beijing), and China (Ningxia) regions: manually specify the S3 endpoint with the
--endpoint-urlcommand-line argument and the region name with the--regionargument. For example, for the AWS GovCloud (US-West) region, specify--endpoint-url https://s3.us-gov-west-1.amazonaws.com --region us-gov-west-1. See the S3 endpoints documentation for more details. These regions will be supported via only the--regioncommand-line argument in the future (awslabs#4). - FIPS, dualstack, and transfer acceleration endpoints: manually specify the S3 endpoint with the
--endpoint-urlcommand-line argument. See the S3 endpoints documentation for more details. These endpoints will be configurable in the future (awslabs#4).
Mountpoint for Amazon S3 is designed for high-performance access to the Amazon S3 service. While it may be functional against other storage services that use S3-like APIs, we aren't able to provide support for those use cases, and they may inadvertently break when we make changes to better support Amazon S3. We welcome contributions of minor compatibility fixes or performance improvements for these services if the changes can be tested against Amazon S3.
First, build the container image:
cd mountpoint-s3/docker
docker build -t mountpoint-s3 .
Before running the container, you'll need to provide AWS credentials as described above.
We recommend using short-term credentials whenever possible.
For example, if you run your containers on an EC2 instance,
you can associate an IAM role with the instance
and Mountpoint for Amazon S3 will automatically assume that role at startup.
Amazon ECS also supports short-term credentials
by associating an IAM role with your ECS task.
If you need to use long-term credentials, set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables, and pass those variables to Docker with the --env AWS_ACCESS_KEY_ID --env AWS_SECRET_ACCESS_KEY arguments.
To launch the container, run the following command,
replacing DOC-EXAMPLE-BUCKET with the name of your S3 bucket:
docker run -it --cap-add SYS_ADMIN --device /dev/fuse \
mountpoint-s3 /bin/bash -c "./mount-s3 DOC-EXAMPLE-BUCKET /mnt"
Inside the container, your bucket will be mounted at the /mnt path.
If you're using Arch Linux, you can use the unofficial AUR package. Using Yay:
yay -S mountpoint-s3-git
We welcome contributions to Mountpoint for Amazon S3! Please see CONTRIBUTING.md for more information on how to report bugs or submit pull requests. We especially welcome contributions to issues tagged as good first issues to work on.
For potential bug reports, see LOGGING.md for details on how to capture logging data. For performance issues, see BENCHMARKING.md for documentation about performance regression testing.
If you discover a potential security issue in this project we ask that you notify AWS Security via our vulnerability reporting page. Please do not create a public GitHub issue.
This project has adopted the Amazon Open Source Code of Conduct. See CODE_OF_CONDUCT.md for more details.
This project is licensed under the Apache-2.0 License. It builds on a number of other awesome projects with open source licenses, primarily:
- The fuser bindings for FUSE (MIT license)
- The AWS Common Runtime for interacting with S3 (Apache-2.0 license)
See deny.toml for a list of licenses used by our dependencies.