Skip to content

Setting up an S3 storage

Jump to bottom
L. Le Meur edited this page Mar 20, 2025 · 9 revisions

S3 storage in the LCP server is managed via the official AWS SDK stored on Github. This document is a quick "how-to" extracted from the AWS SDK documentation.

Create a bucket.

Make files publicly readable

source: https://docs.aws.amazon.com/AmazonS3/latest/dev/WebsiteAccessPermissionsReqd.html

  • Select the bucket.
  • Go to Permissions.
  • Add this bucket strategy:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PublicReadGetObject",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::name-of-your-bucket/*"
        }
    ]
}

Obvious: change name-of-your-bucket for the name of your bucket.

Get your credentials

Source: https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/setting-up.html

If you don’t have your credentials yet, you can create them by using the AWS Management Console. We recommend that you use IAM access keys instead of AWS root account access keys. IAM lets you securely control access to AWS services and resources in your AWS account.

  • Open the IAM console.
  • create a user "lcp-server" + programmatic access
  • create a group "s3-client" + strategy "AmazonS3FullAccess"
  • add the user to the group
  • no optional key
  • download the csv -> access key + secret access key
  • store it securely.

ex. https://s3.console.aws.amazon.com/s3/buckets/name-of-your-bucket?region=eu-west-3&tab=objects

Set environment variables

Source: https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html

The AWS Region must be provided in the AWS config file (.aws/config) or as the AWS_REGION environment variable.

The AWS SDK for Go requires credentials (an access key id and a secret access key) to sign requests to AWS. These are used to sign programmatic requests that you make to AWS. You can specify your credentials either in the AWS credentials file (.aws/credentials) or as the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables.

ex. on MacOS and Linux

$ export AWS_REGION=YOUR_REGION
$ export AWS_ACCESS_KEY_ID=YOUR_AKID
$ export AWS_SECRET_ACCESS_KEY=YOUR_SECRET_KEY

warning: make sure these environment variables are set permanently.

Test

Go to https://github.com/aws/aws-sdk-go/ and read the content of doc.go carefully.

Create a Go module called s3test. Copy/paste the sample found in doc.go as a new file named s3test.go and store it in the s3test folder. Copy an epub file (e.g. moby-dick.epub) into the same folder.

Try

$ go run s3test.go -b name-of-your-bucket -k moby-dick.epub -d 10m < moby-dick.epub

-> is now fetchable via something like https://edrlab-lcp-storage.s3.eu-west-3.amazonaws.com/moby-dick.epub (the exact URL is found in the AWS console, object screen).

Configure the LCP server

Read the LCP Server configuration page carefully.

A few S3 parameters are used in the LCP Server codebase but not described in the LCP Server configuration page because they don't seem to be used in practice.

This is the case for the token parameter, described in AWS Temporary Security Credentials in IAM, potentially used if credentials are defined in the configuration file, by a call to NewsStaticCredentials.

This is also the case for the endpoint parameter, described at the bottom of the AWS Go SDK.

And the same for the disable_ssl and path_style parameters, for which some explanations can be found here and here.

Clone this wiki locally