prometheus-timestream-adapter

Prometheus-timestream-adapter is a service which receives Prometheus metrics through remote_write, and sends them into AWS Timestream.

Building

go build

Testing

go test

Configuring Prometheus

To configure Prometheus to send samples to this binary, add the following to your prometheus.yml:

remote_write:
  - url: "http://prometheus-timestream-adapter:9201/write?databaseName=prometheus&tableName=prometheus"

⚠️ There is a very early remote_reader version! AWS Timestream has a very powerful query language and there is a Grafana Plugin supporting Timestream as a datasource. However, there is a very basic reader implementation.

remote_read:
  - url: "http://prometheus-timestream-adapter:9201/read?databaseName=prometheus&tableName=prometheus"

Access Prometheus Timestream Database

The Session will attempt to load configuration and credentials from the environment, configuration files, and other credential sources.

AWS Policy

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowReadWriteToTable",
            "Effect": "Allow",
            "Action": [
                "timestream:WriteRecords",
                "timestream:Select"
            ],
            "Resource": "arn:aws:timestream:region:AccoundId:database/DatabaseName/table/TableName"
        },
        {
            "Sid": "AllowDescribeEndpoints",
            "Effect": "Allow",
            "Action": "timestream:DescribeEndpoints",
            "Resource": "*"
        },
        {
            "Sid": "AllowValueRead",
            "Effect": "Allow",
            "Action": "timestream:SelectValues",
            "Resource": "*"
        }
    ]
}

FAQ

What does the warning Measure name exceeds the maximum supported length mean?

The maximum number of characters for an AWS Timestream Dimension name is 256 bytes. Is a metric name is bigger than that it can't be written to AWS Timestream.

Timestream Quotas

Running in ECS

AWS ECS require a healthcheck to run this container, this can be archieved by setting the task definition healthcheck command to:

CMD,/usr/local/bin/prometheus-timestream-adapter,--healthCheck

Similar if running in AWS Fargate with a load balancer, the load balancer would require a HTTP-based health check, this can be performed by sending checks to the /healthCheck path of the container.