Kafka Connect Connector for S3

kafka-connect-storage-cloud is the repository for Confluent's Kafka Connectors designed to be used to copy data from Kafka into Amazon S3.

Kafka Connect Sink Connector for Amazon Simple Storage Service (S3)

Documentation for this connector can be found here.

Blogpost for this connector can be found here.

Development

To build a development version you'll need a recent version of Kafka as well as a set of upstream Confluent projects, which you'll have to build from their appropriate snapshot branch. See the kafka-connect-storage-common FAQ for guidance on this process.

You can build kafka-connect-storage-cloud with Maven using the standard lifecycle phases.

Running Integration Tests

Integration tests are run as part of mvn install; however one needs to first configure the environment variableAWS_CREDENTIALS_PATH to point to a json file path with following structure:

{
    "aws_access_key_id": "<key>",
    "aws_secret_access_key": "<secret>"
}

Contribute

• Source Code: https://github.com/confluentinc/kafka-connect-storage-cloud
• Issue Tracker: https://github.com/confluentinc/kafka-connect-storage-cloud/issues
• Learn how to work with the connector's source code by reading our Development and Contribution guidelines.

License

This project is licensed under the Confluent Community License.