Kafka Connect Transformation (SMT) to encrypt/decrypt fields of records with key management services.
- Encryption and decryption using external key management service. Now it supports:
- Encryption and decryption at the field level.
- You can use JsonPath to specify the fields. NOTE: It has limited support for JsonPath syntax for now, please see JsonPath Limitations.
- Parse as a Struct when schema present, or a Map in the case of schemaless data.
Download the jar file from the release page and copy it into a directory that is under one of the plugin.path
.
This doccument would help you.
You can try the demo with Debezium + HashiCopr Vault here: debezium-encrypt-example.
Specifies the type designed for the record key or value:
io.github.rerorero.kafka.connect.transform.encrypt.Transform$Key
io.github.rerorero.kafka.connect.transform.encrypt.Transform$Value
Defines the key management service to encrypt/decrypt. Valid values are:
vault
for Hashicorp Vaultawskms
for Amazon Web Service KMSgcpkms
for Google Cloud Platform KMS
Specifies the mode. Valid values are:
encrypt
decrypt
JsonPath expression strings to specify the field to be encrypted or decrypted. Multiple path can be specified separated by commas.
NOTE: It has limited support for JsonPath syntax for now, please see JsonPath Limitations.
Specifies the conditions under which the transformation is be performed or not.
condition.field
should be JsonPath expression and condition.equals
should be a string.
When both are set, the transformation is performed only if the value of the field specified by condition.field
matches the value of condition.equals
.
All messages are transformed if both are omitted.
Specifies whether the key to encrypt/decrypt is asymmetric. Default is false
(symmetric).
Currently only gcpkms
supports the asymmetric enc/decryption.
You can see the example configuration file here.
URL of the Vault server.
The Vault token used to access Vault Transit Engine.
You can also specify it with the environment variable VAULT_TOKEN
instead.
Specifies the name of the encryption/decryption key to encrypt/decrypt against.
Specifies the Base64 context for key derivation. This is required if key derivation is enabled.
You can see the example configuration file here.
AWS credentials and the region to access KMS.
You can also specify them with the environment variable AWS_ACCESS_KEY_ID
and AWS_SECRET_ACCESS_KEY
instead.
Key ARN of the customer master key (CMK).
Specifies the encryption contexts.
It's parsed as a list of comma delimited key=value
pairs. e.g. key1=context1,key2=context2
Specifies the encryption algorithm.
Specifies the endpoint to access KMS.
See here for the example configuration file.
You can pass the file path to the GCP credential with the environment variable GOOGLE_APPLICATION_CREDENTIALS
.
GCP project id for the key ring.
Location of the key ring,
Key ring of the key.
The key to use for encryption
The version of the key. This is required when asymmetric
is true
because Cloud KMS doesn't support automatic key rotation for asymmetric keys.
Only the following syntaxes are supported for now:
Operator | Description |
---|---|
$ |
The root element. All JsonPath string has to be started with this operator. |
* |
Wildcard. Only supported for use as an array index. |
.<name> |
Dot-notated child. |
['name'] |
Bracket-notated child. Multiple names are not supported. |
[<number>] |
Array index. Multiple indices are not supported. |
Build and test:
gradlew build test
Run integration test:
gradlew build shadowJar
cd e2e
./test.sh
echo $? # should exit with 0