fix(postgres sink): enable TLS flag

[thomasqueirozb]

Summary

User reported issue: sqlx wasn't compiled with any TLS feature flags and therefore the Postgres Sink was unable to be used in a TLS enabled environment.

This PR adds tls-rustls-aws-lc-rs tls-rustls-ring feature to sqlx dependency in (FIPS compliant)

Vector configuration

data_dir: /tmp/vector-data

# Generate test data
sources:
  demo_logs:
    type: demo_logs
    format: json
    interval: 1.0
    count: 10

# Parse the JSON data
transforms:
  parse_json:
    type: remap
    inputs:
      - demo_logs
    source: |
      # Parse the log data and prepare for postgres
      .timestamp = now()
      .level = .severity
      .source = "demo"
      .host = .host
      .metadata = {
        "service": .service
      }

# Postgres sink with SSL
sinks:
  postgres:
    type: postgres
    inputs:
      - parse_json
    endpoint: "postgres://testuser:testpass@localhost:5432/testdb?sslmode=require"
    table: logs

How did you test this PR?

1. docker-compose.yml:

services:
  postgres:
    image: postgres:15-alpine
    environment:
      POSTGRES_USER: testuser
      POSTGRES_PASSWORD: testpass
      POSTGRES_DB: testdb
    ports:
      - "5432:5432"
    volumes:
      - ./init.sql:/docker-entrypoint-initdb.d/init.sql
      - ./certs:/var/lib/postgresql/certs
    command: >
      postgres
      -c ssl=on
      -c ssl_cert_file=/var/lib/postgresql/certs/server.crt
      -c ssl_key_file=/var/lib/postgresql/certs/server.key
      -c ssl_ca_file=/var/lib/postgresql/certs/ca.crt
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U testuser -d testdb"]
      interval: 5s
      timeout: 5s
      retries: 5

2. init.sql

CREATE TABLE logs (
    timestamp TIMESTAMPTZ,
    message TEXT,
    level TEXT,
    source TEXT,
    metadata JSONB,
    host TEXT
);

3. Generated self signed certs:

#!/bin/bash

# Create certs directory
mkdir -p certs
cd certs

# Generate CA private key and certificate
openssl req -new -x509 -days 365 -nodes -out ca.crt -keyout ca.key -subj "/CN=postgres-ca"

# Generate server private key and certificate signing request
openssl req -new -nodes -out server.csr -keyout server.key -subj "/CN=postgres"

# Sign the server certificate with CA
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -days 365

# Set proper permissions
chmod 600 server.key ca.key
chmod 644 server.crt ca.crt

# Clean up CSR
rm server.csr

echo "SSL certificates generated successfully!"
ls -la

4. Ran docker compose up -d

5. Verified SSL was enabled

$ docker exec postgres-ssl-test-postgres-1 psql -U testuser -d testdb -c "SHOW ssl;"
 ssl
-----
 on
(1 row)

6. Verified no data was in there

$ docker exec vector-progress-sqlx-tls-postgres-1 psql -U testuser -d testdb -c "SELECT COUNT(*) FROM logs;"

 count
-------
     0
(1 row)

7. Ran the config with cargo run -- --config postgres.yaml and then verified that the data was indeed there:

$ docker exec vector-progress-sqlx-tls-postgres-1 psql -U testuser -d testdb -c "SELECT COUNT(*) FROM logs;"

 count 
-------
    10

Change Type

• Bug fix
• New feature
• Non-functional (chore, refactoring, docs)
• Performance

Is this a breaking change?

• Yes
• No

Does this PR include user facing changes?

• Yes. Please add a changelog fragment based on our guidelines.
• No. A maintainer will apply the no-changelog label to this PR.

References

• Closes: How to enable ssl on postgres sink #23517
• Closes: Support TLS for sqlx #23069

[datadog-vectordotdev]

✅ Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 2b857b4 | Docs | Was this helpful? Give us feedback!}

[danielecazzari]

Hi, do you have any timeline for this merge? It's really an important feature to ensure that postgreSQL is usable in prod scenario

[GrantMelvin]

Is there any update regarding this?

[thomasqueirozb]

Hi all, this PR is blocked because it failed an arm build when building sqlx with tls-rustls-aws-lc-rs. See run

I root caused this and it is a failure in rustls not being able to build on ARMv7 with the aws-lc-rs feature flag. I opened an issue (rustls/rustls#2725) and the maintainers quickly and correctly pointed out that aws-lc-rs does cross compile correctly (see comment).

This seems to either be a legitimate bug in aws-lc-rs or something wrong with the cross compilation in the current environment (I suspect this is the real issue). I haven't had the time to debug the cross compilation issue further but I did provide a comprehensive example in the rustls issue. Anyone should be able to grab that example and just change the crate from rustls to aws-lc-rs to debug further. If someone knows what is wrong/missing from the cross compilation environment and how to fix it please ping me so I can include the fix in this PR.

Maybe related: aws/aws-lc-rs#769

[pront]

If someone from the community is interested in picking this up, we can use https://docs.rs/crate/sqlx/0.8.6/features#tls-rustls-ring instead.