[FIPS] Test that libbeat ES client will not connect to ES with invalid TLS certificate by ycombinator · Pull Request #45158 · elastic/beats

ycombinator · 2025-07-03T00:39:22Z

Proposed commit message

This PR adds a new test, TestConnectionTLS, that fakes an Elasticsearch HTTPS server that returns a TLS certificate that's been created with a key length of < 2048 bits, making it invalid for FIPS-compliant use.

If running in FIPS mode, the test asserts that the Beat's connection to Elasticsearch will fail with a TLS error.
If not running in FIPS mode, the test asserts that the Beat's connection to Elasticsearch will succeed.

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
~~I have made corresponding changes to the documentation~~
~~I have made corresponding change to the default configuration files~~
I have added tests that prove my fix is effective or that my feature works
~~I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.~~

Disruptive User Impact

None; this PR just adds a new unit test.

How to test this PR locally

In a non-FIPS environment:

$ go test ./libbeat/esleg/eslegclient/... -v -test.run TestConnectionTLS -test.count 1
=== RUN   TestConnectionTLS
    logger.go:146: 2025-07-02T17:38:22.328-0700	INFO	TestConnectionTLS.esclientleg	logp/logger.go:212	elasticsearch url: https://127.0.0.1:56871
    logger.go:146: 2025-07-02T17:38:22.329-0700	DEBUG	TestConnectionTLS.esclientleg	logp/logger.go:207	ES Ping(url=https://127.0.0.1:56871)
    logger.go:146: 2025-07-02T17:38:22.330-0700	DEBUG	TestConnectionTLS.esclientleg	logp/logger.go:207	Completed dialing successfully	{"network.transport": "tcp", "server.address": "127.0.0.1:56871"}
    logger.go:146: 2025-07-02T17:38:22.331-0700	DEBUG	TestConnectionTLS.esclientleg	logp/logger.go:207	Ping status code: 200
    logger.go:146: 2025-07-02T17:38:22.331-0700	INFO	TestConnectionTLS.esclientleg	logp/logger.go:212	Attempting to connect to Elasticsearch version 9.2.0-SNAPSHOT (default)
--- PASS: TestConnectionTLS (0.00s)
PASS
ok  	github.com/elastic/beats/v7/libbeat/esleg/eslegclient	0.398s

In a FIPS environment, i.e. with the Microsoft Go fork installed and with the OpenSSL FIPS provider installed:

$ GOEXPERIMENT=systemcrypto go test --tags=requirefips ./libbeat/esleg/eslegclient/... -v -test.run TestConnectionTLS -test.count 1
=== RUN   TestConnectionTLS
    logger.go:146: 2025-07-02T17:36:54.162-0700	INFO	TestConnectionTLS.esclientleg	logp/logger.go:212	elasticsearch url: https://127.0.0.1:39835
    logger.go:146: 2025-07-02T17:36:54.162-0700	DEBUG	TestConnectionTLS.esclientleg	logp/logger.go:207	ES Ping(url=https://127.0.0.1:39835)
    logger.go:146: 2025-07-02T17:36:54.164-0700	ERROR	TestConnectionTLS.esclientleg	logp/logger.go:222	Error dialing remote error: tls: internal error{"network.transport": "tcp", "server.address": "127.0.0.1:39835"}
        github.com/elastic/elastic-agent-libs/logp.(*Logger).Errorf
        	/home/shaunak/go/pkg/mod/github.com/elastic/elastic-agent-libs@v0.20.0/logp/logger.go:222
        github.com/elastic/elastic-agent-libs/transport/httpcommon.(*HTTPTransportSettings).RoundTripper.LoggingDialer.func2
        	/home/shaunak/go/pkg/mod/github.com/elastic/elastic-agent-libs@v0.20.0/transport/logging.go:39
        github.com/elastic/elastic-agent-libs/transport.DialerFunc.DialContext
        	/home/shaunak/go/pkg/mod/github.com/elastic/elastic-agent-libs@v0.20.0/transport/transport.go:44
        net/http.(*Transport).customDialTLS
        	/home/shaunak/ms-go/go/src/net/http/transport.go:1470
        net/http.(*Transport).dialConn
        	/home/shaunak/ms-go/go/src/net/http/transport.go:1756
        net/http.(*Transport).dialConnFor
        	/home/shaunak/ms-go/go/src/net/http/transport.go:1615
        net/http.(*Transport).startDialConnForLocked.func1
        	/home/shaunak/ms-go/go/src/net/http/transport.go:1597
    logger.go:146: 2025-07-02T17:36:54.164-0700	DEBUG	TestConnectionTLS.esclientleg	logp/logger.go:207	Ping request failed with: Get "https://127.0.0.1:39835": remote error: tls: internal error
--- PASS: TestConnectionTLS (0.00s)
PASS
ok  	github.com/elastic/beats/v7/libbeat/esleg/eslegclient	0.020s

elasticmachine · 2025-07-03T00:39:25Z

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

github-actions · 2025-07-03T00:39:30Z

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

ycombinator · 2025-07-03T16:17:19Z

CI, specifically the Libbeat: Ubuntu x86_64 Unit Tests with requirefips build tag step, is failing on the test written in this PR:

=== Failed
--
  | === FAIL: libbeat/esleg/eslegclient TestConnectionTLS (0.01s)
  | logger.go:146: 2025-07-03T01:29:28.223Z	INFO	TestConnectionTLS.esclientleg	logp/logger.go:212	elasticsearch url: https://127.0.0.1:46361
  | logger.go:146: 2025-07-03T01:29:28.224Z	DEBUG	TestConnectionTLS.esclientleg	logp/logger.go:207	ES Ping(url=https://127.0.0.1:46361)
  | logger.go:146: 2025-07-03T01:29:28.227Z	DEBUG	TestConnectionTLS.esclientleg	logp/logger.go:207	Completed dialing successfully	{"network.transport": "tcp", "server.address": "127.0.0.1:46361"}
  | logger.go:146: 2025-07-03T01:29:28.227Z	DEBUG	TestConnectionTLS.esclientleg	logp/logger.go:207	Ping status code: 200
  | logger.go:146: 2025-07-03T01:29:28.227Z	INFO	TestConnectionTLS.esclientleg	logp/logger.go:212	Attempting to connect to Elasticsearch version 9.2.0-SNAPSHOT (default)
  | connection_fips_test.go:73:
  | Error Trace:	/opt/buildkite-agent/builds/bk-agent-prod-gcp-1751506014588605207/elastic/beats-libbeat/libbeat/esleg/eslegclient/connection_fips_test.go:73
  | Error:      	An error is expected but got nil.
  | Test:       	TestConnectionTLS

This test passes locally for me when I run it on a FIPS-configured VM.

$ GOEXPERIMENT=systemcrypto go test --tags=requirefips ./libbeat/esleg/eslegclient/... -v -test.run TestConnectionTLS -test.count 1
=== RUN   TestConnectionTLS
    logger.go:146: 2025-07-03T09:11:15.725-0700	INFO	TestConnectionTLS.esclientleg	logp/logger.go:212	elasticsearch url: https://127.0.0.1:46507
    logger.go:146: 2025-07-03T09:11:15.726-0700	DEBUG	TestConnectionTLS.esclientleg	logp/logger.go:207	ES Ping(url=https://127.0.0.1:46507)
2025/07/03 09:11:15 http: TLS handshake error from 127.0.0.1:34092: tls: failed to sign handshake: EVP_PKEY_sign_init failed
openssl error(s):
error:1C800069:Provider routines::invalid key length
	providers/common/securitycheck.c:65
    logger.go:146: 2025-07-03T09:11:15.727-0700	ERROR	TestConnectionTLS.esclientleg	logp/logger.go:222	Error dialing remote error: tls: internal error{"network.transport": "tcp", "server.address": "127.0.0.1:46507"}
        github.com/elastic/elastic-agent-libs/logp.(*Logger).Errorf
        	/home/shaunak/go/pkg/mod/github.com/elastic/elastic-agent-libs@v0.20.0/logp/logger.go:222
        github.com/elastic/elastic-agent-libs/transport/httpcommon.(*HTTPTransportSettings).RoundTripper.LoggingDialer.func2
        	/home/shaunak/go/pkg/mod/github.com/elastic/elastic-agent-libs@v0.20.0/transport/logging.go:39
        github.com/elastic/elastic-agent-libs/transport.DialerFunc.DialContext
        	/home/shaunak/go/pkg/mod/github.com/elastic/elastic-agent-libs@v0.20.0/transport/transport.go:44
        net/http.(*Transport).customDialTLS
        	/home/shaunak/ms-go/go/src/net/http/transport.go:1470
        net/http.(*Transport).dialConn
        	/home/shaunak/ms-go/go/src/net/http/transport.go:1756
        net/http.(*Transport).dialConnFor
        	/home/shaunak/ms-go/go/src/net/http/transport.go:1615
        net/http.(*Transport).startDialConnForLocked.func1
        	/home/shaunak/ms-go/go/src/net/http/transport.go:1597
    logger.go:146: 2025-07-03T09:11:15.727-0700	DEBUG	TestConnectionTLS.esclientleg	logp/logger.go:207	Ping request failed with: Get "https://127.0.0.1:46507": remote error: tls: internal error
--- PASS: TestConnectionTLS (0.00s)
PASS
ok  	github.com/elastic/beats/v7/lib

I believe this test fails in CI because we're running it on the same VM we use for all other tests, not on a FIPS-configured VM:

beats/.buildkite/libbeat/pipeline.libbeat.yml

Line 102 in 0b1170f

image: "${IMAGE_UBUNTU_X86_64}"

belimawr

Aside from the failing tests: LGTM.

I'll wait for CI to be green before a final approval.

@ycombinator, just to understand, what is your plan here? Update Beats to use FIPS-enabled VMs before merging this PR? If so, would this be part of this PR?

ycombinator · 2025-07-07T17:55:21Z

@ycombinator, just to understand, what is your plan here? Update Beats to use FIPS-enabled VMs before merging this PR? If so, would this be part of this PR?

@michel-laterman has put up #45199 to use FIPS-enabled VMS for the FIPS unit tests in Beats. Once that PR is merged, I'll rebase this PR here on main, and then the test introduced in this PR should start passing in CI. I updated this PR's description to reflect the dependency.

michel-laterman · 2025-07-17T20:45:56Z

@belimawr, @AndersonQ, could you please review?

belimawr

LGMT

The certificate will expire in 5 years, if you want you could generate them every test like we do in elastic-agent-libs: https://github.com/elastic/elastic-agent-libs/blob/main/transport/tlscommontest/test_helper.go so we'll never have to update the tests to renew certificates.

…d TLS certificate (#45158) Add a new test under libbeat, TestConnectionTLS, that fakes an Elasticsearch HTTPS server that returns a TLS certificate that's been created with a key length of < 2048 bits, making it invalid for FIPS-compliant use. If running in FIPS mode, the test asserts that the Beat's connection to Elasticsearch will fail with a TLS error. If not running in FIPS mode, the test asserts that the Beat's connection to Elasticsearch will succeed. (cherry picked from commit 4db5493)

…d TLS certificate (#45158) (#45596) Add a new test under libbeat, TestConnectionTLS, that fakes an Elasticsearch HTTPS server that returns a TLS certificate that's been created with a key length of < 2048 bits, making it invalid for FIPS-compliant use. If running in FIPS mode, the test asserts that the Beat's connection to Elasticsearch will fail with a TLS error. If not running in FIPS mode, the test asserts that the Beat's connection to Elasticsearch will succeed. (cherry picked from commit 4db5493) Co-authored-by: Shaunak Kashyap <ycombinator@gmail.com>

…d TLS certificate (#45158) (#45595) Add a new test under libbeat, TestConnectionTLS, that fakes an Elasticsearch HTTPS server that returns a TLS certificate that's been created with a key length of < 2048 bits, making it invalid for FIPS-compliant use. If running in FIPS mode, the test asserts that the Beat's connection to Elasticsearch will fail with a TLS error. If not running in FIPS mode, the test asserts that the Beat's connection to Elasticsearch will succeed. (cherry picked from commit 4db5493) Co-authored-by: Shaunak Kashyap <ycombinator@gmail.com>

ycombinator added 2 commits July 2, 2025 13:49

WIP

4055866

Remove unused files

f49074b

ycombinator requested a review from a team as a code owner July 3, 2025 00:39

ycombinator added :Testing Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team backport-8.19 Automated backport to the 8.19 branch labels Jul 3, 2025

ycombinator requested review from AndersonQ and belimawr July 3, 2025 00:39

ycombinator added the backport-9.1 Automated backport to the 9.1 branch label Jul 3, 2025

botelastic Bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jul 3, 2025

botelastic Bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jul 3, 2025

ycombinator requested review from kruskall and michel-laterman July 3, 2025 00:39

mergify Bot assigned ycombinator Jul 3, 2025

Running make fmt

012d1f8

ycombinator changed the title ~~[FIPS] Test that libbeat ES client will reject invalid certificate from ES~~ [FIPS] Test that libbeat ES client will not connect to ES with invalid TLS certificate Jul 3, 2025

ycombinator added 2 commits July 2, 2025 18:04

Run mage update

67eb893

Remove server log

b11f613

michel-laterman approved these changes Jul 3, 2025

View reviewed changes

Comment thread libbeat/esleg/eslegclient/testdata/ca.crt

Refactor: move test into existing test file

2d13a3a

michel-laterman mentioned this pull request Jul 4, 2025

[FIPS][libbeat] Use VM with FIPS provider to run libbeat fips tests #45199

Merged

belimawr reviewed Jul 7, 2025

View reviewed changes

mergify Bot mentioned this pull request Jul 17, 2025

[8.19](backport #45199) [FIPS][libbeat] Use VM with FIPS provider to run libbeat fips tests #45429

Merged

Merge branch 'main' into fips-es-conn

9982d3a

michel-laterman self-assigned this Jul 17, 2025

mergify Bot mentioned this pull request Jul 17, 2025

[9.1](backport #45199) [FIPS][libbeat] Use VM with FIPS provider to run libbeat fips tests #45434

Merged

michel-laterman and others added 3 commits July 17, 2025 11:23

Merge branch 'main' into fips-es-conn

61b0a07

Fix linter

b27af15

Merge branch 'main' into fips-es-conn

febeaba

michel-laterman requested a review from belimawr July 17, 2025 20:43

michel-laterman enabled auto-merge (squash) July 17, 2025 20:44

belimawr approved these changes Jul 28, 2025

View reviewed changes

michel-laterman merged commit 4db5493 into elastic:main Jul 28, 2025
204 checks passed

ycombinator deleted the fips-es-conn branch July 28, 2025 17:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FIPS] Test that libbeat ES client will not connect to ES with invalid TLS certificate#45158

[FIPS] Test that libbeat ES client will not connect to ES with invalid TLS certificate#45158
michel-laterman merged 10 commits into
elastic:mainfrom
ycombinator:fips-es-conn

ycombinator commented Jul 3, 2025 •

edited

Loading

Uh oh!

elasticmachine commented Jul 3, 2025

Uh oh!

github-actions Bot commented Jul 3, 2025

Uh oh!

ycombinator commented Jul 3, 2025 •

edited

Loading

Uh oh!

Uh oh!

belimawr left a comment

Uh oh!

ycombinator commented Jul 7, 2025

Uh oh!

michel-laterman commented Jul 17, 2025

Uh oh!

belimawr left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

ycombinator commented Jul 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed commit message

Checklist

Disruptive User Impact

How to test this PR locally

Uh oh!

elasticmachine commented Jul 3, 2025

Uh oh!

github-actions Bot commented Jul 3, 2025

🤖 GitHub comments

Uh oh!

ycombinator commented Jul 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

belimawr left a comment

Choose a reason for hiding this comment

Uh oh!

ycombinator commented Jul 7, 2025

Uh oh!

michel-laterman commented Jul 17, 2025

Uh oh!

belimawr left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ycombinator commented Jul 3, 2025 •

edited

Loading

ycombinator commented Jul 3, 2025 •

edited

Loading