Skip to content

feat(http-server): Add a KeyStoreScanner#135

Merged
tdcmeehan merged 1 commit intoprestodb:masterfrom
aaneja:keyStoreScanner
Dec 3, 2025
Merged

feat(http-server): Add a KeyStoreScanner#135
tdcmeehan merged 1 commit intoprestodb:masterfrom
aaneja:keyStoreScanner

Conversation

@aaneja
Copy link
Copy Markdown

@aaneja aaneja commented Nov 25, 2025

Rebased clone of #134, since @ShahimSharafudeen will be OOO for a while

Additionally -

  • Fixed the test since it was not asserting on the server cert
  • Addressed sourcery's suggestion to poll for the cert change instead of a fixed sleep in the test

@linux-foundation-easycla
Copy link
Copy Markdown

linux-foundation-easycla bot commented Nov 25, 2025

CLA Signed

The committers listed above are authorized under a signed CLA.

  • ✅ login: aaneja / name: Anant Aneja (c269182)

@sourcery-ai
Copy link
Copy Markdown

sourcery-ai bot commented Nov 25, 2025

Reviewer's Guide

Introduce dynamic HTTPS keystore reloading via Jetty KeyStoreScanner and add configuration, wiring, and tests to validate certificate rotation behavior on both main and admin connectors.

Sequence diagram for HTTPS keystore reload via KeyStoreScanner

sequenceDiagram
    actor Operator
    participant FileSystem
    participant KeyStoreScanner
    participant SslContextFactory
    participant JettyServer
    participant HttpsConnector as HttpsConnector
    participant Client

    Operator->>FileSystem: Replace keystore file with new certificate

    loop Periodic scan
        KeyStoreScanner->>FileSystem: checkLastModified(keystorePath)
        FileSystem-->>KeyStoreScanner: lastModifiedTimestamp
        alt Keystore changed
            KeyStoreScanner->>SslContextFactory: reloadFromKeystore()
            SslContextFactory-->>KeyStoreScanner: reloadComplete
            KeyStoreScanner->>JettyServer: notifyKeyStoreChanged()
            JettyServer-->>KeyStoreScanner: acknowledged
        else No change
            KeyStoreScanner-->>KeyStoreScanner: waitForNextInterval
        end
    end

    Client->>HttpsConnector: new TLS connection
    HttpsConnector->>SslContextFactory: getCurrentCertificateChain()
    SslContextFactory-->>HttpsConnector: certificateChain(with rotated cert)
    HttpsConnector-->>Client: TLS handshake using new certificate
Loading

Class diagram for HttpServerConfig and HttpServer keystore scanning integration

classDiagram
    class HttpServerConfig {
        - boolean httpEnabled
        - boolean httpsEnabled
        - String keyStorePath
        - String keyStorePassword
        - String keyManagerPassword
        - int keyStoreScanIntervalSeconds
        + boolean isHttpEnabled()
        + boolean isHttpsEnabled()
        + String getKeyStorePath()
        + String getKeyStorePassword()
        + String getKeyManagerPassword()
        + HttpServerConfig setKeystoreScanIntervalSeconds(int keyStoreScanIntervalSeconds)
        + int getKeystoreScanIntervalSeconds()
    }

    class HttpServer {
        - HttpServerInfo httpServerInfo
        - HttpServerConfig config
        - Server server
        + HttpServer(HttpServerInfo httpServerInfo, HttpServerConfig config, ...)  
    }

    class Server {
        + void addBean(Object bean)
    }

    class Connector {
        + void addBean(Object bean)
    }

    class SslContextFactory {
        + void setKeyStorePath(String keyStorePath)
        + void setKeyStorePassword(String keyStorePassword)
        + void setKeyManagerPassword(String keyManagerPassword)
    }

    class KeyStoreScanner {
        - SslContextFactory sslContextFactory
        + KeyStoreScanner(SslContextFactory sslContextFactory)
        + void setScanInterval(int seconds)
    }

    class AdminConnector extends Connector

    HttpServerConfig "1" --> "1" HttpServer : provides_config
    HttpServer "1" --> "1" Server : wraps
    HttpServer "1" --> "1" SslContextFactory : configures_https
    Server "1" --> "*" Connector : has_connectors
    Server "1" --> "*" KeyStoreScanner : manages_scanners
    Connector "1" --> "*" KeyStoreScanner : manages_scanners
    HttpServer "1" --> "1" AdminConnector : admin_https
    KeyStoreScanner "1" --> "1" SslContextFactory : scans_and_triggers_reload
Loading

File-Level Changes

Change Details Files
Add configuration option for periodic HTTPS keystore scanning and wire it into Jetty server setup.
  • Introduce keyStoreScanIntervalSeconds field with getter/setter on HttpServerConfig and map it to the http-server.https.keystore.scan-interval-seconds config property.
  • In HttpServer constructor, create a KeyStoreScanner for the HTTPS sslContextFactory, set its scan interval from configuration, and register it as a bean on both the user and admin connectors where HTTPS is enabled.
http-server/src/main/java/com/facebook/airlift/http/server/HttpServerConfig.java
http-server/src/main/java/com/facebook/airlift/http/server/HttpServer.java
Extend TLS test resources and configuration tests to cover keystore scanning.
  • Update clientcert.sh to generate an alternate server certificate and keystore (updatedServer.keystore) signed by the same CA for rotation tests.
  • Adjust HttpServerConfig tests to include the new keystore scan interval property in both default and explicit property mapping assertions.
http-server/src/test/resources/clientcert-java/clientcert.sh
http-server/src/test/java/com/facebook/airlift/http/server/TestHttpServerConfig.java
Add integration test verifying that the server picks up keystore changes and return the updated certificate subject.
  • Introduce a TestNG DataProvider to run keystore update tests with admin HTTPS both disabled and enabled.
  • Refactor the certificate echo servlet helper into a reusable createCertTestServlet method placed near the top of TestHttpServerProvider.
  • Add testKeystoreUpdate which starts the server with a temporary keystore, asserts the original certificate subject, replaces the keystore file, then polls until the updated certificate subject is observed or fails with a timeout.
  • Add helper getServerCertSubject that connects over HTTPS with a trust-all SSLContext, retrieves the server’s leaf certificate, and returns its subject DN string for assertions.
http-server/src/test/java/com/facebook/airlift/http/server/TestHttpServerProvider.java

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@aaneja aaneja force-pushed the keyStoreScanner branch 2 times, most recently from e11ceab to ac569ad Compare November 25, 2025 13:17
@aaneja aaneja marked this pull request as ready for review November 25, 2025 13:17
@aaneja aaneja requested a review from a team as a code owner November 25, 2025 13:17
Copy link
Copy Markdown

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there - I've reviewed your changes - here's some feedback:

  • The new keystore scan interval config is accepted as an int without validation; consider enforcing a non-negative (or strictly positive) range and/or documenting what a value of 0 means to avoid ambiguous behavior at runtime.
  • In testClientCertificateJava, the fixed Thread.sleep(2500) tied to the scan interval can make the test brittle and slow; consider polling for the expected behavior or otherwise synchronizing with the KeyStoreScanner rather than relying on a hard-coded sleep.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The new keystore scan interval config is accepted as an int without validation; consider enforcing a non-negative (or strictly positive) range and/or documenting what a value of 0 means to avoid ambiguous behavior at runtime.
- In testClientCertificateJava, the fixed Thread.sleep(2500) tied to the scan interval can make the test brittle and slow; consider polling for the expected behavior or otherwise synchronizing with the KeyStoreScanner rather than relying on a hard-coded sleep.

## Individual Comments

### Comment 1
<location> `http-server/src/test/java/com/facebook/airlift/http/server/TestHttpServerProvider.java:322-331` </location>
<code_context>

-    @Test
-    public void testClientCertificateJava()
+    @Test(dataProvider = "adminMode")
+    public void testClientCertificateJava(boolean adminMode)
             throws Exception
</code_context>

<issue_to_address>
**suggestion (testing):** Avoid fixed Thread.sleep in test and use a more robust way to wait for keystore reload

Relying on `Thread.sleep(2500)` to wait for `KeyStoreScanner` makes this test flaky and slow, especially in CI. Instead, could you poll for the expected behavior with a timeout (e.g., repeatedly issue requests until the subject reflects the new CN or a max wait is reached)? This should improve reliability and may complete faster when the reload happens quickly.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment on lines +322 to +331
@Test(dataProvider = "adminMode")
public void testClientCertificateJava(boolean adminMode)
throws Exception
{
tempDir = createTempDirectory("test-keystore").toFile().getCanonicalFile();
String tempKeyStorePath = tempDir.toPath().resolve("server.keystore").toAbsolutePath().toString();
String originalKeyStorePath = getResource("clientcert-java/server.keystore").getPath();
String replacementKeyStorePath = getResource("clientcert-java/replacementServer.keystore").getPath();

Files.copy(Path.of(originalKeyStorePath), Path.of(tempKeyStorePath), StandardCopyOption.REPLACE_EXISTING);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (testing): Avoid fixed Thread.sleep in test and use a more robust way to wait for keystore reload

Relying on Thread.sleep(2500) to wait for KeyStoreScanner makes this test flaky and slow, especially in CI. Instead, could you poll for the expected behavior with a timeout (e.g., repeatedly issue requests until the subject reflects the new CN or a max wait is reached)? This should improve reliability and may complete faster when the reload happens quickly.

@aaneja aaneja force-pushed the keyStoreScanner branch 5 times, most recently from 16b2102 to c1ea7bb Compare December 2, 2025 12:45
@aaneja aaneja marked this pull request as draft December 2, 2025 17:59
To allow for hot reloading the tls keystore

See for more details : https://jetty.org/docs/jetty/12.1/programming-guide/server/http.html#connector-protocol-tls-keystore-auto-reload

Co-authored-by: Shahim Sharafudeen <33122287+shahimsharafudeen@users.noreply.github.com>
@aaneja
Copy link
Copy Markdown
Author

aaneja commented Dec 3, 2025

@sourcery-ai review

@aaneja aaneja marked this pull request as ready for review December 3, 2025 06:07
return new Object[][] {{false}, {true}};
}

private static HttpServlet createCertTestServlet()
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Accidental move with code formatting, but good to have this private static on top

Copy link
Copy Markdown

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there - I've reviewed your changes - here's some feedback:

  • In HttpServer, the KeyStoreScanner setup for the main connector and the admin connector is duplicated; consider extracting a small helper that conditionally creates/adds the scanner (e.g., skipping it when the scan interval is zero) to avoid repetition and keep the semantics consistent.
  • The new keyStoreScanIntervalSeconds field in HttpServerConfig appears to accept any int; if negative or very small values are not meaningful, consider adding validation (e.g., via a constraint annotation or explicit check) to prevent misconfiguration.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- In `HttpServer`, the `KeyStoreScanner` setup for the main connector and the admin connector is duplicated; consider extracting a small helper that conditionally creates/adds the scanner (e.g., skipping it when the scan interval is zero) to avoid repetition and keep the semantics consistent.
- The new `keyStoreScanIntervalSeconds` field in `HttpServerConfig` appears to accept any int; if negative or very small values are not meaningful, consider adding validation (e.g., via a constraint annotation or explicit check) to prevent misconfiguration.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Copy link
Copy Markdown

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there - I've reviewed your changes - here's some feedback:

  • Consider only creating/adding the KeyStoreScanner bean when getKeystoreScanIntervalSeconds() is greater than zero to avoid unnecessary scanner setup when the feature is effectively disabled.
  • In testKeystoreUpdate, using Path.of(getResource(...).getPath()) can break with spaces or special characters in the resource path; prefer constructing the Path from getResource(...).toURI() instead for robustness.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- Consider only creating/adding the `KeyStoreScanner` bean when `getKeystoreScanIntervalSeconds()` is greater than zero to avoid unnecessary scanner setup when the feature is effectively disabled.
- In `testKeystoreUpdate`, using `Path.of(getResource(...).getPath())` can break with spaces or special characters in the resource path; prefer constructing the `Path` from `getResource(...).toURI()` instead for robustness.

## Individual Comments

### Comment 1
<location> `http-server/src/main/java/com/facebook/airlift/http/server/HttpServerConfig.java:829-831` </location>
<code_context>
         return httpComplianceViolations;
     }
+
+    @Config("http-server.https.keystore.scan-interval-seconds")
+    @ConfigDescription("Interval (in seconds) to check the keystore file for changes")
+    public HttpServerConfig setKeystoreScanIntervalSeconds(int keyStoreScanIntervalSeconds)
+    {
+        this.keyStoreScanIntervalSeconds = keyStoreScanIntervalSeconds;
</code_context>

<issue_to_address>
**issue (bug_risk):** Consider validating the scan-interval value to guard against negative inputs.

This setter forwards any `int` to `KeyStoreScanner#setScanInterval`. A negative value (from misconfiguration) could lead to unexpected behavior or runtime errors in the Jetty scanner. Adding a guard, e.g. `checkArgument(keyStoreScanIntervalSeconds >= 0, "...")`, would fail fast and surface misconfigurations early.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@aaneja
Copy link
Copy Markdown
Author

aaneja commented Dec 3, 2025

Hey there - I've reviewed your changes - here's some feedback:

* Consider only creating/adding the `KeyStoreScanner` bean when `getKeystoreScanIntervalSeconds()` is greater than zero to avoid unnecessary scanner setup when the feature is effectively disabled.

* In `testKeystoreUpdate`, using `Path.of(getResource(...).getPath())` can break with spaces or special characters in the resource path; prefer constructing the `Path` from `getResource(...).toURI()` instead for robustness.

Prompt for AI Agents

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

The default behavior for the scanner when set to 0s is to do no scanning at all, so this is not an overhead

@aaneja aaneja changed the title feat(server): Add a KeyStoreScanner feat(http-server): Add a KeyStoreScanner Dec 3, 2025
Copy link
Copy Markdown
Member

@agrawalreetika agrawalreetika left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@tdcmeehan tdcmeehan merged commit 23d833a into prestodb:master Dec 3, 2025
3 checks passed
aaneja added a commit to prestodb/presto that referenced this pull request Dec 4, 2025
Airlift release details -
https://github.com/prestodb/airlift/actions/runs/19903584529/job/57053806773

## Motivation and Context
- Upgrade to Jetty 12.0.29 which includes recent CVE fixes
- Add ability to hot swap the keystore, see
prestodb/airlift#135 for details

## Impact
Users can now hot swap their keystore files to update certs

## Test Plan
CI

## Contributor checklist

- [ ] Please make sure your submission complies with our [contributing
guide](https://github.com/prestodb/presto/blob/master/CONTRIBUTING.md),
in particular [code
style](https://github.com/prestodb/presto/blob/master/CONTRIBUTING.md#code-style)
and [commit
standards](https://github.com/prestodb/presto/blob/master/CONTRIBUTING.md#commit-standards).
- [ ] PR description addresses the issue accurately and concisely. If
the change is non-trivial, a GitHub Issue is referenced.
- [ ] Documented new properties (with its default value), SQL syntax,
functions, or other functionality.
- [ ] If release notes are required, they follow the [release notes
guidelines](https://github.com/prestodb/presto/wiki/Release-Notes-Guidelines).
- [ ] Adequate tests were added if applicable.
- [ ] CI passed.
- [ ] If adding new dependencies, verified they have an [OpenSSF
Scorecard](https://securityscorecards.dev/#the-checks) score of 5.0 or
higher (or obtained explicit TSC approval for lower scores).

## Release Notes
Please follow [release notes
guidelines](https://github.com/prestodb/presto/wiki/Release-Notes-Guidelines)
and fill in the release notes below.

```
== RELEASE NOTES ==

General Changes
* Add a new ``http-server.https.keystore.scan-interval-seconds`` configuration flag to scan the keystore file periodically for new certs
* Upgrade Jetty to 12.0.29 in response to `CVE-2025-5115 <https://nvd.nist.gov/vuln/detail/CVE-2025-5115>`_. 
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants