Add configurable query history recording via queryHistoryEnabled

[hpopuri2]

Summary

This PR introduces a configurable option to enable/disable query history recording in Trino Gateway, addressing database performance bottlenecks and infrastructure cost concerns.

Problem

Currently, query history is always saved to the database, which creates several operational challenges:

• Database becomes a bottleneck under high query loads, causing performance degradation
• Forces operators to scale up database instances to handle the write load
• Increases infrastructure costs significantly due to larger DB instance requirements
• Query history may not be needed in all deployment scenarios (dev/test environments, alternative monitoring solutions)

Solution

Added a new configuration option queryHistoryEnabled in DataStoreConfiguration that allows operators to:

• Disable query history recording when not needed
• Use lower-tier database instances, reducing operational costs
• Improve overall gateway performance by eliminating unnecessary database writes
• Maintain flexibility to enable/disable based on deployment requirements

Configuration

yaml
  dataStore:
    jdbcUrl: jdbc:postgresql://localhost:5432/trino_gateway_db
    user: USER
    password: PASSWORD
    driver: org.postgresql.Driver
    queryHistoryHoursRetention: 24
    queryHistoryEnabled: true  # Set to false to disable query history recording

Default behavior: Query history is enabled by default (true) to maintain backward compatibility.

Benefits

• ✅ Eliminated database bottleneck - No more query history writes under high load
• ✅ Significant cost savings - Can use smaller, cheaper database instances
• ✅ Improved gateway performance - Reduced database round trips
• ✅ Operational flexibility - Enable/disable based on environment needs
• ✅ Backward compatible - Existing deployments continue working unchanged

Implementation Details

Files Changed

Production Code (7 files)

• gateway-ha/src/main/java/io/trino/gateway/ha/config/DataStoreConfiguration.java - Added queryHistoryEnabled field
• gateway-ha/src/main/java/io/trino/gateway/ha/config/RoutingConfiguration.java - Removed queryHistoryEnabled (moved to DataStore)
• gateway-ha/src/main/java/io/trino/gateway/proxyserver/ProxyRequestHandler.java - Added conditional query history recording
• gateway-ha/src/main/java/io/trino/gateway/ha/router/RoutingManager.java - Added setExternalUrlForQueryId() method
• gateway-ha/src/main/java/io/trino/gateway/ha/router/BaseRoutingManager.java - Made setExternalUrlForQueryId() public
• gateway-ha/config.yaml - Added configuration documentation
• docs/installation.md - Documented feature with examples

Test Code (5 files)

• gateway-ha/src/test/java/io/trino/gateway/ha/config/TestDataStoreConfiguration.java - New test class
• gateway-ha/src/test/java/io/trino/gateway/ha/config/TestRoutingConfiguration.java - Updated tests
• gateway-ha/src/test/java/io/trino/gateway/proxyserver/TestProxyRequestHandlerQueryHistoryEnabled.java - New test
• gateway-ha/src/test/java/io/trino/gateway/proxyserver/TestProxyRequestHandlerQueryHistoryDisabled.java - New test
• gateway-ha/src/test/resources/test-config-template.yml - Updated config path
• gateway-ha/src/test/resources/test-config-with-query-history-disabled.yml - Test fixture

All tests pass: 292 tests run, 0 failures, 0 errors

Backward Compatibility

✅ Fully backward compatible

• Default value is true, maintaining existing behavior
• Existing configurations without this setting will continue to record query history
• No database schema changes required
• No breaking changes to APIs or interfaces

Follow-up
After this PR is approved and merged, a follow-up PR will be submitted to the Trino Gateway Helm Charts repository to expose this configuration option in the Helm values, making it easier for Kubernetes deployments to configure this setting.

[cla-bot]

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

[kbhatianr]

This seems more apt for DataStoreConfiguration rather than RoutingConfiguration.

Especially since we already have the queryHistoryHoursRetention property available there.

[kbhatianr]

Can you also add the configuration to the docs?

[Chaho12]

Sounds reasonable.
Can you update the docs too? https://github.com/trinodb/trino-gateway/blob/main/docs/installation.md?plain=1#L39

[cla-bot]

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

[hpopuri2]

Sounds reasonable. Can you update the docs too? https://github.com/trinodb/trino-gateway/blob/main/docs/installation.md?plain=1#L39

yup done

[hpopuri2]

This seems more apt for DataStoreConfiguration rather than RoutingConfiguration.

Especially since we already have the queryHistoryHoursRetention property available there.

Yup.Agree. Done

[mosabua]

@cla-bot check

[cla-bot]

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

[cla-bot]

The cla-bot has been summoned, and re-checked this pull request!

[hpopuri2]

@cla-bot check

[cla-bot]

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

[cla-bot]

The cla-bot has been summoned, and re-checked this pull request!