Skip to content

Conversation

@vvvvh123
Copy link
Member

@vvvvh123 vvvvh123 commented Jun 9, 2025

Description

This pull request adds a new config and session property: query_max_write_physical_size, similar to the existing query_max_scan_physical_bytes. It allows users to set an upper limit on the total physical size written by a query.

Due to Trino’s distributed execution model, this value serves as a lower bound rather than a strict limit. Queries that slightly exceed this threshold may still succeed; queries under this threshold will not be failed.

Additional context and related issues

fixes #25955

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text:

## General
* Add `query.max-write-physical-size configuration` property and `query_max_write_physical_size` session property to allow configuring limits on the amount of data written by a query. ({issue}`25955`)

@cla-bot
Copy link

cla-bot bot commented Jun 9, 2025

Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Vikas Harlani.
This is most likely caused by a git client misconfiguration; please make sure to:

  1. check if your git client is configured with an email to sign commits git config --list | grep email
  2. If not, set it up using git config --global user.email [email protected]
  3. Make sure that the git commit email is configured in your GitHub account settings, see https://github.com/settings/emails

@vvvvh123 vvvvh123 changed the title Add query-max-write-physical-bytes session property Add query-max-write-physical-bytes property Jun 15, 2025
@vvvvh123 vvvvh123 changed the title Add query-max-write-physical-bytes property Add query.max-write-physical-bytes property Jun 15, 2025
@vvvvh123 vvvvh123 force-pushed the add-query-max-write branch from 8a57b13 to 1ff39e9 Compare June 18, 2025 07:02
@cla-bot
Copy link

cla-bot bot commented Jun 18, 2025

Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Vikas Harlani.
This is most likely caused by a git client misconfiguration; please make sure to:

  1. check if your git client is configured with an email to sign commits git config --list | grep email
  2. If not, set it up using git config --global user.email [email protected]
  3. Make sure that the git commit email is configured in your GitHub account settings, see https://github.com/settings/emails

@github-actions github-actions bot added the docs label Jun 18, 2025
@vvvvh123 vvvvh123 force-pushed the add-query-max-write branch from 1ff39e9 to 27f472e Compare June 19, 2025 07:52
@cla-bot cla-bot bot added the cla-signed label Jun 19, 2025
@vvvvh123 vvvvh123 force-pushed the add-query-max-write branch 2 times, most recently from 2a3e493 to 9094774 Compare June 25, 2025 06:44
@raunaqmorarka raunaqmorarka marked this pull request as ready for review June 25, 2025 06:45
@raunaqmorarka raunaqmorarka changed the title Add query.max-write-physical-bytes property Add query.max-write-physical-size property Jun 25, 2025
@vvvvh123 vvvvh123 force-pushed the add-query-max-write branch from 9094774 to f7696ff Compare June 25, 2025 07:08
@vvvvh123 vvvvh123 force-pushed the add-query-max-write branch from f7696ff to 5a449ed Compare June 27, 2025 08:27
@raunaqmorarka raunaqmorarka requested a review from Copilot June 27, 2025 11:10
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Adds a new configuration and session property to limit the total physical size of data written by a query, enforces it at runtime in the query manager, and updates docs and tests accordingly.

  • Introduce query.max-write-physical-size config and query_max_write_physical_size session property.
  • Implement enforcement in SqlQueryManager with a new enforceWriteLimits method and exception.
  • Update documentation and add tests to verify write limit behavior.

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated no comments.

Show a summary per file
File Description
testing/trino-tests/src/test/java/io/trino/tests/TestQueryManager.java Add tests to verify queries fail when write limit is exceeded.
docs/src/main/sphinx/admin/properties-query-management.md Document the new query.max-write-physical-size property.
core/trino-spi/src/main/java/io/trino/spi/StandardErrorCode.java Define EXCEEDED_WRITE_LIMIT error code.
core/trino-main/src/test/java/io/trino/execution/TestQueryManagerConfig.java Extend config tests for query.max-write-physical-size.
core/trino-main/src/main/java/io/trino/execution/StateMachine.java Fix Javadoc formatting.
core/trino-main/src/main/java/io/trino/execution/SqlQueryManager.java Add maxQueryWritePhysicalSize field, enforcement loop, and exception handling.
core/trino-main/src/main/java/io/trino/execution/QueryManagerConfig.java Add queryMaxWritePhysicalSize configuration property.
core/trino-main/src/main/java/io/trino/SystemSessionProperties.java Register query_max_write_physical_size session property and getter.
core/trino-main/src/main/java/io/trino/ExceededWriteLimitException.java Implement exception thrown when write limits are exceeded.
Comments suppressed due to low confidence (1)

core/trino-main/src/main/java/io/trino/execution/SqlQueryManager.java:399

  • The code references Ordering.natural() but there is no import for com.google.common.collect.Ordering. Please add the missing import so the code compiles.
                        .flatMap(sessionLimit -> maxQueryWritePhysicalSize.map(serverLimit -> Ordering.natural().min(serverLimit, sessionLimit)))

@raunaqmorarka raunaqmorarka merged commit aa6e927 into trinodb:master Jun 27, 2025
98 checks passed
@github-actions github-actions bot added this to the 477 milestone Jun 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

Add property to limit query write physical bytes

6 participants