Skip to content

Conversation

@radek-kondziolka
Copy link
Contributor

@radek-kondziolka radek-kondziolka commented Mar 14, 2024

Description

When customer fetches results then QueryInfo object is created (if query is not finished) for every such request. QueryInfo object has some redundant fields that are not necessary for query protocol.
Computation of these fields in the sense of cpu and memory is expensive what we observe in our benchmarks and some users. Let's just avoid this computation.

Motivation

A lot of CPU cycles released

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
(x) Release notes are required. Please propose a release note for me.

# General
* Reduce coordinator CPU usage ({issue}`21075`)

( ) Release notes are required, with the following suggested text:

@cla-bot cla-bot bot added the cla-signed label Mar 14, 2024
@radek-kondziolka radek-kondziolka force-pushed the rk/query_info_optimization branch from 3b29807 to 30d906c Compare March 14, 2024 11:41
@radek-kondziolka radek-kondziolka marked this pull request as ready for review March 14, 2024 11:42
@radek-kondziolka radek-kondziolka force-pushed the rk/query_info_optimization branch from 30d906c to 11d20dd Compare March 15, 2024 09:34
Copy link
Member

@sopel39 sopel39 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm % comments

@radek-kondziolka radek-kondziolka force-pushed the rk/query_info_optimization branch from 11d20dd to 8edd451 Compare March 15, 2024 10:14
@radek-kondziolka
Copy link
Contributor Author

@sopel39 , done

Copy link
Member

@sopel39 sopel39 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small comment. Let's wait if @electrum has some comments

@radek-kondziolka radek-kondziolka force-pushed the rk/query_info_optimization branch from 8edd451 to a01648f Compare March 15, 2024 12:44
@sopel39
Copy link
Member

sopel39 commented Mar 18, 2024

please rebase

@radek-kondziolka radek-kondziolka force-pushed the rk/query_info_optimization branch 2 times, most recently from 62f9d18 to 8d2f8f5 Compare March 18, 2024 13:05
radek added 2 commits March 18, 2024 15:13
When a client (e.g. cli) fetches query result then Query#getNextResult
is called. This call constructs QueryInfo every time that is a heavy
operation. This commit introduces lighter query info representation
ResultQueryInfo and uses it in query protocol.
@radek-kondziolka radek-kondziolka force-pushed the rk/query_info_optimization branch from 8d2f8f5 to 95dafc3 Compare March 18, 2024 14:13
@sopel39 sopel39 merged commit ca55c48 into trinodb:master Mar 18, 2024
@sopel39
Copy link
Member

sopel39 commented Mar 18, 2024

thx!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

3 participants