fix(native): Caps the count metric at int64_t's max value#27295
fix(native): Caps the count metric at int64_t's max value#27295aditi-pandit merged 1 commit intoprestodb:masterfrom
Conversation
Reviewer's guide (collapsed on small PRs)Reviewer's GuideChanges the native execution runtime metrics structure so that the count field uses an unsigned 64-bit integer type instead of a signed 64-bit integer, aligning Presto’s count metric representation with Velox and preventing overflow/negative counts while leaving other metric fields unchanged. Updated class diagram for RuntimeMetric count type changeclassDiagram
class RuntimeMetric_before {
String name
RuntimeUnit unit
int64_t sum
int64_t count
int64_t max
int64_t min
}
class RuntimeMetric {
String name
RuntimeUnit unit
int64_t sum
uint64_t count
int64_t max
int64_t min
}
RuntimeMetric_before <.. RuntimeMetric : count field changed to uint64_t
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
There was a problem hiding this comment.
Hey - I've left some high level feedback:
- Since
RuntimeMetric::countis part of a protocol header, double-check all serialization/deserialization and wire formats (e.g., JSON, Thrift, Velox interop) to ensure the signed→unsigned change is reflected consistently and does not break backward compatibility with existing consumers. - Audit all arithmetic and comparisons involving
RuntimeMetric::countto ensure there are no remaining implicit conversions between signed and unsigned types that could introduce subtle bugs or compiler warnings.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- Since `RuntimeMetric::count` is part of a protocol header, double-check all serialization/deserialization and wire formats (e.g., JSON, Thrift, Velox interop) to ensure the signed→unsigned change is reflected consistently and does not break backward compatibility with existing consumers.
- Audit all arithmetic and comparisons involving `RuntimeMetric::count` to ensure there are no remaining implicit conversions between signed and unsigned types that could introduce subtle bugs or compiler warnings.Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
| RuntimeUnit unit = {}; | ||
| int64_t sum = {}; | ||
| int64_t count = {}; | ||
| uint64_t count = {}; |
There was a problem hiding this comment.
Thanks for letting me know! Based on your suggestion, we’ll probably need this PR: #27276. Could you please advise on the preferred order for merging? Would it be better to merge the Velox refactor first, or should we merge this fix into Presto first to avoid potentially breaking anything?
There was a problem hiding this comment.
Yes the linked PR would work, I think it is better to advance velox submodule and make the Presto metric fix in the same commit to avoid breaking anything.
There was a problem hiding this comment.
Thanks for confirming!
There was a problem hiding this comment.
Hi @pramodsatya, after discussing with the Velox community, it turns out the Velox CI jobs won’t pass without this Presto layer to ensure no overflow occurs. Could you help review and land this patch first? I’ll switch to using saturateCast in a follow-up for simplicity once the Velox submodule is advanced. Thanks for your help!
| const RuntimeMetric& metric) { | ||
| // Presto's RuntimeMetric uses int64_t for count, but Velox's RuntimeMetric | ||
| // uses uint64_t. To avoid overflow, we cap the count at int64_t's max value. | ||
| auto count = static_cast<int64_t>(std::min( |
There was a problem hiding this comment.
Thanks for the prompt feedback. I've made the update.
aditi-pandit
left a comment
There was a problem hiding this comment.
Thanks @rui-mo for this code.
| // uses uint64_t. To avoid overflow, we cap the count at int64_t's max value. | ||
| const auto count = static_cast<int64_t>(std::min( | ||
| metric.count, | ||
| static_cast<uint64_t>(std::numeric_limits<int64_t>::max()))); |
There was a problem hiding this comment.
Can you abstract this as a static constant as its used per toRuntimeMetric call ?
There was a problem hiding this comment.
Thanks for helping review. I made the update.
|
Hi @pramodsatya @aditi-pandit, thanks for the review! If there are no further comments, can we go ahead and land this change? |
aditi-pandit
left a comment
There was a problem hiding this comment.
Thanks @rui-mo. Just one minor comment.
|
Hi @aditi-pandit @pramodsatya, could you please help land this change if no further comments? Thanks! |
NivinCS
left a comment
There was a problem hiding this comment.
Thanks for the change. LGTM.
|
Thanks everyone for the reviews and for helping get this PR merged. I’ll follow up with cleanup once the Velox version is updated. |
…7295) ## Description <!---Describe your changes in detail--> Presto's RuntimeMetric uses int64_t for count, but Velox's RuntimeMetric uses uint64_t. To avoid overflow, we cap the count at int64_t's max value. ## Motivation and Context <!---Why is this change required? What problem does it solve?--> <!---If it fixes an open issue, please link to the issue here.--> A refactor to change the count metric from int64_t to uint64_t is going on in Velox. To avoid overflow when converting to Presto metric, this PR updates Presto count metric to consistent type. facebookincubator/velox#15536 ``` == NO RELEASE NOTE == ```
Description
Presto's RuntimeMetric uses int64_t for count, but Velox's RuntimeMetric
uses uint64_t. To avoid overflow, we cap the count at int64_t's max value.
Motivation and Context
A refactor to change the count metric from int64_t to uint64_t is going on in
Velox. To avoid overflow when converting to Presto metric, this PR updates
Presto count metric to consistent type.
facebookincubator/velox#15536
Impact
Ensures no overflow when converting the uint64 count metric to int64.
Test Plan
Contributor checklist
Release Notes
Please follow release notes guidelines and fill in the release notes below.
If release note is NOT required, use: