Skip to content

[native] Add memory reclaim stats to task runtime stats#21794

Merged
xiaoxmeng merged 1 commit intoprestodb:masterfrom
xiaoxmeng:stats
Jan 26, 2024
Merged

[native] Add memory reclaim stats to task runtime stats#21794
xiaoxmeng merged 1 commit intoprestodb:masterfrom
xiaoxmeng:stats

Conversation

@xiaoxmeng
Copy link
Contributor

@xiaoxmeng xiaoxmeng commented Jan 26, 2024

Add task runtime stats to collect memory reclaim stats during a task execution.
This helps to debug how much time spent on task memory reclaim of a slow query
execution. If the memory reclaim is triggered by the task itself, then the time will
also be counted in task cpu execution time. If not, it is counted in the task's scheduled
time but not execution time as the task will be stopped before memory reclamation.

@xiaoxmeng xiaoxmeng changed the title [native]Add memory reclaim stats to task runtime stats [native] Add memory reclaim stats to task runtime stats Jan 26, 2024
@xiaoxmeng xiaoxmeng marked this pull request as ready for review January 26, 2024 08:08
@xiaoxmeng xiaoxmeng requested a review from a team as a code owner January 26, 2024 08:08
Copy link
Contributor

@mbasmanova mbasmanova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xiaoxmeng Thank you for adding this stat. Looks good % small comment.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice to report these stats live while query is running.

There is code later that calls processTaskStats which clears taskRuntimeStats if query is not running.

Also, I noticed that Coordinator UI shows stats based on unit. Hence, you need to specify kNanos unit to ensure nice display.

For example,

  taskRuntimeStats.insert(
      {"nativeProcessCpuTime",
       RuntimeMetric(processCpuTime_, RuntimeCounter::Unit::kNanos)});

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean to call processTaskStats with tryToSkipIfRunning to false?

Add task runtime stats to collect memory reclaim stats during a task execution.
This helps to debug how much time spent on task memory reclaim of a slow query
execution. If the memory reclaim is triggered by the task itself, then the time will
also be counted in task cpu execution time. If not, it is counted in the task's scheduled
time but not execution time as the task will be stopped before memory reclamation.
Copy link
Contributor

@mbasmanova mbasmanova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks.

@xiaoxmeng xiaoxmeng merged commit 1a9009c into prestodb:master Jan 26, 2024
@wanglinsong wanglinsong mentioned this pull request Feb 12, 2024
64 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants