Trash overgrown schedulers with many usage queues#1672
Trash overgrown schedulers with many usage queues#1672ryoqun merged 2 commits intoanza-xyz:masterfrom
Conversation
| const DEFAULT_POOL_CLEANER_INTERVAL: Duration = Duration::from_secs(10); | ||
| const DEFAULT_MAX_POOLING_DURATION: Duration = Duration::from_secs(180); | ||
| // Rough estimate of max UsageQueueLoader size in bytes: | ||
| // UsageFromTask * UsageQeueue's capacity * DEFAULT_MAX_USAGE_QUEUE_COUNT |
There was a problem hiding this comment.
typo:
| // UsageFromTask * UsageQeueue's capacity * DEFAULT_MAX_USAGE_QUEUE_COUNT | |
| // UsageFromTask * UsageQueue's capacity * DEFAULT_MAX_USAGE_QUEUE_COUNT |
| // 16 bytes * 128 items * 262_144 entries == 512 MiB | ||
| // It's expected that there will be 2 or 3 pooled schedulers constantly when running against | ||
| // mainnnet-beta. That means the total memory consumption for the idle close-to-be-trashed pooled | ||
| // schedulers is set to high: 1.0 ~ 1.5 GiB. This value is chosen to maximize performance under the |
There was a problem hiding this comment.
the "high" reads a bit confusingly to me. maybe justs "is set to 1.0 ~ 1.5 GiB"
| @@ -648,6 +671,10 @@ where | |||
| } | |||
|
|
|||
| fn is_trashed(&self) -> bool { | |||
There was a problem hiding this comment.
so want to make a clarification here in the review, doesn't necessarily need a code comment.
This is only called after the scheduler is terminated due to abort or successful completion of the slot, is that right?
I want to make sure that the limits from is_overgrown will not cause us to skip a slot if such a large one comes through.
There was a problem hiding this comment.
This is only called after the scheduler is terminated due to abort or successful completion of the slot, is that right?
yes. it's only called by ::return_to_pool().
I want to make sure that the limits from
is_overgrownwill not cause us to skip a slot if such a large one comes through.
this is true, again.
* Trash overgrown schedulers with many usage queues * Fix a typo and improve comment wording
2 participants
Problem
From #1211:
Summary of Changes
This pr addresses (4).