Introduce query limits at the resource group level#16303
Introduce query limits at the resource group level#16303tdcmeehan merged 2 commits intoprestodb:masterfrom
Conversation
5e3db7e to
85f46c1
Compare
...com/facebook/presto/resourceGroups/reloading/ReloadingResourceGroupConfigurationManager.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/ExceededCpuLimitException.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/dispatcher/FailedDispatchQuery.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/execution/QueryTracker.java
Outdated
Show resolved
Hide resolved
...ource-group-managers/src/main/java/com/facebook/presto/resourceGroups/ResourceGroupSpec.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/execution/SqlQueryManager.java
Outdated
Show resolved
Hide resolved
...resource-group-managers/src/main/java/com/facebook/presto/resourceGroups/StaticSelector.java
Outdated
Show resolved
Hide resolved
d4b4731 to
84c028e
Compare
mayankgarg1990
left a comment
There was a problem hiding this comment.
At a high level - 2 main comments here -
- Lets rename the field names to start with
perQueryand end withLimit. If the variable type isDataSize/Duration- then the name can end withLimit. However, if the fields have the typelong- in that case we need to put the unit at the end of the name likesMillisorBytesso that it is clear to the reader of the code what is the unit in question for the long value. - Use
requireNonNullwherever possible in constructors to ensure that we are not passing aroundnulls
...o-main/src/main/java/com/facebook/presto/execution/resourceGroups/InternalResourceGroup.java
Outdated
Show resolved
Hide resolved
...o-main/src/main/java/com/facebook/presto/execution/resourceGroups/InternalResourceGroup.java
Outdated
Show resolved
Hide resolved
...o-main/src/main/java/com/facebook/presto/execution/resourceGroups/InternalResourceGroup.java
Outdated
Show resolved
Hide resolved
...o-main/src/main/java/com/facebook/presto/execution/resourceGroups/InternalResourceGroup.java
Outdated
Show resolved
Hide resolved
...o-main/src/main/java/com/facebook/presto/execution/resourceGroups/InternalResourceGroup.java
Outdated
Show resolved
Hide resolved
...ource-group-managers/src/main/java/com/facebook/presto/resourceGroups/ResourceGroupSpec.java
Outdated
Show resolved
Hide resolved
...p-managers/src/main/java/com/facebook/presto/resourceGroups/db/ResourceGroupSpecBuilder.java
Outdated
Show resolved
Hide resolved
...ce-group-managers/src/main/java/com/facebook/presto/resourceGroups/db/ResourceGroupsDao.java
Outdated
Show resolved
Hide resolved
...ce-group-managers/src/main/java/com/facebook/presto/resourceGroups/db/ResourceGroupsDao.java
Outdated
Show resolved
Hide resolved
presto-spi/src/main/java/com/facebook/presto/spi/resourceGroups/ResourceGroupQueryLimits.java
Outdated
Show resolved
Hide resolved
mayankgarg1990
left a comment
There was a problem hiding this comment.
One thing I realized after reviewing this PR is that the values that we passed to InternalResourceGroup is not really being used since we are just using the values from ResourceGroupConfigurationManager - lets see if we need those changes - we can make the changes a lot simpler
presto-main/src/main/java/com/facebook/presto/ExceededCpuLimitException.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/ExceededCpuLimitException.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/ExceededMemoryLimitException.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/dispatcher/FailedDispatchQuery.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/execution/QueryTracker.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/execution/SqlQueryManager.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/execution/SqlQueryManager.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/execution/SqlQueryManager.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/execution/LimitWithLevelContext.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/memory/ClusterMemoryManager.java
Outdated
Show resolved
Hide resolved
|
haven't reviewed the full PR - will continue the same tomorrow |
6afefbf to
1ed5b7d
Compare
mayankgarg1990
left a comment
There was a problem hiding this comment.
Review for commit " Add resource limits per query on a resource group "
...o-main/src/main/java/com/facebook/presto/execution/resourceGroups/InternalResourceGroup.java
Outdated
Show resolved
Hide resolved
presto-spi/src/main/java/com/facebook/presto/spi/resourceGroups/ResourceGroup.java
Outdated
Show resolved
Hide resolved
1ed5b7d to
97d8dfe
Compare
presto-main/src/main/java/com/facebook/presto/ExceededCpuLimitException.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/ExceededCpuLimitException.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/dispatcher/LocalDispatchQuery.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/dispatcher/LocalDispatchQuery.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/execution/DataDefinitionExecution.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/execution/LimitWithLevelContext.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/execution/QueryTracker.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/execution/SqlQueryManager.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/memory/ClusterMemoryManager.java
Outdated
Show resolved
Hide resolved
...resource-group-managers/src/main/java/com/facebook/presto/resourceGroups/SelectorResult.java
Outdated
Show resolved
Hide resolved
presto-spi/src/main/java/com/facebook/presto/spi/resourceGroups/ResourceGroupQueryLimits.java
Outdated
Show resolved
Hide resolved
presto-spi/src/main/java/com/facebook/presto/spi/resourceGroups/ResourceGroupQueryLimits.java
Outdated
Show resolved
Hide resolved
presto-spi/src/main/java/com/facebook/presto/spi/resourceGroups/ResourceGroupQueryLimits.java
Outdated
Show resolved
Hide resolved
7969aa4 to
9b45461
Compare
mayankgarg1990
left a comment
There was a problem hiding this comment.
Add resource limits per query on a resource group
For this commit - mostly good - just some naming issues
You should rename the commit title to Support passing per query limits in resource group configuration
The commit description starts with saying Allows for more intelligent thread allocation for queues - this diff commit doesn't do that. Please ensure that the commit description and title matches with what the commit actually does
...ource-group-managers/src/main/java/com/facebook/presto/resourceGroups/ResourceGroupSpec.java
Outdated
Show resolved
Hide resolved
...ce-group-managers/src/test/java/com/facebook/presto/resourceGroups/TestingResourceGroup.java
Outdated
Show resolved
Hide resolved
mayankgarg1990
left a comment
There was a problem hiding this comment.
Also - please make the SelectorResult/SelectionContext changes we talked about
presto-main/src/main/java/com/facebook/presto/execution/QueryLimit.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/execution/SqlQueryExecution.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/execution/SqlQueryManager.java
Outdated
Show resolved
Hide resolved
presto-main/src/test/java/com/facebook/presto/dispatcher/TestLocalDispatchQuery.java
Outdated
Show resolved
Hide resolved
presto-main/src/test/java/com/facebook/presto/dispatcher/TestLocalDispatchQuery.java
Outdated
Show resolved
Hide resolved
.../src/main/java/com/facebook/presto/resourceGroups/FileResourceGroupConfigurationManager.java
Outdated
Show resolved
Hide resolved
9b45461 to
77026a9
Compare
|
@tdcmeehan - this diff is ready for an initial look. We are still looking at unittests and code style fixes - but otherwise - would love to get an idea on the approach and the PR should be ready for final review by tomorrow. |
0757343 to
9361c39
Compare
mayankgarg1990
left a comment
There was a problem hiding this comment.
Getting there - this should be last major set of review comments
presto-main/src/main/java/com/facebook/presto/execution/DataDefinitionExecution.java
Outdated
Show resolved
Hide resolved
...resource-group-managers/src/main/java/com/facebook/presto/resourceGroups/SelectorResult.java
Outdated
Show resolved
Hide resolved
...s/src/main/java/com/facebook/presto/resourceGroups/AbstractResourceConfigurationManager.java
Outdated
Show resolved
Hide resolved
...s/src/main/java/com/facebook/presto/resourceGroups/AbstractResourceConfigurationManager.java
Outdated
Show resolved
Hide resolved
...s/src/main/java/com/facebook/presto/resourceGroups/AbstractResourceConfigurationManager.java
Outdated
Show resolved
Hide resolved
...managers/src/main/java/com/facebook/presto/resourceGroups/db/DbSourceExactMatchSelector.java
Outdated
Show resolved
Hide resolved
...com/facebook/presto/resourceGroups/reloading/ReloadingResourceGroupConfigurationManager.java
Outdated
Show resolved
Hide resolved
...managers/src/main/java/com/facebook/presto/resourceGroups/db/DbSourceExactMatchSelector.java
Outdated
Show resolved
Hide resolved
.../src/main/java/com/facebook/presto/resourceGroups/FileResourceGroupConfigurationManager.java
Outdated
Show resolved
Hide resolved
...s/src/main/java/com/facebook/presto/resourceGroups/AbstractResourceConfigurationManager.java
Outdated
Show resolved
Hide resolved
9361c39 to
4dd45d8
Compare
...ource-group-managers/src/main/java/com/facebook/presto/resourceGroups/ResourceGroupSpec.java
Outdated
Show resolved
Hide resolved
presto-spi/src/main/java/com/facebook/presto/spi/resourceGroups/ResourceGroupQueryLimits.java
Outdated
Show resolved
Hide resolved
presto-spi/src/main/java/com/facebook/presto/spi/resourceGroups/ResourceGroupQueryLimits.java
Outdated
Show resolved
Hide resolved
409786c to
ccb9d9a
Compare
mayankgarg1990
left a comment
There was a problem hiding this comment.
mostly naming changes and things here and there
...o-main/src/main/java/com/facebook/presto/execution/resourceGroups/InternalResourceGroup.java
Outdated
Show resolved
Hide resolved
...ource-group-managers/src/main/java/com/facebook/presto/resourceGroups/ResourceGroupSpec.java
Outdated
Show resolved
Hide resolved
...p-managers/src/main/java/com/facebook/presto/resourceGroups/db/ResourceGroupSpecBuilder.java
Outdated
Show resolved
Hide resolved
presto-spi/src/main/java/com/facebook/presto/spi/resourceGroups/ResourceGroupQueryLimits.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/execution/SqlQueryExecution.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/execution/SqlQueryExecution.java
Outdated
Show resolved
Hide resolved
...o-main/src/main/java/com/facebook/presto/execution/resourceGroups/InternalResourceGroup.java
Outdated
Show resolved
Hide resolved
...facebook/presto/resourceGroups/reloading/TestReloadingResourceGroupConfigurationManager.java
Outdated
Show resolved
Hide resolved
...facebook/presto/resourceGroups/reloading/TestReloadingResourceGroupConfigurationManager.java
Outdated
Show resolved
Hide resolved
ccb9d9a to
a20caa3
Compare
...o-main/src/main/java/com/facebook/presto/execution/resourceGroups/InternalResourceGroup.java
Outdated
Show resolved
Hide resolved
presto-tests/src/test/resources/resource_groups_config_simple.json
Outdated
Show resolved
Hide resolved
ebecbcf to
27fe716
Compare
presto-main/src/main/java/com/facebook/presto/dispatcher/LocalDispatchQuery.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/dispatcher/LocalDispatchQuery.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/dispatcher/LocalDispatchQuery.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/execution/QueryTracker.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/execution/QueryTracker.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/execution/SqlQueryManager.java
Outdated
Show resolved
Hide resolved
...o-main/src/main/java/com/facebook/presto/execution/resourceGroups/InternalResourceGroup.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/memory/ClusterMemoryManager.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/execution/QueryLimit.java
Outdated
Show resolved
Hide resolved
4029568 to
5ff2403
Compare
Setting resource group resources on a query for cpu time, execution time, and memory usage, adds greater granularity to resource allocation.
5ff2403 to
a151fbf
Compare
presto-main/src/main/java/com/facebook/presto/execution/DataDefinitionExecution.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/execution/SqlQueryExecution.java
Outdated
Show resolved
Hide resolved
Enables query manager to kill a query which exceeds its resource group allocation.
a151fbf to
b237980
Compare
|
testTwoQueriesAtSameTime is failing most of the times. I have several open PRs and i have seen that failing at least 3 times. Error: Tests run: 1874, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1,800.32 s <<< FAILURE! - in TestSuite Often fails. This is the last change that touched this code. Could this be a regression or should the test be just disabled ? |
Introduce the ability to add query limits (cpu time, execution time, and memory limit) at the resource group level. These limits can complement the
SelectorResourceEstimatepassed toSelectorSpecto ensure that if the query exceeds the indented query limits, we can fail the query and protect the resource group.This is enabled by adding to the
ResourceGroupSpecthe fieldperQueryLimits, a JSON object of the optional fieldsexecutionTimeLimit,totalMemoryLimit,cpuTimeLimitwhich is then passed to the coordinator to implement the appropriate limits.Test plan - Unit tests added and build success are sufficient for first commit. Second commit will have e2e by deploying fbpkg to test cluster to ensure resource group limits can be enforced by
SqlQueryManager. Unit testing also on second commit.Depends on https://github.com/facebookexternal/presto-facebook/pull/1599