Skip to content

[ML] Validates and limits threading parameters for starting model deployment #171921

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
darnautov merged 4 commits into from
Nov 27, 2023
Merged

[ML] Validates and limits threading parameters for starting model deployment #171921

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
ea06a35
validate threading params
darnautov Nov 24, 2023
cf7b44e
update message
darnautov Nov 24, 2023
8679e47
Update x-pack/plugins/ml/public/application/model_management/deployme…
darnautov Nov 27, 2023
cc94724
Merge branch 'main' into ml-171883-limit-allocation-params
kibanamachine Nov 27, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions x-pack/plugins/ml/common/types/ml_server_info.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ export interface MlServerDefaults {
export interface MlServerLimits {
max_model_memory_limit?: string;
effective_max_model_memory_limit?: string;
max_single_ml_node_processors?: number;
total_ml_processors?: number;
}

export interface MlInfoResponse {
Expand Down
55 changes: 41 additions & 14 deletions x-pack/plugins/ml/public/application/model_management/deployment_setup.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ import type { I18nStart, OverlayStart, ThemeServiceStart } from '@kbn/core/publi
import { css } from '@emotion/react';
import { numberValidator } from '@kbn/ml-agg-utils';
import { toMountPoint } from '@kbn/react-kibana-mount';
import { isCloudTrial } from '../services/ml_server_info';
import { getNewJobLimits, isCloudTrial } from '../services/ml_server_info';
import {
composeValidators,
dictionaryValidator,
Expand All @@ -42,7 +42,7 @@ import { ModelItem } from './models_list';
interface DeploymentSetupProps {
config: ThreadingParams;
onConfigChange: (config: ThreadingParams) => void;
errors: Partial<Record<keyof ThreadingParams, object>>;
errors: Partial<Record<keyof ThreadingParams, Record<string, unknown>>>;
isUpdate?: boolean;
deploymentsParams?: Record<string, ThreadingParams>;
}
Expand All @@ -66,6 +66,11 @@ export const DeploymentSetup: FC<DeploymentSetupProps> = ({
isUpdate,
deploymentsParams,
}) => {
const {
total_ml_processors: totalMlProcessors,
max_single_ml_node_processors: maxSingleMlNodeProcessors,
} = getNewJobLimits();

const numOfAllocation = config.numOfAllocations;
const threadsPerAllocations = config.threadsPerAllocations;

Expand All @@ -76,17 +81,20 @@ export const DeploymentSetup: FC<DeploymentSetupProps> = ({

const threadsPerAllocationsOptions = useMemo(
() =>
new Array(THREADS_MAX_EXPONENT).fill(null).map((v, i) => {
const value = Math.pow(2, i);
const id = value.toString();

return {
id,
label: id,
value,
};
}),
[]
new Array(THREADS_MAX_EXPONENT)
.fill(null)
.map((v, i) => Math.pow(2, i))
.filter(maxSingleMlNodeProcessors ? (v) => v <= maxSingleMlNodeProcessors : (v) => true)
.map((value) => {
const id = value.toString();

return {
id,
label: id,
value,
};
}),
[maxSingleMlNodeProcessors]
);

const disableThreadingControls = config.priority === 'low';
Expand Down Expand Up @@ -251,11 +259,28 @@ export const DeploymentSetup: FC<DeploymentSetupProps> = ({
}
hasChildLabel={false}
isDisabled={disableThreadingControls}
isInvalid={!!errors.numOfAllocations}
error={
errors?.numOfAllocations?.min ? (
<FormattedMessage
id="xpack.ml.trainedModels.modelsList.startDeployment.numbersOfAllocationsMinError"
defaultMessage="At least one allocation is required."
/>
) : errors?.numOfAllocations?.max ? (
<FormattedMessage
id="xpack.ml.trainedModels.modelsList.startDeployment.numbersOfAllocationsMaxError"
defaultMessage="Cannot exceed {max} - the total number of ML processors."
values={{ max: totalMlProcessors }}
/>
) : null
}
>
<EuiFieldNumber
disabled={disableThreadingControls}
isInvalid={!!errors.numOfAllocations}
fullWidth
min={1}
max={totalMlProcessors}
Copy link

@droberts195 droberts195 Nov 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does undefined mean "no max" here?

Copy link
Contributor Author

@darnautov darnautov Nov 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, exactly

step={1}
name={'numOfAllocations'}
value={disableThreadingControls ? 1 : numOfAllocation}
Expand Down Expand Up @@ -346,6 +371,8 @@ export const StartUpdateDeploymentModal: FC<StartDeploymentModalProps> = ({
}) => {
const isUpdate = !!initialParams;

const { total_ml_processors: totalMlProcessors } = getNewJobLimits();

const [config, setConfig] = useState<ThreadingParams>(
initialParams ?? {
numOfAllocations: 1,
Expand Down Expand Up @@ -373,7 +400,7 @@ export const StartUpdateDeploymentModal: FC<StartDeploymentModalProps> = ({

const numOfAllocationsValidator = composeValidators(
requiredValidator(),
numberValidator({ min: 1, integerOnly: true })
numberValidator({ min: 1, max: totalMlProcessors, integerOnly: true })
);

const numOfAllocationsErrors = numOfAllocationsValidator(config.numOfAllocations);
Expand Down