Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/vale/styles/OpenSearch/SubstitutionsError.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ swap:
'allowlist': allow list
'autoscaling': auto scaling
'command-line interface': command line interface
'coordinator node': coordinating node
'coordinator nodes': coordinating nodes
'data are': data is
'data set': dataset
'for information on': for information about
Expand Down
71 changes: 71 additions & 0 deletions .github/vale/styles/OpenSearch/SubstitutionsSuggestion.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,35 @@ swap:
'app server': application server
'as well as': and
'bring up': start or launch
'brings up': starts or launches
'bringing up': starting or launching
'build out': expand or implement
'builds out': expands or implements
'building out': expanding or implementing
'catch up': synchronize with
'catches up': synchronizes with
'catching up': synchronizing with
'clean up': remove or normalize
'cleans up': removes or normalizes
'cleaning up': removing or normalizing
'deal with': process or resolve
'deals with': processes or resolves
'dealing with': processing or resolving
'due to': because of
'fall back': revert
'fall back to': default to
'falls back': reverts
'falls back to': defaults to
'falling back': reverting
'figure out': determine or identify
'figures out': determines or identifies
'figuring out': determining or identifying
'fire up': start or launch
'fires up': starts or launches
'firing up': starting or launching
'flesh out': elaborate or define
'fleshes out': elaborates or defines
'fleshing out': elaborating or defining
'from scratch': manually, from the beginning, or without a template
'get': receive, obtain, or retrieve
'gets': receives, returns, or retrieves
Expand All @@ -28,47 +46,100 @@ swap:
'handling': managing, processing, or resolving
'it is recommended': we recommend
'kick off': start or initiate
'kicks off': starts or initiates
'kicking off': starting or initiating
'know about': detect or recognize
'knows about': detects or recognizes
'leverage': use
'life cycle': lifecycle
'live in': are located in
'lives': is located
'lives in': is located in
'look into': investigate
'looks into': investigates
'looking into': investigating
'looks like this': appears as follows
'navigate in': navigate to
'one-off': single-use or ad-hoc
'out of the box': by default, natively, or is built-in
'out-of-the-box': default, native, or built-in
'pick up': load or detect
'picks up': loads or detects
'picking up': loading or detecting
'plug in': integrate or use
'plugs in': integrates or uses
'plugging in': integrating or using
'plug into': integrate with
'plugs into': integrates with
'plumbing': configuration or infrastructure
'point to': reference
'points to': references
'pointing to': referencing
'pull in': import or retrieve
'pulls in': imports or retrieves
'pulling in': importing or retrieving
'push back': postpone
'pushes back': postpones
'pushing back': postponing
'read in': ingest or parse
'reads in': ingests or parses
'reading in': ingesting or parsing
'roll out': deploy or release
'rolls out': deploys or releases
'rolling out': deploying or releasing
'scale up': scale or increase capacity
'scales up': scales or increases capacity
'scaling up': scaling or increasing capacity
'see': view
'sees': detects or finds
'set up': configure or initialize
'sets up': configures or initializes
'setting up': configuring or initializing
'show up': appear
'shows up': appears
'showing up': appearing
'shut down': stop or terminate
'shuts down': stops or terminates
'shutting down': stopping or terminating
'slow down': delay, reduce throughput, or increase latency
'slows down': delays or reduces throughput
'slowing down': delaying or reducing throughput
'speed up': accelerate
'speeds up': accelerates
'speeding up': accelerating
'spin up': provision or start
'spins up': provisions or starts
'spinning up': provisioning or starting
'split up': divide or partition
'splits up': divides or partitions
'splitting up': dividing or partitioning
'start here': getting started
'stitching together': chaining or combining
'talk to': communicate with or connect to
'talks to': communicates with or connects to
'talking to': communicating with or connecting to
'touching': modifying or changing
'tear down': remove or decommission
'tears down': removes or decommissions
'tearing down': removing or decommissioning
'teardown': removal or decommission
'tell': instruct or configure
'tells': instructs or configures
'telling': instructing or configuring
'think': evaluate or determine
'thinks': evaluates or determines
'thinking': evaluating or determining
'under the hood': internally
'walks you through': describes or explains
'want': require or expect
'wants': requires or expects
'wire up': connect or configure
'wires up': connects or configures
'wiring up': connecting or configuring
'wiring': configuration or networking
'wish|desire': want
'work around': mitigate or bypass
'works around': mitigates or bypasses
'working around': mitigating or bypassing
'workaround': mitigation or alternative approach
'works with': supports, integrates with, or is compatible with
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ Then restart your cluster. For more information, see [Installing plugins]({{site

## Workload groups

A _workload group_ is a logical grouping of tasks with defined resource limits. System administrators can dynamically manage workload groups using the Workload Management APIs. These workload groups can be used to create search requests with resource limits. For more information, see [Workload groups]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/workload-management/workload-groups/).
A _workload group_ is a logical grouping of tasks with defined resource limits. System administrators can dynamically manage workload groups using the Workload Management APIs. These workload groups can be used to create search requests with resource limits. You can also define group-specific settings that are applied automatically to every request routed to the group. For more information, see [Workload groups]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/workload-management/workload-groups/) and [Workload group settings]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/workload-management/workload-group-settings/).

The following example request adds a workload group named `analytics`:

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
layout: default
title: Workload group rules
nav_order: 30
nav_order: 25
parent: Workload management
grand_parent: Availability and recovery
redirect_from:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
---
layout: default
title: Workload group settings
nav_order: 30
parent: Workload management
grand_parent: Availability and recovery
---

# Workload group settings
**Introduced 3.7**
{: .label .label-purple }

OpenSearch operation is normally controlled by cluster-wide defaults and per-request parameters. In a multi-tenant cluster, you may need to apply different limits to different tenants. Without workload group settings, you can either lower the defaults for everyone, which restricts tenants that operate within limits, or trust every client to send the correct request parameters, which is difficult to enforce.

Workload group settings solve this problem by letting you attach group-specific configuration directly to a [workload group]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/workload-management/workload-groups/). When a request is routed to a group, the group's settings are applied automatically, letting you define guardrails per tenant:

- Apply stricter limits to resource-intensive or unverified tenants while keeping generous defaults for others, all without modifying cluster settings.
- Limits are bound to the workload group, so they apply to every request routed to the group regardless of which client sent it. No client-side configuration is required.
- A workload group can optionally take precedence over lenient request-level values, protecting the cluster from uncontrolled queries without rejecting them entirely.
- All guardrails for a tenant are located in one place alongside the group's `resource_limits` and `resiliency_mode`.

## Supported settings

You can configure settings in the `settings` object of a workload group. All settings are optional. Only the settings you explicitly define on a workload group are enforced; any setting you omit defaults to the corresponding request parameter or cluster default. Each workload group setting accepts the same value range as the underlying request parameter or cluster setting it maps to.

The following table lists the supported workload group settings.

| Setting | Type | Description | Equivalent request parameter | Equivalent cluster setting |
| :--- | :--- | :--- | :--- | :--- |
| `search.default_search_timeout` | Time unit | The maximum amount of time a shard can spend on query execution. When a shard exceeds this timeout, it stops collecting hits and returns its current results to the coordinating node, which may produce partial results. | [`timeout`]({{site.url}}{{site.baseurl}}/api-reference/search-apis/search/) | [`search.default_search_timeout`]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/search-settings/) |
| `search.cancel_after_time_interval` | Time unit | The maximum amount of time the entire search request can run at the coordinating node level. When the interval is reached, the request and all associated tasks are canceled and the client receives an error rather than partial results. | [`cancel_after_time_interval`]({{site.url}}{{site.baseurl}}/api-reference/search-apis/search/) | [`search.cancel_after_time_interval`]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/search-settings/) |
| `search.max_concurrent_shard_requests` | Integer | The maximum number of concurrent shard-level requests a single search may issue per node. Limits search fan-out. | [`max_concurrent_shard_requests`]({{site.url}}{{site.baseurl}}/api-reference/search-apis/search/) | -- |
| `search.batched_reduce_size` | Integer | The number of shard results combined into one batch on the coordinating node before the final reduction step. Lower values reduce coordinator memory usage when a search spans many shards. | [`batched_reduce_size`]({{site.url}}{{site.baseurl}}/api-reference/search-apis/search/) | -- |
| `search.max_buckets` | Integer | The maximum number of aggregation buckets allowed in a single response. Guards against excessive memory use from large aggregations. | -- | [`search.max_buckets`]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/search-settings/) |
| `override_request_values` | Boolean | Whether the workload group's settings take precedence over values supplied on the request. Default is `false`. See [Setting precedence](#setting-precedence). | -- | -- |

## Setting precedence

When a setting is defined on a workload group, OpenSearch resolves the effective value at request time using the following precedence rules:

- A workload group setting always takes precedence over the corresponding cluster setting when both are defined.
- By default, an explicit value supplied on a request takes precedence over the workload group's setting. You can reverse this behavior by setting `override_request_values` to `true`.

The following table summarizes how the effective value is resolved.

| `override_request_values` | Precedence (highest to lowest) |
| :--- | :--- |
| `false` (Default) | Request parameter > Workload group setting > Cluster setting |
| `true` | Workload group setting > Request parameter > Cluster setting |

## Creating a workload group containing settings

Add a `settings` object alongside the existing workload group fields:

```json
PUT _wlm/workload_group
{
"name": "analytics",
"resiliency_mode": "enforced",
"resource_limits": {
"cpu": 0.4,
"memory": 0.2
},
"settings": {
"search.default_search_timeout": "30s",
"search.cancel_after_time_interval": "1m",
"search.max_concurrent_shard_requests": 5,
"search.batched_reduce_size": 512,
"search.max_buckets": 10000
}
}
```
{% include copy-curl.html %}

## Updating workload group settings

You can update individual settings without affecting the other settings.

For example, to change only the search timeout for the `analytics` workload group:

```json
PUT _wlm/workload_group/analytics
{
"settings": {
"search.default_search_timeout": "1m"
}
}
```
{% include copy-curl.html %}

To remove a single setting, set its value to `null`:

```json
PUT _wlm/workload_group/analytics
{
"settings": {
"search.batched_reduce_size": null
}
}
```
{% include copy-curl.html %}

To clear all settings, send an empty `settings` object:

```json
PUT _wlm/workload_group/analytics
{
"settings": {}
}
```
{% include copy-curl.html %}

## Retrieving workload group settings

To retrieve workload group settings, use the [Workload Group API]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/workload-management/workload-groups/#retrieving-a-workload-group):

```json
GET _wlm/workload_group/analytics
```
{% include copy-curl.html %}

## Deleting workload group settings

Settings are removed when the workload group is deleted. To remove individual settings without deleting the group, see [Updating workload group settings](#updating-workload-group-settings).
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ When creating or updating a workload group, you can specify the following parame
| `name` | Create | The name of the workload group. |
| `resiliency_mode` | Create or update | The resiliency mode of the workload group. Valid values are:<br>- `enforced` (queries are rejected if thresholds are exceeded). <br>- `soft` (queries can exceed thresholds if resources are available). <br>- `monitor` (queries are monitored but not canceled or rejected). <br> **Note**: These settings take effect only if the cluster-level `wlm.workload_group.mode` setting is `enabled`. See [Operating modes]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/workload-management/wlm-feature-overview/#operating-modes). |
| `resource_limits` | Create or update | The resource limits for query requests in the workload group. Valid resources are `cpu` and `memory`. When creating a workload group, make sure that the sum of the resource limits for a single resource, either `cpu` or `memory`, does not exceed 1. |
| `settings` | Create or update | Group-specific settings that are applied automatically to requests routed to the workload group. For supported settings and update behavior, see [Workload group settings]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/workload-management/workload-group-settings/). |

## Updating a workload group

Expand All @@ -70,11 +71,16 @@ PUT _wlm/workload_group/analytics
"resource_limits": {
"cpu": 0.41,
"memory": 0.21
},
"settings": {
"search.default_search_timeout": "1m"
}
}
```
{% include copy-curl.html %}

For more information about the `settings` field, including supported settings and update behavior, see [Workload group settings]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/workload-management/workload-group-settings/).

## Retrieving a workload group

To retrieve all workload groups, use the following request:
Expand Down
Loading