Skip to content
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
224 changes: 224 additions & 0 deletions _dashboards/admin-ui-index/index-management.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,224 @@
---
layout: default
title: Index management
parent: Admin UI for index operations in OpenSearch Dashboards
Comment thread
ariamarble marked this conversation as resolved.
nav_order: 10
---

# Index management
Introduced 2.5
{: .label .label-purple }

The Index Management section in the admin UI allows you to perform the operations available in the [Index API]({{site.url}}{{site.baseurl}}/api-reference/index-apis/index/) from OpenSearch Dashboards' web UI.
Comment thread
ariamarble marked this conversation as resolved.

## Index policies

[Policies]({{site.url}}{{site.baseurl}}/im-plugin/ism/policies/) are configurations that define the possible states of an index, the actions to perform when an index enters a state, and the conditions that must be met to transition between states:
Comment thread
ariamarble marked this conversation as resolved.

1. **States**: The possible states of an index, including the default state for new indexes. For example, you might name your states `hot`, `warm`, or `delete`. For more information, see [States]({{site.url}}{{site.baseurl}}/im-plugin/ism/policies/#states).
2. **Actions**: Any actions that you want the plugin to take when an index enters a state, such as performing a rollover. For more information, see [Actions]({{site.url}}{{site.baseurl}}/im-plugin/ism/policies/#actions).
Comment thread
ariamarble marked this conversation as resolved.
3. **Transitions**: The conditions that must be met for an index to move into a new state. For example, if an index is more than eight weeks old, you might want to move it to the `delete` state. For more information, see [Transitions]({{site.url}}{{site.baseurl}}/im-plugin/ism/policies/#transitions).
Comment thread
ariamarble marked this conversation as resolved.

You can also upload a JSON document to specify an index policy.
{: .note}

You have complete flexibility in the way you can design your policies. You can create any state, transition to any other state, and specify any number of actions in each state.
Comment thread
ariamarble marked this conversation as resolved.

To attach policies to indices, perform the following steps:
Comment thread
ariamarble marked this conversation as resolved.

1. Under **Index Management**, select **Index policies**.
Comment thread
ariamarble marked this conversation as resolved.
2. Choose the index or indexes to which you want to attach your policy.
Comment thread
ariamarble marked this conversation as resolved.
3. Choose **Apply policy**.
4. From the **Policy ID** menu, choose the policy that you created.
Comment thread
ariamarble marked this conversation as resolved.
View the preview of your policy.
5. (Optional): Specify a rollover alias if your policy includes a rollover operation. Make sure that the alias that you enter already exists. For more information about the rollover operation, see [rollover]({{site.url}}{{site.baseurl}}/im-plugin/ism/policies#rollover).
Comment thread
ariamarble marked this conversation as resolved.
6. Choose **Apply**.
Comment thread
ariamarble marked this conversation as resolved.
Outdated
After you attach a policy to an index, ISM creates a job that runs every 5 minutes by default to perform policy actions, check conditions, and transition the index into different states. To change the default time interval for this job, see Settings.
Comment thread
ariamarble marked this conversation as resolved.

Policy jobs don't run if the cluster state is red.
{: .note}

## Managed indexes

To attach policies to indexes, perform the following steps:

1. Under **Index Management**, select **Manage Indices**.
Comment thread
ariamarble marked this conversation as resolved.
2. Choose the index or indexes to which you want to attach your policy.
Comment thread
ariamarble marked this conversation as resolved.
3. Select the **Change policy** button.
Comment thread
ariamarble marked this conversation as resolved.
4. Choose **Apply policy**.

## Indexes

The **Indices** section displays a list of indexes in your OpenSearch cluster. For each index including, you can see its heath status (`green`, `yellow`, or `red`), policy (if the index is managed by a policy), status, total size, size of primaries, total documents, deleted documents, primaries and replicas.

#### Create index
Comment thread
ariamarble marked this conversation as resolved.

While you can [create an index]({{site.url}}{{site.baseurl}}/api-reference/index-apis/create-index/) by using a document as a base, you can also create an empty index for later use.

To create an index, select the **Create Index** button located under the **Indices** section of **Index Management**. Then define the index by setting the following parameters:

- Index name
- Number of primary shards
- Number of replicas
- Refresh interval
You can also add fields and objects using either the visual editor or the JSON editor.

**Advanced settings** allows you to upload a JSON configuration.
Comment thread
ariamarble marked this conversation as resolved.

#### Apply policy
Comment thread
ariamarble marked this conversation as resolved.

If you analyze time series data, you likely want to prioritize new data over old data. You might periodically perform certain operations on older indexes, such as reducing replica count or deleting them.
Comment thread
ariamarble marked this conversation as resolved.

[Index State Management]({{site.url}}{{site.baseurl}}/im-plugin/ism/index/) (ISM) is a plugin that lets you automate these periodic administrative operations by triggering them based on changes in the index age, index size, or number of documents. The admin UI uses the ISM plugin to define policies that automatically handle index rollovers or deletions to fit your use case.
Comment thread
ariamarble marked this conversation as resolved.

For example, you can define a policy that moves your index into a **read_only** state after 30 days and then deletes it after a set period of 90 days. You can also set up the policy to send you a notification message when the index is deleted.

You might want to perform an index rollover after a certain amount of time or run a **force_merge** operation on an index during off-peak hours to improve search performance during peak hours.

To apply a policy, select the index to which you want to apply the policy in the **Indices** list under **Index Management**. Then select the **Actions** button, and select **Apply policy** from the dropdown list.
Comment thread
ariamarble marked this conversation as resolved.

<img src="{{site.url}}{{site.baseurl}}/images/admin-ui-index/apply-policy.PNG" alt="User interface showing apply policy prompt">
Comment thread
ariamarble marked this conversation as resolved.

#### Close
Comment thread
ariamarble marked this conversation as resolved.

The [close index]({{site.url}}{{site.baseurl}}/api-reference/index-apis/close-index/) operation closes an index. Once an index is closed, you cannot add data to it or search for any data within the index.

To close an index, select the index you want to close in the **Indices** list under **Index Management**. Then select the **Actions** button, and select **Close** from the dropdown list.
Comment thread
ariamarble marked this conversation as resolved.

#### Open
Comment thread
ariamarble marked this conversation as resolved.

The [open index]({{site.url}}{{site.baseurl}}/api-reference/index-apis/open-index/) operation opens a closed index, letting you add or search for data within the index.
Comment thread
ariamarble marked this conversation as resolved.

To open an index, select the index you want to open in the **Indices** list under **Index Management**. Then select the **Actions** button, and select **Open** from the dropdown list.
Comment thread
ariamarble marked this conversation as resolved.

#### Reindex
Comment thread
ariamarble marked this conversation as resolved.

The [reindex]({{site.url}}{{site.baseurl}}/api-reference/document-apis/reindex/) operation lets you copy all your data or a subset of data from a source index into a destination index.
Comment thread
ariamarble marked this conversation as resolved.

To reindex an index, select the index in the **Indices** list under **Index Management**. Then select the **Actions** button, and select **Reindex** from the dropdown list.
Comment thread
ariamarble marked this conversation as resolved.

<img src="{{site.url}}{{site.baseurl}}/images/admin-ui-index/reindex-expanded.png" alt="User interface showing reindex prompt">
Comment thread
ariamarble marked this conversation as resolved.

#### Shrink
Comment thread
ariamarble marked this conversation as resolved.

The [shrink]({{site.url}}{{site.baseurl}}/api-reference/index-apis/shrink-index/) index operation moves all of your data in an existing index into a new index with fewer primary shards.
Comment thread
ariamarble marked this conversation as resolved.

To shrink an index, select the index you want to shrink in the **Indices** list under **Index Management**. Then select the **Actions** button, and select **Shrink** from the dropdown list.
Comment thread
ariamarble marked this conversation as resolved.

<img src="{{site.url}}{{site.baseurl}}/images/admin-ui-index/shrink.png" alt="User interface showing shrink prompt">
Comment thread
ariamarble marked this conversation as resolved.

#### Split
Comment thread
ariamarble marked this conversation as resolved.

The [split index]({{site.url}}{{site.baseurl}}/api-reference/index-apis/split/) operation splits an existing read-only index into a new index, splitting each primary shard into a number of primary shards in the new index.

To split an index, select the index you want to split in the **Indices** list under **Index Management**. Then select the **Actions** button, and select **Split** from the dropdown list.
Comment thread
ariamarble marked this conversation as resolved.

<img src="{{site.url}}{{site.baseurl}}/images/admin-ui-index/split-expanded.png" alt="User interface showing split page">
Comment thread
ariamarble marked this conversation as resolved.

#### Delete
Comment thread
ariamarble marked this conversation as resolved.

If you no longer need an index, you can use the [delete index]({{site.url}}{{site.baseurl}}/api-reference/index-apis/delete-index/) operation to delete it.

To delete an index, select the index you want to delete in the **Indices** list under **Index Management**. Then select the **Actions** button, and select **Delete** from the dropdown list.
Comment thread
ariamarble marked this conversation as resolved.

## Templates

[Index templates]({{site.url}}{{site.baseurl}}/opensearch/index-templates/) let you initialize new indexes with predefined mappings and settings. For example, if you continuously index log data, you can define an index template so that all of these indexes have the same number of shards and replicas.
Comment thread
ariamarble marked this conversation as resolved.

<img src="{{site.url}}{{site.baseurl}}/images/admin-ui-index/templates.PNG" alt="User interface showing Templates page">
Comment thread
ariamarble marked this conversation as resolved.

To create a template, select the **Create template** button on the **Templates** page under **Index Management**.
Comment thread
ariamarble marked this conversation as resolved.

Next, define the template:

1. Enter the template name.
2. Select the template type.
3. Specify any index patterns you would like to use.
4. Set the priority of the template.
5. Select an index alias.
6. Set the number of primary shards.
7. Set the number of replicas.
8. Set the refresh intervals.
9. Add fields and objects for your index mapping using either the visual editor or the JSON editor.

<img src="{{site.url}}{{site.baseurl}}/images/admin-ui-index/create-template-expanded.png" alt="User interface showing Create Template page">
Comment thread
ariamarble marked this conversation as resolved.

## Aliases

An alias is a virtual index name that can point to one or more indexes. If your data is spread across multiple indexes, rather than keeping track of which indexes to query, you can create an alias and query it instead.

<img src="{{site.url}}{{site.baseurl}}/images/admin-ui-index/aliases.PNG" alt="User interface showing Alias page">
Comment thread
ariamarble marked this conversation as resolved.

To create an alias, perform the following steps:

1. Select the **Create Alias** button on the **Aliases** page under **Index Management**.
Comment thread
ariamarble marked this conversation as resolved.
2. Specify the alias name.
3. Enter the index, or index patterns, to be part of the alias.
Comment thread
ariamarble marked this conversation as resolved.
4. Select **Create alias**.
Comment thread
ariamarble marked this conversation as resolved.

<img src="{{site.url}}{{site.baseurl}}/images/admin-ui-index/create-alias.PNG" alt="User interface showing creat Alias page">
Comment thread
ariamarble marked this conversation as resolved.

To edit an alias, perform the following steps:

1. Select the alias you want to edit.
2. Select the **Actions** button.
Comment thread
ariamarble marked this conversation as resolved.
3. Select **Edit** from the dropdown list.
Comment thread
ariamarble marked this conversation as resolved.

To delete an alias, perform the following steps:

1. Select the alias you want to edit.
2. Select the **Actions** button.
Comment thread
ariamarble marked this conversation as resolved.
3. Select **Delete** from the dropdown list.
Comment thread
ariamarble marked this conversation as resolved.

## Rollup jobs

The **Rollup Jobs** section under **Index Management** allows you to create or update index rollup jobs.

To create a rollup job, perform the following steps:

1. Select the **Create rollup job** button on the **Rollup Jobs** page under **Index Management**.
Comment thread
ariamarble marked this conversation as resolved.
2. Set the name, source index, and target index.
3. Select **Next**.
Comment thread
ariamarble marked this conversation as resolved.
4. Set the timestamp field and interval type.
5. Optionally, set additional aggregations and additional metrics.
Comment thread
ariamarble marked this conversation as resolved.
6. Select **Next**.
Comment thread
ariamarble marked this conversation as resolved.
7. Under **Schedule**, check or uncheck **Enable job by default**.
8. Set the **Continuous**, **Execution frequency**, **Rollup interval**, and **Pages per execution** settings.
9. Additionally, you can set an execution delay.
10. Select **Next**.
Comment thread
ariamarble marked this conversation as resolved.
11. Review the settings for the rollup job and select **Create**.
Comment thread
ariamarble marked this conversation as resolved.

You can also disable and enable rollup jobs by selecting the corresponding buttons on the **Rollup Jobs** page.
Comment thread
ariamarble marked this conversation as resolved.

## Transform jobs

Using the admin UI in OpenSearch Dashboards, you can create, start, stop, and complete operations with [transform]({{site.url}}{{site.baseurl}}/im-plugin/index-transforms/transforms-apis/) jobs.

To create a transform job, perform the following steps:

1. Select the **Create transform job** button on the **Transform Jobs** page under **Index Management**.
Comment thread
ariamarble marked this conversation as resolved.
2. Set the name, source index, and target index.
3. Select **Next**.
Comment thread
ariamarble marked this conversation as resolved.
4. Select the fields to transform. From the table above, select a field you want to transform by clicking **+** next to the field name.
Comment thread
ariamarble marked this conversation as resolved.
5. Select **Next**.
Comment thread
ariamarble marked this conversation as resolved.
6. Check or uncheck **Job enabled by default**.
7. Set wheter the schedule is continuous and the transform execution interval.
8. Optionally, set pages per execution under the **Advanced** dropdown list.
9. Select **Next**.
Comment thread
ariamarble marked this conversation as resolved.
10. Review the settings for the rollup job and select **Create**.
Comment thread
ariamarble marked this conversation as resolved.

You can also disable and enable rollup jobs by selecting the corresponding buttons on the **Transform Jobs** page.
Comment thread
ariamarble marked this conversation as resolved.

## Long running operation status check
Comment thread
ariamarble marked this conversation as resolved.

Certain index operations take time to complete (usually more than 30 seconds, up to tens of minutes or hours). This is tracked on the index status on **Indices** page.
Comment thread
ariamarble marked this conversation as resolved.

You can check the status of the reindex, shrink, and split operations because they are one-time non-recursive operations.
Comment thread
ariamarble marked this conversation as resolved.

## Security integration

Permission control is managed with existing [permissions]({{site.url}}{{site.baseurl}}/security-plugin/access-control/permissions/) or action groups that are enforced at the API level. There is currently no UI-level permission control. Users with permissions to access the ISM plugin are able to view new pages. They can also make changes if they have permissions to run the related APIs.
Comment thread
ariamarble marked this conversation as resolved.

## Error handling

Similar to API calls, if the operation fails immediately, you will be notified with an error message. However, if it is a long-running operation, you will be notified of the failure at the time of failure, or you can check the index status on the **Indices** page.
16 changes: 16 additions & 0 deletions _dashboards/admin-ui-index/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
layout: default
title: Admin UI for index operations in OpenSearch Dashboards
nav_order: 1
has_children: true
redirect_from:
- /dashboards/admin-ui-index/
---

# Admin UI for index operations in OpenSearch Dashboards
Introduced 2.5
{: .label .label-purple }

The admin user interface (UI) for index operation allows you to perform common index operations through a user-friendly interface and prevent unintentional changes.
Comment thread
ariamarble marked this conversation as resolved.

The admin UI interface allows you to perform basic cluster administrative operations, or to view cluster health metrics and status. While there have been some administration user experiences in OpenSearch Dashboards (ISM, Alerting on HTTP inputs, Security, etc.), there is now a unified web UI experience for cluster administrators to administer their deployments.
Comment thread
ariamarble marked this conversation as resolved.
Original file line number Diff line number Diff line change
@@ -1,15 +1,30 @@
---
layout: default
title: Snapshot Management in OpenSearch Dashboards
parent: Snapshots
nav_order: 35
has_children: false
title: Using Snapshot Management in OpenSearch Dashboards
parent: Admin UI for index operations in OpenSearch Dashboards
nav_order: 30
---

# Using Snapshot Management in OpenSearch Dashboards

You can set up Snapshot Management (SM) in OpenSearch Dashboards.

Snapshots]({{site.url}}{{site.baseurl}}/opensearch/snapshots/index/) are backups of a cluster’s indices and state. The state includes cluster settings, node information, index metadata (mappings, settings, templates, etc.), and shard allocation.
Comment thread
ariamarble marked this conversation as resolved.
Outdated

Snapshots have two main uses:

1. Recovering from failure

For example, if cluster health goes red, you might restore the red indexes from a snapshot.

2. Migrating from one cluster to another

For example, if you’re moving from a proof-of-concept to a production cluster, you might take a snapshot of the former and restore it on the latter.
Comment thread
ariamarble marked this conversation as resolved.

You can take and restore snapshots using the admin UI in Dashboards.

If you need to automate taking snapshots, you can use a Snapshot policy.
Comment thread
ariamarble marked this conversation as resolved.
Outdated

## Create a repository
Comment thread
ariamarble marked this conversation as resolved.

Before you create an SM policy, you need to set up a repository for snapshots.
Expand All @@ -30,6 +45,10 @@ Before you create an SM policy, you need to set up a repository for snapshots.
```
1. Select the **Add** button.
Comment thread
ariamarble marked this conversation as resolved.

#### Delete a repository
Comment thread
ariamarble marked this conversation as resolved.

To delete a snapshot repository configuration, select the repository in the **Repositories** list and select the **Delete** button above.
Comment thread
ariamarble marked this conversation as resolved.
Outdated

## Create an SM policy
Comment thread
ariamarble marked this conversation as resolved.

Create an SM policy to set up automatic snapshots. An SM policy defines an automated snapshot creation schedule and an optional automated deletion schedule.
Expand Down Expand Up @@ -146,4 +165,11 @@ Use the steps below to take a snapshot manually:

After the restore operation is complete, the restored indexes are listed in the **Indices** panel. To view the indexes, in the left panel, under **Index Management**, select **Indices**.
Comment thread
ariamarble marked this conversation as resolved.

<img src="{{site.url}}{{site.baseurl}}/images/restore-snapshot/restore-snapshot-indices-panel.png" alt="View Indices">{: .img-fluid}
<img src="{{site.url}}{{site.baseurl}}/images/restore-snapshot/restore-snapshot-indices-panel.png" alt="View Indices">{: .img-fluid}

#### Delete a snapshot
Comment thread
ariamarble marked this conversation as resolved.
Outdated

[Deletes]({{site.url}}{{site.baseurl}}/api-reference/snapshots/delete-snapshot/) a snapshot from a repository.
Comment thread
ariamarble marked this conversation as resolved.
Outdated

1. To view a list of your repositories, select **Repositories** under the **Snapshot Management** section.
2. To view a list of your snapshots, select **Snapshots** under the **Snapshot Management** section.
Binary file added images/admin-ui-index/aliases.PNG
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/admin-ui-index/apply-policy.PNG
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/admin-ui-index/create-alias.PNG
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/admin-ui-index/create-template.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/admin-ui-index/reindex-expanded.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/admin-ui-index/reindex.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/admin-ui-index/shrink.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/admin-ui-index/split-expanded.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/admin-ui-index/split.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/admin-ui-index/templates.PNG
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/admin-ui-index/view-template.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.