Skip to content

Add challenge to the elastic/log track which upgrades a data stream#755

Merged
parkertimmins merged 13 commits intomasterfrom
reindex-perf-test
Mar 19, 2025
Merged

Add challenge to the elastic/log track which upgrades a data stream#755
parkertimmins merged 13 commits intomasterfrom
reindex-perf-test

Conversation

@parkertimmins
Copy link
Contributor

Add a new challenge which test the performance of upgrading a data stream using the reindex data stream endpoints.

This challenge does the following:

  • restores a data stream from a snapshot
  • starts a data stream upgrade with a call to /_migration/reindex
  • waits for the upgrade to complete, polling /_migration/reindex/<data-stream>/_status once a second

The following upgrade specific parameters can be set. These describe the settings described here.

  • reindex_max_concurrent_indices
  • reindex_max_requests_per_second

The only data stream indices which (currently) require upgrades are 7.x indices in version 8 ES, in preparation for an upgrade to 9.x. To obtain a data stream with 7.x indices we load the data stream from a snapshot. This challenge reuses the existing snapshot parameters used by the logging-snapshot-restore challenge.
The following snapshot parameters should be set:

  "restore_data_streams"
  "snapshot_repo_name"
  "snapshot_repo_type"
  "snapshot_repo_settings"
  "snapshot_name"
  "snapshot_rename_suffix"

Co-authored-by: Jason Bryan <jbryan@elastic.co>
Copy link
Member

@inqueue inqueue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, nice work

@parkertimmins parkertimmins merged commit 1cf951b into master Mar 19, 2025
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Comments