Fix 1.x compatibility bug with stored Tasks by andrross · Pull Request #5412 · opensearch-project/OpenSearch

andrross · 2022-11-30T18:53:49Z

When the new 'cancelled' field was introduced it was a miss not to increment the version number on the mapping definitions for the .tasks index. This commit fixes that oversight, as well as modifies the existing backward compatiblity test to ensure that it will catch future mistakes like this one.

Closes #5376

Signed-off-by: Andrew Ross andrross@amazon.com

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff
Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

When the new 'cancelled' field was introduced it was a miss not to increment the version number on the mapping definitions for the .tasks index. This commit fixes that oversight, as well as modifies the existing backward compatiblity test to ensure that it will catch future mistakes like this one. Closes opensearch-project#5376 Signed-off-by: Andrew Ross <andrross@amazon.com>

reta · 2022-11-30T19:12:10Z

+     */
+    private static void createAndVerifyStoredTask() throws Exception {
+        // Use update by query to create an async task
+        final Request updateByQueryRequest = new Request("POST", "/test_index_old/_update_by_query");


Just curious, _reindex was replaced with _update_by_query, is there some particular reason?

Because it is done twice, once for the old cluster and again for the updated cluster. Reindex creates a new index so you can't make the exact same request twice (unless you closed or deleted the new index). Update by query is simpler because it takes no parameters and doesn't create any new resources :)

(Previously this step was only being done on the old cluster, so there was no problem. My change does this test for both the old and updated clusters.)

github-actions · 2022-11-30T19:22:14Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/7452/
CommitID: d674d14

github-actions · 2022-11-30T19:30:39Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      1 org.opensearch.index.IndexServiceTests.testAsyncTranslogTrimTaskOnClosedIndex

URL: https://build.ci.opensearch.org/job/gradle-check/7453/
CommitID: 319aecc
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

dreamer-89

LGTM! Thanks @andrross for this quick fix.

dreamer-89 · 2022-11-30T19:45:30Z

      1 org.opensearch.index.IndexServiceTests.testAsyncTranslogTrimTaskOnClosedIndex

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕

TEST FAILURES:
      1 org.opensearch.index.IndexServiceTests.testAsyncTranslogTrimTaskOnClosedIndex
URL: https://build.ci.opensearch.org/job/gradle-check/7453/

CommitID: 319aecc
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Created #5414 to track this.

opensearch-trigger-bot · 2022-11-30T19:48:38Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-5412-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 4616dfac5382062a5149c799554dda4ff601369f
# Push it to GitHub
git push --set-upstream origin backport/backport-5412-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-5412-to-2.x.

When the new 'cancelled' field was introduced it was a miss not to increment the version number on the mapping definitions for the .tasks index. This commit fixes that oversight, as well as modifies the existing backward compatiblity test to ensure that it will catch future mistakes like this one. Closes opensearch-project#5376 Signed-off-by: Andrew Ross <andrross@amazon.com> (cherry picked from commit 4616dfa) Signed-off-by: Andrew Ross <andrross@amazon.com>

When the new 'cancelled' field was introduced it was a miss not to increment the version number on the mapping definitions for the .tasks index. This commit fixes that oversight, as well as modifies the existing backward compatiblity test to ensure that it will catch future mistakes like this one. Closes #5376 Signed-off-by: Andrew Ross <andrross@amazon.com> (cherry picked from commit 4616dfa) Signed-off-by: Andrew Ross <andrross@amazon.com>

dblock · 2022-12-01T15:10:38Z

👏

opensearch-trigger-bot · 2022-12-03T00:41:16Z

The backport to 2.4 failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.4 2.4
# Navigate to the new working tree
pushd ../.worktrees/backport-2.4
# Create a new branch
git switch --create backport/backport-5412-to-2.4
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 4616dfac5382062a5149c799554dda4ff601369f
# Push it to GitHub
git push --set-upstream origin backport/backport-5412-to-2.4
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.4

Then, create a pull request where the base branch is 2.4 and the compare/head branch is backport/backport-5412-to-2.4.

When the new 'cancelled' field was introduced it was a miss not to increment the version number on the mapping definitions for the .tasks index. This commit fixes that oversight, as well as modifies the existing backward compatiblity test to ensure that it will catch future mistakes like this one. Closes opensearch-project#5376 Signed-off-by: Andrew Ross <andrross@amazon.com> (cherry picked from commit 4616dfa)

When the new 'cancelled' field was introduced it was a miss not to increment the version number on the mapping definitions for the .tasks index. This commit fixes that oversight, as well as modifies the existing backward compatiblity test to ensure that it will catch future mistakes like this one. Closes opensearch-project#5376 Signed-off-by: Andrew Ross <andrross@amazon.com> (cherry picked from commit 4616dfa) Signed-off-by: Andrew Ross <andrross@amazon.com>

When the new 'cancelled' field was introduced it was a miss not to increment the version number on the mapping definitions for the .tasks index. This commit fixes that oversight, as well as modifies the existing backward compatiblity test to ensure that it will catch future mistakes like this one. Closes #5376 Signed-off-by: Andrew Ross <andrross@amazon.com> (cherry picked from commit 4616dfa) Signed-off-by: Andrew Ross <andrross@amazon.com> Signed-off-by: Andrew Ross <andrross@amazon.com>

andrross force-pushed the task-mapping-fix branch from d674d14 to 319aecc Compare November 30, 2022 18:55

andrross changed the title ~~Fix 1.x compatiblity bug with stored Tasks~~ Fix 1.x compatibility bug with stored Tasks Nov 30, 2022

andrross marked this pull request as ready for review November 30, 2022 18:57

andrross requested review from a team and reta as code owners November 30, 2022 18:57

andrross mentioned this pull request Nov 30, 2022

Upgrade the mappings on .tasks index if needed #5377

Closed

6 tasks

reta reviewed Nov 30, 2022

View reviewed changes

reta approved these changes Nov 30, 2022

View reviewed changes

dreamer-89 approved these changes Nov 30, 2022

View reviewed changes

dreamer-89 mentioned this pull request Nov 30, 2022

[BUG] org.opensearch.index.IndexServiceTests.testAsyncTranslogTrimTaskOnClosedIndex flaky #5414

Closed

andrross merged commit 4616dfa into opensearch-project:main Nov 30, 2022

andrross added the backport 2.x Backport to 2.x branch label Nov 30, 2022

andrross added the backport 2.4 Backport to 2.4 branch label Dec 3, 2022

andrross deleted the task-mapping-fix branch December 16, 2022 18:35

ketanv3 mentioned this pull request Jan 10, 2023

[BUG] Tasks not added to .tasks after upgrade from 1.3 to 2.3 #5708

Closed

gaobinlong mentioned this pull request Oct 12, 2023

[BUG] Tasks API will no longer be able to store results of tasks in .tasks index after upgrade from ES 7.10 to OS 2.3 #10240

Closed

Navigation Menu

Uh oh!

Fix 1.x compatibility bug with stored Tasks - #5412

Fix 1.x compatibility bug with stored Tasks#5412
andrross merged 1 commit into
opensearch-project:mainfrom
andrross:task-mapping-fix

andrross commented Nov 30, 2022

Uh oh!

reta Nov 30, 2022

Uh oh!

andrross Nov 30, 2022

Uh oh!

andrross Nov 30, 2022

Uh oh!

github-actions Bot commented Nov 30, 2022

Uh oh!

github-actions Bot commented Nov 30, 2022

Uh oh!

dreamer-89 left a comment

Uh oh!

dreamer-89 commented Nov 30, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

opensearch-trigger-bot Bot commented Nov 30, 2022

Uh oh!

dblock commented Dec 1, 2022

Uh oh!

opensearch-trigger-bot Bot commented Dec 3, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

andrross commented Nov 30, 2022

Check List

Uh oh!

reta Nov 30, 2022

Choose a reason for hiding this comment

Uh oh!

andrross Nov 30, 2022

Choose a reason for hiding this comment

Uh oh!

andrross Nov 30, 2022

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Nov 30, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions Bot commented Nov 30, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

dreamer-89 left a comment

Choose a reason for hiding this comment

Uh oh!

dreamer-89 commented Nov 30, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

opensearch-trigger-bot Bot commented Nov 30, 2022

Uh oh!

dblock commented Dec 1, 2022

Uh oh!

opensearch-trigger-bot Bot commented Dec 3, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants