Skip to content

[Refactor] InternalEngine to always use soft deletes - #1933

Merged
nknize merged 7 commits into
opensearch-project:mainfrom
nknize:refactor/EngineAlwaysUsesSoftDeletes
Jan 26, 2022
Merged

[Refactor] InternalEngine to always use soft deletes#1933
nknize merged 7 commits into
opensearch-project:mainfrom
nknize:refactor/EngineAlwaysUsesSoftDeletes

Conversation

@nknize

@nknize nknize commented Jan 18, 2022

Copy link
Copy Markdown
Contributor

Soft Deletes have been enabled by default since Legacy version 7.0 and made
mandatory in Version 2.0.0. This PR refactors the InternalEngine to always
use soft-deletes. It is a follow on to making soft deletes mandatory in 2.0.0.

@nknize nknize added enhancement Enhancement or improvement to existing feature or request v2.0.0 Version 2.0.0 Indexing & Search labels Jan 18, 2022
@nknize
nknize requested a review from a team as a code owner January 18, 2022 21:01
@opensearch-ci-bot

opensearch-ci-bot commented Jan 18, 2022

Copy link
Copy Markdown
Contributor

Can one of the admins verify this patch?

@opensearch-ci-bot

opensearch-ci-bot commented Jan 18, 2022

Copy link
Copy Markdown
Contributor

❌   Gradle Check failure bcfa7a9f598b5a9809e58d6f5328c51ae209128f
Log 1974

Reports 1974

@opensearch-ci-bot

opensearch-ci-bot commented Jan 18, 2022

Copy link
Copy Markdown
Contributor

❌   Gradle Check failure a59fd51caa159a30a92aeb77c7daf57d26089ee5
Log 1976

Reports 1976

nknize added 3 commits January 25, 2022 10:02
@nknize
[Refactor] InternalEngine to always use soft deletes
be3b5b1 
Soft Deletes have been enabled by default since Legacy version 7.0 and made
mandatory in Version 2.0.0. This commit refactors the InternalEngine to always
use soft-deletes. It is a follow on to making soft deletes mandatory in 2.0.0.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
@nknize
spotless
a1b6ebc 
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
@nknize
fix failing tests add random w/o source testing
4665aa2 
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
@nknize
nknize force-pushed the refactor/EngineAlwaysUsesSoftDeletes branch from a59fd51 to 4665aa2 Compare January 25, 2022 16:43
@opensearch-ci-bot

opensearch-ci-bot commented Jan 25, 2022

Copy link
Copy Markdown
Contributor

❌   Gradle Check failure 4665aa2
Log 2046

Reports 2046

@nknize
spotless
384f836 
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
@opensearch-ci-bot

opensearch-ci-bot commented Jan 25, 2022

Copy link
Copy Markdown
Contributor

❌   Gradle Check failure 384f836
Log 2047

Reports 2047

@nknize
fix failing tests
0303cdf 
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
@opensearch-ci-bot

opensearch-ci-bot commented Jan 25, 2022

Copy link
Copy Markdown
Contributor

❌   Gradle Check failure 0303cdf
Log 2048

Reports 2048

@nknize

nknize commented Jan 25, 2022

Copy link
Copy Markdown
Contributor Author

Repeat timeout failure reported in #1957

REPRODUCE WITH: ./gradlew ':server:test' --tests "org.opensearch.cluster.routing.MovePrimaryFirstTests.testClusterGreenAfterPartialRelocation" -Dtests.seed=2DE1E5F63CA0EA53 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=el-GR -Dtests.timezone=America/Yakutat -Druntime.java=17

org.opensearch.cluster.routing.MovePrimaryFirstTests > testClusterGreenAfterPartialRelocation FAILED
    java.lang.AssertionError: timed out waiting for green state
        at __randomizedtesting.SeedInfo.seed([2DE1E5F63CA0EA53:7E2B819317E49547]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.opensearch.test.OpenSearchIntegTestCase.ensureColor(OpenSearchIntegTestCase.java:986)
        at org.opensearch.test.OpenSearchIntegTestCase.ensureGreen(OpenSearchIntegTestCase.java:925)
        at org.opensearch.test.OpenSearchIntegTestCase.ensureGreen(OpenSearchIntegTestCase.java:914)
        at org.opensearch.cluster.routing.MovePrimaryFirstTests.testClusterGreenAfterPartialRelocation(MovePrimaryFirstTests.java:115)

@nknize

nknize commented Jan 25, 2022

Copy link
Copy Markdown
Contributor Author

start gradle check

@opensearch-ci-bot

opensearch-ci-bot commented Jan 25, 2022

Copy link
Copy Markdown
Contributor

✅   Gradle Check success 0303cdf
Log 2049

Reports 2049

reta
reta reviewed Jan 25, 2022
View reviewed changes
Comment thread qa/full-cluster-restart/src/test/java/org/opensearch/upgrades/FullClusterRestartIT.java Outdated
reta
reta reviewed Jan 25, 2022
View reviewed changes
Comment thread server/src/main/java/org/opensearch/index/engine/InternalEngine.java Outdated
@nknize
PR tweaks
7f04a9e 
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
@opensearch-ci-bot

opensearch-ci-bot commented Jan 25, 2022

Copy link
Copy Markdown
Contributor

✅   Gradle Check success 7f04a9e
Log 2051

Reports 2051

andrross
andrross approved these changes Jan 25, 2022
View reviewed changes
Comment thread qa/full-cluster-restart/src/test/java/org/opensearch/upgrades/FullClusterRestartIT.java Outdated
@nknize
revert random source testing
165ef98 
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
@opensearch-ci-bot

opensearch-ci-bot commented Jan 26, 2022

Copy link
Copy Markdown
Contributor

❌   Gradle Check failure 165ef98
Log 2057

Reports 2057

@nknize

nknize commented Jan 26, 2022

Copy link
Copy Markdown
Contributor Author

Another #1908 failure

REPRODUCE WITH: ./gradlew ':server:internalClusterTest' --tests "org.opensearch.gateway.QuorumGatewayIT.testQuorumRecovery" -Dtests.seed=DBB99468CB97443F -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=sr-Latn-ME -Dtests.timezone=Atlantic/Stanley -Druntime.java=17

org.opensearch.gateway.QuorumGatewayIT > testQuorumRecovery FAILED
    java.lang.AssertionError: timed out waiting for green state
        at __randomizedtesting.SeedInfo.seed([DBB99468CB97443F:3E86A8BB6AEA9324]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.opensearch.test.OpenSearchIntegTestCase.ensureColor(OpenSearchIntegTestCase.java:986)
        at org.opensearch.test.OpenSearchIntegTestCase.ensureGreen(OpenSearchIntegTestCase.java:925)
        at org.opensearch.test.OpenSearchIntegTestCase.ensureGreen(OpenSearchIntegTestCase.java:914)
        at org.opensearch.gateway.QuorumGatewayIT.testQuorumRecovery(QuorumGatewayIT.java:111)

@nknize

nknize commented Jan 26, 2022

Copy link
Copy Markdown
Contributor Author

start gradle check

@opensearch-ci-bot

opensearch-ci-bot commented Jan 26, 2022

Copy link
Copy Markdown
Contributor

✅   Gradle Check success 165ef98
Log 2059

Reports 2059

@nknize
nknize merged commit 57ac788 into opensearch-project:main Jan 26, 2022
@nknize nknize mentioned this pull request Feb 9, 2022
Merged
@luyuncheng luyuncheng mentioned this pull request Apr 23, 2025
Open
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@reta reta reta approved these changes
@andrross andrross andrross approved these changes

Assignees

No one assigned

Labels

enhancement Enhancement or improvement to existing feature or request Indexing & Search v2.0.0 Version 2.0.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@nknize @opensearch-ci-bot @reta @andrross