Add QA test module for Lucene N-2 version#118363
Conversation
tlrx
added
>test
elasticsearchmachine
added
the
Team:Distributed Indexing (obsolete)
|
Pinging @elastic/es-distributed-indexing (Team:Distributed Indexing) |
|
This reminded me of a similar effort for archive indices in https://github.com/elastic/elasticsearch/pull/116565/files#diff-0127cb27f9161cd76ae90b86b80ee56bad117effcf0bcdcc5941af2a9dc32543 . At this point, I don't expect that PR to get merged. Thanks a lot for working on this! I see how we could use this as a base to add more tests, similar to what we do for archive indices, but more, for instance test specific mappings compatibility etc. |
build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/BwcVersions.java
Outdated
Show resolved
Hide resolved
...bility/src/javaRestTest/java/org/elasticsearch/lucene/SearchableSnapshotCompatibilityIT.java
Outdated
Show resolved
Hide resolved
| logger.fatal("--> mounting index [{}] as [{}]", index, mountedIndex); | ||
|
|
||
| // Mounting the index will fail as Elasticsearch does not support reading N-2 yet | ||
| var request = new Request("POST", "/_snapshot/" + repository + "/" + snapshot + "/_mount"); |
There was a problem hiding this comment.
Just a comment for later that we should also add a test where we mount the N-2 index on N-1 and then search it on N.
| import static org.hamcrest.Matchers.is; | ||
| import static org.hamcrest.Matchers.notNullValue; | ||
|
|
||
| public class ArchiveCompatibilityIT extends AbstractLuceneIndexCompatibilityTestCase { |
There was a problem hiding this comment.
Can we call this LuceneCompatibilityIT instead? I think of it as different from archive indices.
There was a problem hiding this comment.
Sure. I also renamed testRestoreArchive to testRestoreIndex
| } | ||
|
|
||
| public List<Version> getReadOnlyIndexCompatible() { | ||
| // Lucene can read indices in version N-2 |
There was a problem hiding this comment.
I wonder if we should assert that N-2 == 7? We definitely want to utiliize this for the V9 upgrade, but we are not fully committed to necessarily having N-2 support in V10 (not saying we won't either).
There was a problem hiding this comment.
I added an explicit check with a comment in the test suite. I think it achieves the same goal, while avoiding it to be in the build logic.
Relates ES-10274
tlrx
force-pushed
the
2024/12/10/ES-10274-BWC-test
branch
from
a37da6b to
3c2041c
Compare
|
@breskeby hope you have the time to review the few changes in the BwcVersion class 🙏🏻 |
|
@tlrx @mark-vieira just had touched this file a few days ago and made some changes. I'll delegate to him for having a look at this |
| tasks.named("javaRestTest").configure { | ||
| systemProperty("tests.minimum.index.compatible", bwcVersion) | ||
| usesBwcDistribution(bwcVersion) | ||
| enabled = true |
There was a problem hiding this comment.
why would you need explicitly enable = true here?
There was a problem hiding this comment.
For some reason it complained that it was disabled by default.
There was a problem hiding this comment.
ah yes, bwc-test plugin by default disables this task
| tasks.named("javaRestTest").configure { | ||
| systemProperty("tests.minimum.index.compatible", bwcVersion) | ||
| usesBwcDistribution(bwcVersion) | ||
| enabled = true |
There was a problem hiding this comment.
ah yes, bwc-test plugin by default disables this task
|
Thanks all! |
5 participants
This pull request introduces a new QA project to test Lucene support for reading indices created in version N-2.
The test suite is inspired from the various full-cluster restart suites we already have. It creates a cluster in version N-2 (today 7.17.25), then upgrades the cluster to N-1 (today 8.18.0) and finally upgrades the cluster to the current version (today 9.0), allowing to execute test methods after every upgrade.
The test suite has two variants: one for searchable snapshots and one for snapshot restore. The suites demonstrates that Elasticsearch does not allow reading indices written in version N-2 but we hope to make this feasible. Also, the tests can be used for investigation and debug (launched with
--debug-server-jvm).Relates ES-10274