Skip to content

Fix standard filter BWC check to allow for cacheing bug #62649

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Sep 21, 2020
Merged

Fix standard filter BWC check to allow for cacheing bug #62649

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7a90048
Fix standard filter BWC check to allow for cacheing bug
romseygeek Sep 18, 2020
2f41c46
add comment
romseygeek Sep 20, 2020
bb3e110
Merge branch 'master' into bug/standard-tokenfilter
elasticmachine Sep 21, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion server/src/main/java/org/elasticsearch/indices/analysis/AnalysisModule.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,10 @@ static Map<String, PreConfiguredTokenFilter> setupPreConfiguredTokenFilters(List
// Add "standard" for old indices (bwc)
preConfiguredTokenFilters.register( "standard",
PreConfiguredTokenFilter.elasticsearchVersion("standard", true, (reader, version) -> {
if (version.before(Version.V_7_0_0)) {
// This was originally removed in 7_0_0 but due to a cacheing bug it was still possible
// in certain circumstances to create a new index referencing the standard token filter
// until version 7_5_2
if (version.before(Version.V_7_6_0)) {
Copy link
Contributor

@jimczi jimczi Sep 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a comment here so that we don't forget why it's 7.6 and not 7.0 ?

deprecationLogger.deprecate("standard_deprecation",
"The [standard] token filter is deprecated and will be removed in a future version.");
} else {
Expand Down
24 changes: 19 additions & 5 deletions server/src/test/java/org/elasticsearch/indices/analysis/AnalysisModuleTests.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -228,14 +228,28 @@ public void testUnderscoreInAnalyzerName() throws IOException {
}
}

public void testStandardFilterBWC() {
Version version = VersionUtils.randomVersionBetween(random(), Version.V_7_0_0, Version.CURRENT);
final Settings settings = Settings.builder().put("index.analysis.analyzer.my_standard.tokenizer", "standard")
public void testStandardFilterBWC() throws IOException {
// standard tokenfilter should have been removed entirely in the 7x line. However, a
// cacheing bug meant that it was still possible to create indexes using a standard
// filter until 7.6
{
Version version = VersionUtils.randomVersionBetween(random(), Version.V_7_6_0, Version.CURRENT);
final Settings settings = Settings.builder().put("index.analysis.analyzer.my_standard.tokenizer", "standard")
.put("index.analysis.analyzer.my_standard.filter", "standard")
.put(Environment.PATH_HOME_SETTING.getKey(), createTempDir().toString()).put(IndexMetadata.SETTING_VERSION_CREATED, version)
.build();
IllegalArgumentException exc = expectThrows(IllegalArgumentException.class, () -> getIndexAnalyzers(settings));
assertThat(exc.getMessage(), equalTo("The [standard] token filter has been removed."));
IllegalArgumentException exc = expectThrows(IllegalArgumentException.class, () -> getIndexAnalyzers(settings));
assertThat(exc.getMessage(), equalTo("The [standard] token filter has been removed."));
}
{
Version version = VersionUtils.randomVersionBetween(random(), Version.V_7_0_0, Version.V_7_5_2);
final Settings settings = Settings.builder().put("index.analysis.analyzer.my_standard.tokenizer", "standard")
.put("index.analysis.analyzer.my_standard.filter", "standard")
.put(Environment.PATH_HOME_SETTING.getKey(), createTempDir().toString()).put(IndexMetadata.SETTING_VERSION_CREATED, version)
.build();
getIndexAnalyzers(settings);
assertWarnings("The [standard] token filter is deprecated and will be removed in a future version.");
}
}

/**
Expand Down