Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat(peliasAdmin): Remove word delimiter filter
The first error seen when trying to use our current schema with Elasticsearch 7 is: ``` [illegal_argument_exception] Token filter [word_delimiter] cannot be used to parse synonyms ``` The [word delimiter](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-word-delimiter-tokenfilter.html) token filter is only used in one place: the `peliasAdmin` analyzer. Looking at the documentation for `word_delimiter`, it does _a lot_: splitting words, handling punctuation, and even some basic stemming. It really feels like an extremely broad tool and at this point feels like something that Elasticsearch would deprecate in the future. Furthermore, looking at our integration tests, it seems one of the key reasons we used it was to tokenize on hyphens, which we have done using the `peliasNameTokenizer` since #375. Considering how complicated this token filter is, and how it's now being used with relatively little effect, it seems like something we can remove. Connects pelias/pelias#831
- Loading branch information