Adding elastic plugins to photon

[kenseii]

Hello,

I am trying to setup add a few plugins to allow better indexing and searching for japanese addresses however when i compile the package with the mvn package command, i get an error that normally should happen if the plugin is not installed.

Please let me know if i am doing something wrong.

I am using this statement inside the index-settings:

"normalize": {
	"type": "icu_normalizer",
	"name": "nfkc",
	"mode": "compose"
}

I installed the plugin jars this way:

mvn install:install-file -Dfile=.../analysis-icu-5.6.16/elasticsearch/analysis-icu-5.6.16.jar -DgroupId=org.elasticsearch.plugin -DartifactId=analysis-icu -Dversion=5.6.16 -Dpackaging=jar

mvn install:install-file -Dfile=.../analysis-icu-5.6.16/elasticsearch/lucene-analyzers-icu-6.6.1.jar -DgroupId=org.elasticsearch.plugin -DartifactId=lucene-analyzers-icu -Dversion=5.6.16 -Dpackaging=jar

mvn install:install-file -Dfile=.../analysis-icu-5.6.16/elasticsearch/icu4j-54.1.jar -DgroupId=org.elasticsearch.plugin -DartifactId=icu4j-54.1 -Dversion=5.6.16 -Dpackaging=jar

I also added the artifactitem as well as dependency in the pom.xml file

The error

2021-03-10 11:59:24,190 [main] INFO  de.komoot.photon.elasticsearch.Server - started elastic search node
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 4.867 sec <<< FAILURE!
addSimpleDoc(de.komoot.photon.elasticsearch.ImporterTest)  Time elapsed: 4.867 sec  <<< ERROR!
java.lang.IllegalArgumentException: Unknown char_filter type [icu_normalizer] for [normalize]
        at org.elasticsearch.index.analysis.AnalysisRegistry.getAnalysisProvider(AnalysisRegistry.java:387)
        at org.elasticsearch.index.analysis.AnalysisRegistry.buildMapping(AnalysisRegistry.java:338)
        at org.elasticsearch.index.analysis.AnalysisRegistry.buildCharFilterFactories(AnalysisRegistry.java:181)
        at org.elasticsearch.index.analysis.AnalysisRegistry.build(AnalysisRegistry.java:153)
        at org.elasticsearch.index.IndexService.<init>(IndexService.java:145)
        at org.elasticsearch.index.IndexModule.newIndexService(IndexModule.java:363)
        at org.elasticsearch.indices.IndicesService.createIndexService(IndicesService.java:448)
        at org.elasticsearch.indices.IndicesService.createIndex(IndicesService.java:413)
        at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$1.execute(MetaDataCreateIndexService.java:378)
        at org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:45)
        at org.elasticsearch.cluster.service.ClusterService.executeTasks(ClusterService.java:634)
        at org.elasticsearch.cluster.service.ClusterService.calculateTaskOutputs(ClusterService.java:612)
        at org.elasticsearch.cluster.service.ClusterService.runTasks(ClusterService.java:571)
        at org.elasticsearch.cluster.service.ClusterService$ClusterServiceTaskBatcher.run(ClusterService.java:263)
        at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150)
        at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188)
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:576)
        at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:247)
        at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:210)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)


Results :

Tests in error: 
  testApiReverse(de.komoot.photon.ApiIntegrationTest): Unknown char_filter type [icu_normalizer] for [normalize]
  testSearchForBerlin(de.komoot.photon.ApiIntegrationTest): Unknown char_filter type [icu_normalizer] for [normalize]
  testCorsOriginIsSetToSpecificDomain(de.komoot.photon.ApiIntegrationTest): Unknown char_filter type [icu_normalizer] for [normalize]
  testCorsAny(de.komoot.photon.ApiIntegrationTest): Unknown char_filter type [icu_normalizer] for [normalize]
  testNoCors(de.komoot.photon.ApiIntegrationTest): Unknown char_filter type [icu_normalizer] for [normalize]
  testApiWithLocationBias(de.komoot.photon.ApiIntegrationTest): Unknown char_filter type [icu_normalizer] for [normalize]
  testKeyTourismAndKeyNotAmenity(de.komoot.photon.query.PhotonQueryBuilderSearchTest): Unknown char_filter type [icu_normalizer] for [normalize]
  testFilterWithTagTourismAttraction(de.komoot.photon.query.PhotonQueryBuilderSearchTest): Unknown char_filter type [icu_normalizer] for [normalize]
  testTagNotTourismAttraction(de.komoot.photon.query.PhotonQueryBuilderSearchTest): Unknown char_filter type [icu_normalizer] for [normalize]
  testFilterWithoutTagTourismAttraction(de.komoot.photon.query.PhotonQueryBuilderSearchTest): Unknown char_filter type [icu_normalizer] for [normalize]
  testKeyTourism(de.komoot.photon.query.PhotonQueryBuilderSearchTest): Unknown char_filter type [icu_normalizer] for [normalize]
  testKeyNotTourismAndKeyNotAmenity(de.komoot.photon.query.PhotonQueryBuilderSearchTest): Unknown char_filter type [icu_normalizer] for [normalize]
  testValueInformationButKeyNotAmenity(de.komoot.photon.query.PhotonQueryBuilderSearchTest): Unknown char_filter type [icu_normalizer] for [normalize]
  testKeyTourismAndValueNotInformation(de.komoot.photon.query.PhotonQueryBuilderSearchTest): Unknown char_filter type [icu_normalizer] for [normalize]
  testValueNotInformation(de.komoot.photon.query.PhotonQueryBuilderSearchTest): Unknown char_filter type [icu_normalizer] for [normalize]
  testKeyTourismButValueNotInformation(de.komoot.photon.query.PhotonQueryBuilderSearchTest): Unknown char_filter type [icu_normalizer] for [normalize]
  testValueAttraction(de.komoot.photon.query.PhotonQueryBuilderSearchTest): Unknown char_filter type [icu_normalizer] for [normalize]
  testKeyNotTourism(de.komoot.photon.query.PhotonQueryBuilderSearchTest): Unknown char_filter type [icu_normalizer] for [normalize]
  addNameToDoc(de.komoot.photon.elasticsearch.UpdaterTest): Unknown char_filter type [icu_normalizer] for [normalize]
  removeNameFromDoc(de.komoot.photon.elasticsearch.UpdaterTest): Unknown char_filter type [icu_normalizer] for [normalize]
  addSimpleDoc(de.komoot.photon.elasticsearch.ImporterTest): Unknown char_filter type [icu_normalizer] for [normalize]

Tests run: 93, Failures: 0, Errors: 21, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:35 min
[INFO] Finished at: 2021-03-10T11:59:25+09:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test (default-test) on project photon: There are test failures.
[ERROR] 
[ERROR] Please refer to /home/oasis/Workspace/photon/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test (default-test) on project photon: There are test failures.

Please refer to /home/oasis/Workspace/photon/target/surefire-reports for the individual test results.
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: org.apache.maven.plugin.MojoFailureException: There are test failures.

Please refer to /home/oasis/Workspace/photon/target/surefire-reports for the individual test results.
    at org.apache.maven.plugin.surefire.SurefireHelper.reportExecution (SurefireHelper.java:83)
    at org.apache.maven.plugin.surefire.SurefirePlugin.writeSummary (SurefirePlugin.java:176)
    at org.apache.maven.plugin.surefire.SurefirePlugin.handleSummary (SurefirePlugin.java:150)
    at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked (AbstractSurefireMojo.java:650)
    at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute (AbstractSurefireMojo.java:586)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
[ERROR] 
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the followig ```

[kenseii]

this the current branch https://github.com/MIERUNE/photon/tree/feature/adopt-japanese-lang