Stable Analysis Plugin API

[pgomulka]

Description

Description
This is a meta issue tracking subtasks to introduce a new api for analysis plugins.

The analysis plugin api will offer a stable a stable api upon which a plugin developer can depend. Plugins using the new api will no longer have to have an elasticsearch:server dependency, and therefore will be less prone to be affected by breaking changes whenever ES server internals are changed.

The API is derived from org.elasticsearch.index.analysis and contains the minimum set of interfaces and methods to implement analysis plugin.

Goals:

1. Design a minimum set of interfaces to allow for analysis plugin development
2. Forbid server dependency to prevent plugins be affected by changes in server internals

Phase 0 - will allow to implement a plugin with no settings

1. Design a minimum set of interfaces and methods which are subset of current analysis api. This task should create new gradle modules which will later be used as dependencies by plugin developers. The Api will also contain annotations which will allow analysis components (i.e. CharTokenFilterFactory), TokenizerFactory to be found. AnalysisPlugin interface will no longer be needed. Stable Plugin API module and analysis interfaces #88775 Fix module and package names for stable plugin api #89772 Rename NamedComponent name parameter to value #91306

• @ Extensible annotation [Stable plugin api] Extensible annotation #89903

3. Search for analysis components. Currently done with the use of AnalysisPlugin interface which have getters returning Map of Names to AnalysisProvider interface. The new stable plugins will no longer need to implement AnalysisPlugin interface. The analysis componentsannotated with@NamedComponent` will be scanned upon install. As a result of a scan during install time a cache file will be created describing what components are exported by a plugin. (to be done in a separate task) This task is loading the named_component.json file and extensibles.json and storest taht information in StablePluginRegistry [Stable plugin API] Load plugin named components #89969
4. Analysis component registration and creation. The analysis components implementing the new api will have to be wrapped in the old analysis interfaces to allow the code to work. This logic is currently done in AnalysisModule#setup* methods. Ideally the analysis component classes are only loaded when first used. Register stable plugins in ActionModule #90067
5. An example plugin to be build presenting how the api can be used. Example stable plugin #90805
6. Plugin Loading - A new stable plugin classloader to be implemented, preventing plugin to access server components. This plugin will load a group of unmodularized jars into a synthetic module, that is, as if all the classes in the jars were part of a single module.

• Stable Plugin API: Add Ubermodule ClassLoader for non-modularized stable plugins #89365
• Create placeholder plugin when loading stable plugins #90870
• Load stable plugins as synthetic modules #91869
• Check stable plugin version at install and load time #91780

9. Plugin descriptor - a new plugin descriptor does not require AnalsysPlugin class and contains isModular property Add support for reading stable plugin descriptors #88731 and Add stable indicator for plugin descriptor #88823
10. gradle es stable plugin - build plugin that creates stable-plugin-descriptor.properties and named-components.json Create gradle plugin for ES stable plugins #90355

Phase 1 - will allow to implement a plugin with analysis settings

1. Introduce @Inject and @AnalysisSettings. @Inject should be used by plugin developer to mark constructor in his analysis component where settings will be injected. @AnalsysiSettings should be used by plugin developer to mark an interface with getter methods for analysis settings. Upon analysis component creation, an implementation should be generated for the annotated settings interface. Ideally we should ASM to have good performance (dynamic proxy is another option, but might have poor performance). Within plugin an interface is used and it should result in more readable code in a plugin. Example from PoC https://github.com/elastic/elasticsearch/compare/main...pgomulka::annotations?expand=1#diff-b7343d12bb8b9a99d1b06deedfe600755ab3e4bae45be6c2b6d634d851eae771R37

• basic API and settings injection Settings api for stable plugins #91467
• Example plugin using settings Example stable plugins with settings #92334
• Documentation and socialisation
• component scanning upon install if cache file is missing Scan stable plugins for named components upon install #92528 and Refactor plugin scanning into lib #92437
• William - a BWC compatibility tool - checking for breaking changes in the plugin api
• Package name refactor (drop the api suffix) [Stable plugin api] Drop api suffix in package names #92905 and [Stable plugin api] update stable api example plugin after package rename #92908
• update the release manager for the new group name (same as package name) https://github.com/elastic/infra/pull/38248

Phase 3. [ ] An existing plugin migration

Phase x. additional features and nice to have

• Support for @NodeSettings, @IndexSettings and @ClusterSetting - equivalent of Setting.Property.NodeScope, IndexScope and Property.Dynamic
• Allow for custom parsers - an infrastructure, those new parsers should live in server (String, Integer etc)
• Extend settings api for Lists of custom types
• fallback to file option for List of Strings
• settings can be marked exclusive, and only one should be used. Like stopTags and stopTagsPath in nori plugin.
• exclusive options can be marked with fallback. When option is not present, the other one should be used instead. Example solution https://github.com/elastic/elasticsearch/compare/main...pgomulka::annotations?expand=1#diff-6059f7f7f17b0dae7cd79dcbe3b9b0ca726ae156d1b4acda9556d5dbb4a6f199R21
• default value for lists [Stable plugin api] Add a default parameter to ListSetting  #92874

Validation of plugin's setting interface

• setting's name clashes with node (index, cluster? ) setting
• validaiton upon settings creation types validation? range validation? This will introduce the Validator interface

[elasticsearchmachine]

Pinging @elastic/es-core-infra (Team:Core/Infra)