Skip to content

Conversation

@andsel
Copy link
Contributor

@andsel andsel commented Apr 2, 2021

If the plugin name is an alias and no other plugin exists with same name then load the alised one

Release notes

Introduce the concept of aliased plugin, to let a plugin be renamed easily

What does this PR do?

An alias for a plugin is a just a rename of an existing plugin, in this way the alias can be used in pipeline configuration. If later a real plugin with same name of the alias is released then that plugin takes precedence over the alias name.
Suppose we a plugin filter named teleport and we also alias it as teleport_over_fiber, the teleport_over_fiber alias can be used in interchangeable way as teleport with same parameters and behavior. In a future when a real plugin that leverage fiber transmission for teleport a plugin teleport_over_fiber name is effectively released, this real plugin takes precedence to over the alias.

The changes to logstash-plugin tool used to manage the installation, listing and removal of aliased plugin is handled by the PR #12821

Why is it important/What is the impact to the user?

It lets the user to star using today plugins names that will be released as independent plugin in near future.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files (and/or docker env variables)
  • I have added tests that prove my fix is effective or that my feature works

Author's Checklist

  • try a local run

How to test this PR locally

Use an alias in a pipeline config, for example:

input {
  elastic_agent { # beats alias
     port => 5044
   }
}
output { stdout {} }

check that it runs correctly with bin/logstash -f pipeline.conf

Related issues

Use cases

Screenshots

Logs

addAliasedPlugins(CODECS);
}

private static <T> void addAliasedPlugins(Map<String, Class<T>> pluginCache) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note for review.
Here I had the doubt to use streams, introducing a support class:

private static final class NameAndClass<T> {
    final String name;
    final Class<T> clazz;

    NameAndClass(String name, Class<T> clazz) {
        this.name = name;
        this.clazz = clazz;
    }

    @SuppressWarnings("unchecked")
    static <T> NameAndClass<T> fromEntry(Map.Entry<String, T> e) {
        return new NameAndClass<T>(e.getKey(), (Class<T>) e.getValue());
    }

    @SuppressWarnings("unchecked")
    public <T> NameAndClass<T> withName(String newName) {
        return new NameAndClass<T>(newName, (Class<T>) clazz);
    }
}


final Map<String, Class<T>> aliasedToAdd = pluginCache.entrySet().stream()
        .map(NameAndClass::fromEntry)
        .filter(e -> isAliased(e.name))
        .map(n -> n.withName(aliasFromOriginal(n.name)))
        .filter(n -> !pluginCache.containsKey(n.name))
        .collect(Collectors.<NameAndClass, String, Class<T>>toMap(n -> n.name, n -> n.clazz));

But we have to create a support class only for this and has a lot of object creation due to stream

andsel and others added 11 commits April 20, 2021 21:28
…ased plugin and that a pipeline is correcrtly instantiated when an aliased plugin is used
Co-authored-by: Ry Biesemeyer <[email protected]>
It generates 2 problems:
- a reference overwriting when the aliases plugin is registered before the alias itself,
  there is an overwriting of an instance already created and possibly used by some other parts of the code, so a dangling reference
- a runtime error when a real plugin with same name of the alias is released in the future,
  due to a variable (resolved_plugin_name) not initialized
Because `Registry#namespace_lookup` will load the _first_ valid plugin it finds,
we cannot rely on it finding the concrete class when an alias of the same name
exists. Instead we need to revert `Registry#is_a_plugin?` to its original
definition so that `namespace_lookup` will only select the specific plugin that
has been requested, and introduce a new `Registry#is_a_plugin_or_alias?` to
use in the one context where we are willing to accept either.
@andsel andsel force-pushed the feature/load_and_use_aliased_plugins branch from 9587b46 to 1b72aac Compare April 20, 2021 19:28
Copy link
Member

@yaauie yaauie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I've got one last set of changes to suggest, but otherwise LGTM.

I'd like to get this merged/backported and chase down the unrelated DLQ test failures separately.

andsel and others added 2 commits April 20, 2021 21:41
Copy link
Member

@yaauie yaauie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

Let's merge & backport as soon as the ruby tests pass CI.

aliasesToAdd.put(aliasName, e.getValue());
final String typeStr = pluginType.name().toLowerCase();
LOGGER.info("Plugin {}-{} is aliased as {}-{}", typeStr, realPluginName, typeStr, aliasName);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤦🏼

Suggested change
}

@andsel andsel merged commit 5e77597 into elastic:master Apr 20, 2021
andsel added a commit to andsel/logstash that referenced this pull request Apr 20, 2021
Introcuce the concept of alias for a plugin.

Creates an AliasRegistry to map plugin aliases to original plugins.
If a real plugin with same name of the an alias is present in the system, then the real plugin take precedence during the
instantiation of the pipeline.

Simplified the error handling in class lookup

Co-authored-by: Ry Biesemeyer <[email protected]>
andsel added a commit that referenced this pull request Apr 20, 2021
Introcuce the concept of alias for a plugin (#12796) and removes static part from PluginRegistry to avoid static initializer (#12799)

Creates an AliasRegistry to map plugin aliases to original plugins.
If a real plugin with same name of the an alias is present in the system, then the real plugin take precedence during the
instantiation of the pipeline.

Simplified the error handling in class lookup

Co-authored-by: Ry Biesemeyer <[email protected]>
kares added a commit to kares/logstash that referenced this pull request May 10, 2021
* master:
  fix DLQ integration tests (elastic#12871)
  Fix ES HOW integration tests on `master` (elastic#12872)
  Update logstash_releases.json
  Support for UTF-16 and other multi-byte-character logfiles (elastic#9702)
  change download path for geoip plugin (elastic#12863)
  Change Gradle's :logstash-integration-tests:integrationTests task to depends on copyES (elastic#12847)
  Doc: Keystore must be accessible to logstash user (elastic#12775)
  Remove json ~>1 pinning (elastic#12851)
  Adapted install/uninstall/list PluginManager's command to respect the alised plugins (elastic#12821)
  Load a plugin by alias name. (elastic#12796)
@andsel andsel added the v7.13.0 label Jul 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants