Add pipeline reference example to coredns#108
Merged
ruflin merged 1 commit intoelastic:masterfrom Sep 25, 2019
Merged
Conversation
Pipelines are input specific. Currently there is no simple way for the integrations manager to know which pipeline should be reference in an input. This introduces the convention that the pipeline name must be a variable which matches one of the pipeline file names in the package. On creating the pipeline instance, the integrations manager will change and prefix the name. The above convention makes it possible to convert it in both places. As an example: The package has an ingest pipeline `foo.yml` or `foo.json`, in the input it will reference `foo`. Then the integration manager builds the datasource and calls it `bar-foo`. It will the replace the input variable with `bar-foo`. This also has consequences on elastic/integrations#3. It means the developer building integrations must ensure that no two pipelines have the same name in one package. To make sure the name in an input matches a pipeline, later on also a validation step should be added.
Collaborator
Author
Collaborator
Author
|
@jsoriano Pinging you on this one as it's relevant for elastic/integrations#3 |
Collaborator
Author
|
Merging for now to have the change in as an example. Overall rethinking of inputs will happen in #110 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pipelines are input specific. Currently there is no simple way for the integrations manager to know which pipeline should be reference in an input. This introduces the convention that the pipeline name must be a variable which matches one of the pipeline file names in the package.
On creating the pipeline instance, the integrations manager will change and prefix the name. The above convention makes it possible to convert it in both places. As an example: The package has an ingest pipeline
foo.ymlorfoo.json, in the input it will referencefoo. Then the integration manager builds the datasource and calls itbar-foo. It will the replace the input variable withbar-foo.This also has consequences on elastic/integrations#3. It means the developer building integrations must ensure that no two pipelines have the same name in one package.
To make sure the name in an input matches a pipeline, later on also a validation step should be added.