Conversation
| # "false" will disable any extra processing necessary for preserving ordering. | ||
| # | ||
| pipeline.ordered: auto | ||
| # pipeline.ordered: auto |
There was a problem hiding this comment.
the default value is auto. commenting this out makes it match the rest of the config file where we don't declare default values in the config file we ship.
| ==== Opting Out | ||
|
|
||
| In Logstash 8, these plugins are run in ECS mode by default, but you can opt-out and use the legacy behaviour in one of several ways. | ||
| This can be helpful if you have very complex pipelines that were defined pre-ECS, to allow you to either upgrade them or to avoid doing so independently of your {ls} 8.x upgrade. |
There was a problem hiding this comment.
| This can be helpful if you have very complex pipelines that were defined pre-ECS, to allow you to either upgrade them or to avoid doing so independently of your {ls} 8.x upgrade. | |
| This can be helpful if you have very complex pipelines that were defined pre-ECS, to allow you to continue running them without modification in 8.0 to produce the same shape of data that they produced before your upgrade. |
| path.config: "/etc/path/to/ecs-pipeline.config" | ||
| pipeline.ecs_compatibility: v1 | ||
|
|
||
| NOTE: Until the General Availability of Logstash 8.0.0, any value for `pipeline.ecs_compatibility` other than `disabled` -- including the default value on this pre-release branch -- may have undesireable consequences when performing upgrades. |
There was a problem hiding this comment.
This caveat will go away before Logstash 8.0 reaches GA, which is why I call out the value on the pre-release branch. I want people hitting this doc to have a "wait a minute" moment.
For the 7.x backport, the text will be more like:
NOTE: Until the General Availability of Logstash 8.0.0 and the final minor release of Logstash 7.x, any value for
pipeline.ecs_compatibilityother thandisabledmay have undesireable consequences when performing upgrades.
As we continue to release updated plugins with ECS-Compatibility modes, opting into them at a pipeline or process level will cause you to automatically consume breaking changes with each upgrade, which may change the shape of data your pipeline produces.
And 7.final will simply not need this caveat at all.
yaauie
force-pushed
the
ecs-compatibility-docs
branch
2 times, most recently
from
53c61e3 to
939f93b
Compare
roaksoax
added
status:needs-review
status:work-in-progress
and removed
status:needs-review
status:work-in-progress
labels
karenzone
left a comment
karenzone
left a comment
There was a problem hiding this comment.
Suggestions for your consideration.I have tested these suggestions locally, and hope that I moved them over correctly. Github isn't an ideal tool for collaborating on docs, and I have no way to test until these suggestions are incorporated or rejected. This review is about structure and standards so there might be a bit more fine tuning (esp. behavior between versions) to do. Fantastic job on the docs.
yaauie
force-pushed
the
ecs-compatibility-docs
branch
3 times, most recently
from
48eb209 to
070df2b
Compare
|
jenkins test this again please |
karenzone
left a comment
karenzone
left a comment
There was a problem hiding this comment.
LGTM 🎉
Nice explanation. Thanks for adding this.
yaauie
force-pushed
the
ecs-compatibility-docs
branch
3 times, most recently
from
087bf44 to
a956af6
Compare
|
Ongoing test failures are unrelated; I've filed https://github.com/elastic/logstash/pull/13002/files to fix. Since this is a "breaking" change, I will wait until Monday when other folks are around to get both merged. |
Co-authored-by: Karen Metts <karen.metts@elastic.co>
We know that ECS version 8 will release along-side Logstash 8.0, but its scope is still coming into focus. In this changeset, we change the default value of `pipeline.ecs_compatibility` from `disabled` to `v1`, which is a significantly closer approximation to what will eventually ship in Logstash 8.0.0.
Co-authored-by: Karen Metts <karen.metts@elastic.co>
yaauie
force-pushed
the
ecs-compatibility-docs
branch
from
a956af6 to
9e00a7e
Compare
* noop: avoid declaring default value in config file (cherry picked from commit f8fe4b6) * docs: ecs compatibility from 7.x perspective Co-authored-by: Karen Metts <karen.metts@elastic.co> (cherry picked from commit 69702ca) * ecs docs: pr feedback a11y link text Co-authored-by: Karen Metts <35154725+karenzone@users.noreply.github.com> Co-authored-by: Karen Metts <35154725+karenzone@users.noreply.github.com>
* master: (41 commits) Test: resolve integration failure due ECS mode (elastic#13044) Feat: event factory support (elastic#13017) Doc: Add geoip database API to node stats (elastic#13019) Add geoip database metrics to /node/stats API (elastic#13004) ecs: on-by-default plus docs (elastic#12830) ispec: fix cross-spec leak from fatal error integration specs (elastic#13002) Fix UBI source URL (elastic#13008) update fpm to allow pkg creation on jdk11+jruby 9.2 (elastic#13005) Add unit test to grant that production aliases correspond to a published RubyGem (elastic#12993) Fix logstash.bat not setting exit code (elastic#12948) Use the OS separator to invoke gradlew from Rake script (elastic#13000) Allow per-pipeline config of ECS Compatibility mode via Central Management (elastic#12861) Update jinja2 dependency in docker build (elastic#12994) fix database manager with multiple pipelines (elastic#12862) Fix Reflections stack traces when process yml files in classpath and debug is enabled (elastic#12991) Fix/log4j routing to avoid create spurious file (elastic#12965) Deps: update JRuby to 9.2.19.0 (elastic#12989) Doc: Add tip for checking for existing field (elastic#12899) Added test to cover the installation of aliased plugins (elastic#12967) CI: Update logstash_release.json after 7.3.12 (elastic#12986) ...
3 participants
PREVIEW: https://logstash_12830.docs-preview.app.elstc.co/guide/en/logstash/master/ecs-ls.html
Release notes
Sets ECS to on-by-default (
v1, for now). This setting will again change tov8when ECS v8 goes into public Alpha in advance of the Stack 8.0.0 release.What does this PR do?
This change makes ECS Compatibility modes on-by-default (as previously promised in #11623) and adds documentation about how to configure ECS Compatibility Modes at the plugin instance, pipeline, and global levels.
These docs are from the perspective of someone running (the unreleased, pre-alpha) Logstash 8, and a backport will need to be entirely re-worded from the perspective of Logstash 7.x to lead people toward the upgrade.
If a user wants to "lock in" pre-ECS behaviour in 7.x, they can do at different levels so by:
ecs_compatibility => disabledon each individual plugin; ORpipeline.ecs_compatibility: disabledin each pipeline's definition (config/pipelines.ymlor Central Management)pipeline.ecs_compatibility: disabledinconfig/logstash.ymlto provide a global default for pipelines.Why is it important/What is the impact to the user?
The transition to ECS being on-by-default with Logstash 8.0 is a potentially sharp edge, as the plugins that implement ECS Compatibility modes will operate differently than in their legacy versions when run in Logstash 8, unless specific effort is made to opt-out of the breaking change.
Checklist