Generate full Beats field definitions, including nested fields#379
Merged
webmat merged 11 commits intoelastic:masterfrom Mar 8, 2019
Merged
Generate full Beats field definitions, including nested fields#379webmat merged 11 commits intoelastic:masterfrom
webmat merged 11 commits intoelastic:masterfrom
Conversation
added 3 commits
March 7, 2019 11:11
This also fixes a bug where the `group` fieldset was not actually being nested in all places where user is nested (e.g. `source.user.group.*`).
webmat
force-pushed
the
beats-field-definitions
branch
from
c16c812 to
f146f11
Compare
1 task
Contributor
Author
|
@ruflin Could you check this out? A big difference between this file and the previous one we crafted manually (other than the YAML rendering of strings) is that fields no longer are in the original order. If that's good with you for this, I think this is ready for final review. |
webmat
changed the title
webmat
force-pushed
the
beats-field-definitions
branch
from
7c6a7c3 to
6f6efb7
Compare
andrewkroh
reviewed
Mar 8, 2019
Contributor
Author
|
Fields listed under each |
Member
If it's not being used then I'd drop it. |
andrewkroh
approved these changes
Mar 8, 2019
Member
andrewkroh
left a comment
andrewkroh
left a comment
There was a problem hiding this comment.
LGTM. I looked over the generated fields.ecs.yml. This is nice. I didn't spend much time on the Python.
Contributor
Author
|
It's a Rubyist's Python, so probably better like that 😆 |
Contributor
Author
|
@ruflin Will ping you on the backport for review on Monday. Happy to adjust anything needed. But I'm moving forward here to prepare the 1.0 backport of this, then make elastic/beats#11150 final, so it can hopefully be merged Monday. |
webmat
added a commit
to webmat/ecs
that referenced
this pull request
Mar 8, 2019
…ncluding nested fields (elastic#379) Backport of PR elastic#379 to 1.0 branch. Original message: This work has revealed a subtle bug in the generated files schema.csv and both ES templates, which were missing the group fieldset in all places where user is reused. (e.g. `host.user.group.*`) Upon comparing this new file to the fields definition file we had handcrafted for Beats (prior to this), it also revealed we had missed a few things in the Beats field definitions: - We had forgotten to define the reusable `user` fieldset `client`, `destination`, `server` and `source`. They previously had been missed. - We had forgotten to define the reusable `geo` fieldset at `host.geo.*` and `observer.geo.*`
webmat
added a commit
that referenced
this pull request
Mar 11, 2019
…g nested fields (#379) (#381) Backport of PR #379 to 1.0 branch. Original message: This work has revealed a subtle bug in the generated files schema.csv and both ES templates, which were missing the group fieldset in all places where user is reused. (e.g. `host.user.group.*`) Upon comparing this new file to the fields definition file we had handcrafted for Beats (prior to this), it also revealed we had missed a few things in the Beats field definitions: - We had forgotten to define the reusable `user` fieldset in `client`, `destination`, `server` and `source`. They previously had been missed. - We had forgotten to define the reusable `geo` fieldset at `host.geo.*` and `observer.geo.*`
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This work has revealed a subtle bug in the generated files schema.csv and both ES templates, which were missing the
groupfieldset in all places whereuseris reused. (e.g.host.user.group.*)Notes
reusable). This new generator will filter them, to let through only what matters. Question: should we let through the ECSlevel?