Add flexibility to the field set reuse mechanism#820
Add flexibility to the field set reuse mechanism#820webmat wants to merge 12 commits intoelastic:masterfrom
Conversation
|
Ah I hadn't noticed the broken asciidoc links that cause the docs build to fail. I'll fix that shortly. The rest can already be reviewed. |
|
Ok this is ready for a review. |
|
|
||
| ==== Field Reuse | ||
|
|
||
| The `process` fields are expected to be nested at: `process.parent`. |
There was a problem hiding this comment.
Have you thought of ways to explain what each nesting means? I think we just copy and paste fieldsets over, but I don't know if we explain what the context of each nesting should mean.
There was a problem hiding this comment.
The new structure for how we specify field reuse will easily allow for that:
reusable:
top_level: true
expected:
- at: process
as: parent
description: The process that gave birth to the one detailed at the root of the event.And I think it's very much needed. However I think we should tackle this independently, as this will be a significant endeavour to add all of these contextual definitions and redesign the docs section about field reuse.
|
@rw-access I pinged you on this PR in part because we had forgotten to nest |
| raise ValueError( | ||
| 'Reusable fields cannot be put inside other reusable fields except when the destination reusable is at the top level') | ||
| nesting_destination = nesting_destination.setdefault('fields', {}) | ||
| nesting_destination[nest_as] = copy.deepcopy(schema) |
There was a problem hiding this comment.
Inserting a copy of the schema here instead of a reference to the original schema will lead to the same bug that was fixed by #722. I think to solve this we can make this line nesting_destination[nest_as] = schema and change line 177 below to schema['self_nestings'][self_nesting['as']]['fields'] = schema['fields']. This way the both the regular and self reused fieldsets will also receive any modifications to the schema fields that occur after this reusable is inserted.
Then, on line 230 when the fieldsets references are being recursively replaced with independent copies, self_nestings can be combined with fields to produce the final set of fields.
|
This PR has been superseded by #864 |
New possibilities:
Most of the time, self-nesting and "nest as" will be used together. Nothing currently prevents the nesting of a fieldset elsewhere as another name. But this should be used with caution, as it could lead to confusion.
Examples:
processatprocess.parentuseratuser.targetand the other locations discussed in [ECS] Multiple users in an event proposal #809.It's important to highlight one detail about self-nesting. Field sets like
userthat are also reused within other field sets (e.g.destination.user) and that will soon also have self-nesting (e.g.user.target,user.effective) will not carry the self-nesting in all other places (there won't be adestination.user.target).In addition to improving the reuse mechanism, this PR replaces all of the
process.parent.*manually duplicated fields. The process of removing this manual duplication exposed a few problems with the prior manual duplication, which are fixed by this PR:process.*andprocess.parent.*fields.pefield set was nestedprocess.pebut notprocess.parent.pe.process.parent.*fields were all explicit before, they were all explicitly listed in the asciidoc field listing. They are now indicated only via the normal "field reuse" section at the bottom of the page.This PR does not address #809. This will be done as a separate PR.