Feat: ECS compatibility (RFC email fields)

[kares]

ATM, this PR extends #55 and support for ECS (unreleased) email.* fields using a target => [email] default in ECS mode.

Current DIFF from PR #55

There's a few extra bits that make sense from a user perspective having the email fields in ECS mode:

• ability to skip processing all the headers headers_target => ''
• also attachments target becomes redundant thus user might want to disable attachments_target => ''

---

This PR will need a review once the email RFC is pushed, there's a few items to implement/consider :

• not expanding fields such as [email][from] into [email][from][address] (breaking ECS compatibility)
• skip the headers_target => '[@metadata][input][imap][headers]' default is ECS mode?
• skip (and deprecate) the attachments_target => '[@metadata][input][imap][attachments]' default is ECS mode?

[yaauie]

I wonder if it makes sense to differentiate here between v1 of ECS (where the ECS RFC for email certainly won't be present) and v8 of ECS (where it may be present)? Or if we need another parameter so that users can opt into the RFC shape?

I am concerned about the transition, of users who update this plugin and begin using ECS mode now, only to consume an upgrade at some point in the near future and have defaults change out of under them. Planning a transition for these users will take some care, and I will think about that in the next day or so.

[kares]

I wonder if it makes sense to differentiate here between v1 of ECS (where the ECS RFC for email certainly won't be present) and v8 of ECS (where it may be present)? Or if we need another parameter so that users can opt into the RFC shape?

would prefer to support one thing but this gets messy - if we would to strictly follow the logic:
in v1 the RFC was at stage 1, in v8 the RFC will likely be updated to stage 2 with a breaking change ([email][from] vs [email][from][address]). also it's still an RFC so the generated template for ES does not contain the mappings.

I am concerned about the transition, of users who update this plugin and begin using ECS mode now, only to consume an upgrade at some point in the near future and have defaults change out of under them.

yeah so changing the defaults (switching to no attachments_target default in ECS mode) was meant if we ship this with the #55 work as one plugin update. otherwise obviously we would likely not want a breaking change and we'll need to live with duplicate inconsistent information in ECS mode:

• metadata attachments that use filename and can optionally contain the file contents (if save_attachments => true)
• [email][attachments] info about attachments that never contains the payload

Planning a transition for these users will take some care, and I will think about that in the next day or so.

atm, given the RFC nature of email.* in ECS v8 I am leaning towards NOT shipping these updates until the email.* mappings are part of the v8 release. we can plan ahead (in #55) and log a warning in v8 mode that the plugin is planning to fill in email.* fields in the near future (just not sure hot to phrase it 😸 ).