Add dataset fields to all packages

[ruflin]

This is part of elastic/package-registry#491 This PR adds the new dataset fields to each package. As soon as all PRs in the registry, Kibana and Agent are merged, the old stream fields can be removed.

[elasticmachine]

💚 Build Succeeded

Expand to view the summary

Build stats

• Build Cause: [Pull request #56 opened]

• Start Time: 2020-06-08T13:23:30.708+0000

• Duration: 3 min 36 sec

[mtojek]

As soon as all PRs in the registry, Kibana and Agent are merged, the old stream fields can be removed.

What about dashboards in integrations? Will they stop working? I think you should follow the flow (first update integrations, then rollout changes to the package-storage).

[ruflin]

@mtojek Good point. The dashboards will stop working as soon as the Agent is changed. I suggest the following:

• Get this PR in, this unblocks Update registry for stream.* to dataset.* change package-registry#492 This has no effect yet on the data shipped
• Release a new version for all integrations with updated dashboards
• Merged Kibana / Agent PR

[mtojek]

Is it possible to rollout changes in a safe way without breaking anything? e.g. agent rewrites/copies fields.
It's better not to have any dependencies between pieces rather than risking breaking something.

[ruflin]

It would be possible to add both for now on the Agent side and later remove it in a follow up. @michalpristas WDYT about changing elastic/beats#18967 accordingly?

[ruflin]

@mtojek In any case, we will need the definitions for both in the packages for some time too, meaning this PR is valid in any case as it doesn't break anything.

[mtojek]

Approved, preserving there are no time restrictions on rolling out changes in other places.

Please create an issue for the integrations repository to fix all dashboards.

Just saw the PR, thanks! looking into this one.

[ruflin]

Thanks, merged. The only effect this has so far it is unblocking my registry PR and we have both fields in the templates for now.