Additional cleanup and improvements noticed when working on #864

[webmat]

In order to minimize the amount of noise on #864 (already a big rewrite) many small improvements have been held off. This PR introduces these pending improvements. If anything in there is contentious, I'm happy to split them off in their own PR.

Here are the changes:

• Field reuse: Small wording improvement in the field reuse docs. We now say whether fields are expected 'at the root of events', rather than 'at the top level'.
• Allowed values: We're now cleaning up leading and trailing spaces from the descriptions of allowed values in the categorization fields. We were already doing this kind of cleanup for all other textual attributes.
• object_type: We used to add a default datatype attribute of object_type=keyword for all object fields in ECS. This is no longer the case. A bit more on this below.
• dashed_name: Replace @ with - in dashed_name. This attribute is meant to be a URL-safe attribute that more or less represents the field name.
• reused_here: Deprecate the nestings attribute in favour of reused_here. The old representation could not capture additional information about a given reuse (e.g. specifying what field set is being reused, which is important when reusing as another name).
  • I'm only doing this via a deprecation notice in the changelog for now, as these YAML files aren't documented yet.
• Short descriptions: The main generator (and CI) now raises when short descriptions are longer than 120 chars. 19 short descriptions have been adjusted or introduced accordingly.

More on object_type

The effect of setting object_type=keyword on object fields is that Beats will constrain subkeys of this object field to a certain type, via a mix of dynamic template properties. However there's no reason to introduce this as a default on all object fields in ECS. This decision should be made on a case by case basis. Note also that this attribute is currently only supported by the Beats template generator, the ECS template generator currently does nothing with it.

Object fields that are no longer constrained by this property added as a default are:

• dns.answers
• log.syslog
• network.inner
• observer.egress
• observer.ingress

None of these object fields were meant to have dynamic templates and dynamically defined fields under them. So this should not be an issue.

The only place object_type makes sense in ECS is for the various labels object fields. They are expected to be constrained to only keyword key-value pairs. These fields are currently labels and container.labels (where object_type=keyword) is already defined explicitly.

[webmat]

@andrewkroh or @adriansr I would like your thoughts on the object_type discussion above.

[webmat]

I added a few changelog entries for the changelog-worthy changes.

[ebeahan]

LGTM 👍