ECS: Set host.name along with host.hostname in exported data

[simianhacker]

APM server should be shipping host.name along with host.hostname per ECS. host.name should default to host.hostname but allow the user to override it with their host identifier if it's different. This is necessary to ensure cross-linking between Infra UI and APM is consistent.

[roncohen]

Related: elastic/ecs#498 (comment)

[roncohen]

We should probably set this in agents instead of in APM Server

[graphaelli]

In what situation does cross-linking happen based on host.name?

[simianhacker]

Everything except APM. In the Infra UI we have to put exceptions in for APM because they using host.hostname instead of host.name. As a company, we really need to standardize on one field across everything to represent an ID for a host. For Kubernetes Pods and Docker Containers we are all using the same ID fields. Although, it looks like there is host.id in the ECS documentation but Metricbeat doesn't use it for some reason.

Work Around Examples:

• https://github.com/elastic/kibana/pull/42197/files#diff-0f965401975f16849fcc929d348aeebcR221-R227
• https://github.com/elastic/kibana/blob/master/x-pack/legacy/plugins/infra/public/components/waffle/node_context_menu.tsx#L49-L57

@elastic/beats @ruflin Any reason why we are not setting host.id in Metricbeat/Filebeat? Seems like that should be the blessed identifier field for Hosts instead of host.name

[simitt]

Linking to some context around discussions of host.name vs. host.hostname: #1609 (comment). Haven't followed changes on host since then, but it seemed fair to leave the field empty if not user overwritten (which the agents don't support yet afaik).

[graphaelli]

Thanks for digging that up @simitt, I had forgotten some of that context.

We can certainly copy host.hostname to host.name for APM, and let agents override that value. We'd still need to keep some workaround in place as the curated UIs should fall back to linking based on host.hostname if host.name is not present, to support older 7.x data.

host.id would be great if multiple processes could arrive at the same value for it independently - that's why kubernetes and docker links work so well - but I don't think there is a portable way to do that well.