Add docker `event` metricset by exekias · Pull Request #9856 · elastic/beats

exekias · 2019-01-02T23:30:42Z

This metricset will retrieve events coming from the Docker events
API [0].

Example output:

"docker": {
  "event": {
    "id": "8c229155b039c2adcb4fab1f987f35a0d1f913dfaa95f3113ed6e4f91eb5398c",
    "from": "busybox",
    "type": "container",
    "action": "die",
    "actor": {
      "id": "8c229155b039c2adcb4fab1f987f35a0d1f913dfaa95f3113ed6e4f91eb5398c",
      "attributes": {
        "image": "busybox",
        "name": "distracted_lichterman",
        "exitCode": "0"
      }
    },
    "time": "2019-01-02T22:41:02.000Z",
    "status": "die"
  }
}

Actor attributes will contain labels in the case of container
events, the module will perform dedotting (if enabled) on them.

[0] https://docs.docker.com/engine/api/v1.37/#operation/SystemEvents

closes #6799

This metricset will retrieve events coming from the Docker events API [0]. Example output: ``` "docker": { "event": { "id": "8c229155b039c2adcb4fab1f987f35a0d1f913dfaa95f3113ed6e4f91eb5398c", "from": "busybox", "type": "container", "action": "die", "actor": { "id": "8c229155b039c2adcb4fab1f987f35a0d1f913dfaa95f3113ed6e4f91eb5398c", "attributes": { "image": "busybox", "name": "distracted_lichterman", "exitCode": "0" } }, "time": "2019-01-02T22:41:02.000Z", "status": "die" } } ``` Actor attributes will container labels in the case of container events, the module will perform dedotting (if enabled) on them. [0] https://docs.docker.com/engine/api/v1.37/#operation/SystemEvents

andrewkroh

Is the schema here setup to follow the format of the existing metricset’s in this module? Based only on a brief look I think there are some ECS fields that could be populated either in addition to or as an alternative.

andrewkroh · 2019-01-03T05:15:14Z

metricbeat/module/docker/event/event.go

+func (m *MetricSet) reportEvent(reporter mb.PushReporterV2, event events.Message) {
+	time := time.Unix(event.Time, 0)
+
+	attributes := common.MapStr{}


You could allocate a map based on len(event.Actor.Attributes).

jsoriano

This is looking good, just a couple of comments.

metricbeat/module/docker/event/event.go

metricbeat/module/docker/event/event_integration_test.go

metricbeat/module/docker/event/event.go

ruflin · 2019-01-07T08:13:12Z

metricbeat/module/docker/event/_meta/fields.yml

+      migration: true
+      description: >
+        The type of object emitting the event
+    - name: action


You set migration: true here. Is this an alias we only want to be enabled if migration is enabled or have it always? Same for the other aliases.

I want it to stay, I'm guessing I should not have put migration there 😇 ? I can open a PR to remove them 👍

yes, otherwise they will disappear.

@exekias BTW this is a more general discussion if we want to have fields in ECS only or also the "original" name around.

* Add docker `event` metricset This metricset will retrieve events coming from the Docker events API [0]. Example output: ``` "docker": { "event": { "id": "8c229155b039c2adcb4fab1f987f35a0d1f913dfaa95f3113ed6e4f91eb5398c", "from": "busybox", "type": "container", "action": "die", "actor": { "id": "8c229155b039c2adcb4fab1f987f35a0d1f913dfaa95f3113ed6e4f91eb5398c", "attributes": { "image": "busybox", "name": "distracted_lichterman", "exitCode": "0" } }, "time": "2019-01-02T22:41:02.000Z", "status": "die" } } ``` Actor attributes will container labels in the case of container events, the module will perform dedotting (if enabled) on them. [0] https://docs.docker.com/engine/api/v1.37/#operation/SystemEvents * Add tests * Update changelog * Add missing header * Add missing file * Adapt to ECS * Use a cheaper event for tests * Close docker client on module shutdown * Update data * clean created container * fix image pull (cherry picked from commit eef102d)

* Add docker `event` metricset (#9856) * Add docker `event` metricset This metricset will retrieve events coming from the Docker events API [0]. Example output: ``` "docker": { "event": { "id": "8c229155b039c2adcb4fab1f987f35a0d1f913dfaa95f3113ed6e4f91eb5398c", "from": "busybox", "type": "container", "action": "die", "actor": { "id": "8c229155b039c2adcb4fab1f987f35a0d1f913dfaa95f3113ed6e4f91eb5398c", "attributes": { "image": "busybox", "name": "distracted_lichterman", "exitCode": "0" } }, "time": "2019-01-02T22:41:02.000Z", "status": "die" } } ``` Actor attributes will container labels in the case of container events, the module will perform dedotting (if enabled) on them. [0] https://docs.docker.com/engine/api/v1.37/#operation/SystemEvents * Add tests * Update changelog * Add missing header * Add missing file * Adapt to ECS * Use a cheaper event for tests * Close docker client on module shutdown * Update data * clean created container * fix image pull (cherry picked from commit eef102d) * Disable migration in docker event fields (#9947) * Move back docker event fields to it's namespace (#10073) * Move back docker event fields to it's namespace

Carlos Pérez-Aradros Herce added 2 commits January 2, 2019 23:44

Add tests

be75f8b

exekias added enhancement review Metricbeat Metricbeat containers Related to containers use case Team:Integrations Label for the Integrations team labels Jan 2, 2019

exekias requested a review from a team as a code owner January 2, 2019 23:30

Carlos Pérez-Aradros Herce added 3 commits January 3, 2019 00:34

Update changelog

8f647ab

Add missing header

a9b753e

Add missing file

820c7b1

exekias force-pushed the docker-events branch from 4a0bad4 to 820c7b1 Compare January 2, 2019 23:55

urso removed the request for review from a team January 3, 2019 00:12

andrewkroh reviewed Jan 3, 2019

View reviewed changes

Adapt to ECS

9366732

exekias force-pushed the docker-events branch from e0c5835 to 9366732 Compare January 3, 2019 11:13

jsoriano reviewed Jan 3, 2019

View reviewed changes

metricbeat/module/docker/event/event.go Show resolved Hide resolved

metricbeat/module/docker/event/event_integration_test.go Outdated Show resolved Hide resolved

metricbeat/module/docker/event/event.go Show resolved Hide resolved

Carlos Pérez-Aradros Herce added 4 commits January 3, 2019 12:39

Use a cheaper event for tests

6bb547f

Close docker client on module shutdown

1eea2a7

Update data

ed19c9e

clean created container

a819932

jsoriano approved these changes Jan 3, 2019

View reviewed changes

fix image pull

8a490c8

exekias requested review from a team as code owners January 3, 2019 13:49

exekias added needs_backport PR is waiting to be backported to other branches. v6.7.0 labels Jan 3, 2019

exekias merged commit eef102d into elastic:master Jan 4, 2019

ruflin reviewed Jan 7, 2019

View reviewed changes

exekias self-assigned this Jan 7, 2019

This was referenced Jan 8, 2019

Disable migration in docker event fields #9947

Merged

Cherry-pick #9856 to 6.x: Add docker event metricset #9962

Merged

exekias removed the needs_backport PR is waiting to be backported to other branches. label Jan 9, 2019

exekias mentioned this pull request Jan 15, 2019

Move back docker event fields to it's namespace #10073

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add docker `event` metricset#9856

Add docker `event` metricset#9856
exekias merged 11 commits intoelastic:masterfrom
exekias:docker-events

exekias commented Jan 2, 2019 •

edited

Loading

Uh oh!

andrewkroh left a comment

Uh oh!

andrewkroh Jan 3, 2019

Uh oh!

jsoriano left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ruflin Jan 7, 2019

Uh oh!

exekias Jan 7, 2019

Uh oh!

ruflin Jan 7, 2019

Uh oh!

ruflin Jan 7, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

exekias commented Jan 2, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

andrewkroh left a comment

Choose a reason for hiding this comment

Uh oh!

andrewkroh Jan 3, 2019

Choose a reason for hiding this comment

Uh oh!

jsoriano left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ruflin Jan 7, 2019

Choose a reason for hiding this comment

Uh oh!

exekias Jan 7, 2019

Choose a reason for hiding this comment

Uh oh!

ruflin Jan 7, 2019

Choose a reason for hiding this comment

Uh oh!

ruflin Jan 7, 2019

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

exekias commented Jan 2, 2019 •

edited

Loading