filter_log_to_metrics: add new option 'discard_logs'

[edsiper]

[updated]

By default, the filter don't drop the logs from the pipeline, there are cases where only metrics are desired to reduce log volume.

This patch introduces a new option called discard_logs (default: false) which when enabled will discard the logs once they are processed.

usage:

service:
  log_level: info

pipeline:
  inputs:
    - name: dummy
      tag: dummy.log
      dummy: '{"message":"dummy", "kubernetes":{"namespace_name": "default", "docker_id":
        "abc123", "pod_name": "pod1", "container_name": "mycontainer", "pod_id": "def456"},
        "duration": 20, "color": "red", "shape": "circle"}'

      processors:
        logs:
          - name: log_to_metrics
            tag: test_metric
            metric_name: count_all_dummy_messages
            metric_mode: counter
            metric_description: This metric counts dummy messages
            discard_logs: true

  outputs:
    - name: prometheus_exporter
      match: '*'
      host: 0.0.0.0
      port: '2021'

    - name: stdout
      match: '*'

In addition, other changes were made:

• replacement of flb_malloc() with flb_calloc()
• context structure definition changes: e.g: do not malloc ints
• fix double free on cmetrics context destroy (histograms)
• general cleanups...

NOTE: I noticed that record accessor contexts are being created for fixed patterns (e.g: $kubernetes['pod_name']), we will need to fix this since the creation of those contexts are expensive, is times better to have them in the context and use it on demand.

---

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

[drbugfinder-work]

Is this indentation correct?