Skip to content

processor_content_modifier: add support for OTel Logs Resource and Scopes#8989

Merged
edsiper merged 1 commit intomasterfrom
otel-content-modifier
Jun 24, 2024
Merged

processor_content_modifier: add support for OTel Logs Resource and Scopes#8989
edsiper merged 1 commit intomasterfrom
otel-content-modifier

Conversation

@edsiper
Copy link
Member

@edsiper edsiper commented Jun 20, 2024

The following patch extends the processor to allow to modify the resources and scopes of Logs generated by an OpenTelemetry source.

The following new contexts are supported:

  • otel_resource_attributes: alter resource attributes
  • otel_scope_name: manipulate the scope name
  • otel_scope_version: manipulate the scope version
  • otel_scope_attributes: alter the scope attributes

example: fluent-bit.yaml

service:
  flush: 1
  log_level: info
  http_server: on
  http_port: ${FLUENT_BIT_HTTP_MONITORING_PORT}

  pipeline:
    inputs:
      - name: opentelemetry
        port: ${FLUENT_BIT_TEST_LISTENER_PORT}
        processors:
          logs:
            - name: content_modifier
              context: otel_resource_attributes
              action: upsert
              key: "new_attr"
              value: "my_val"

            - name: content_modifier
              context: otel_resource_attributes
              action: delete
              key: "service.name"

            - name: content_modifier
              context: otel_scope_attributes
              action: upsert
              key: "my_new_scope_attr"
              value: "123"

            - name: content_modifier
              context: otel_scope_name
              action: upsert
              value: "new scope name"

            - name: content_modifier
              context: otel_scope_version
              action: upsert
              value: "3.1.0"

  outputs:
    - name: stdout
      match: '*'

    - name: opentelemetry
      match: '*'
      host: 127.0.0.1
      port: ${TEST_SUITE_HTTP_PORT}

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
  • Debug log output from testing the change
  • Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • Run local packaging test showing all targets (including any new ones) build.
  • Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • Documentation required for this feature

Backporting

  • Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

…opes

The following patch extends the processor to allow to modify the resources and scopes
of Logs generated by an OpenTelemetry source.

The following new contexts are supported:

 - otel_resource_attributes: alter resource attributes
 - otel_scope_name: manipulate the scope name
 - otel_scope_version: manipulate the scope version
 - otel_scope_attributes: alter the scope attributes

example:

----- fluent-bit.yaml -----

pipeline:
  inputs:
    - name: opentelemetry
      port: ${FLUENT_BIT_TEST_LISTENER_PORT}
      processors:
        logs:
          - name: content_modifier
            context: otel_resource_attributes
            action: upsert
            key: "new_attr"
            value: "my_val"

          - name: content_modifier
            context: otel_resource_attributes
            action: delete
            key: "service.name"

          - name: content_modifier
            context: otel_scope_attributes
            action: upsert
            key: "my_new_scope_attr"
            value: "123"

          - name: content_modifier
            context: otel_scope_name
            action: upsert
            value: "new scope name"

          - name: content_modifier
            context: otel_scope_version
            action: upsert
            value: "3.1.0"

  outputs:
    - name: stdout
      match: '*'

    - name: opentelemetry
      match: '*'
      host: 127.0.0.1
      port: ${TEST_SUITE_HTTP_PORT}

----- end of file -----

Signed-off-by: Eduardo Silva <eduardo@calyptia.com>
@edsiper edsiper force-pushed the otel-content-modifier branch from 0576b6f to e05bdff Compare June 20, 2024 23:40
@edsiper edsiper changed the title procesor_content_modifier: add support for OTel Logs Resource and Scopes processor_content_modifier: add support for OTel Logs Resource and Scopes Jun 20, 2024
@edsiper edsiper added this to the Fluent Bit v3.1.0 milestone Jun 21, 2024
@edsiper edsiper merged commit 7060979 into master Jun 24, 2024
@edsiper edsiper deleted the otel-content-modifier branch June 24, 2024 04:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant