opentelemetry: fix handling of numeric attributes #10815
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
WalkthroughThe PR updates OpenTelemetry attribute handling to accept numeric strings for intValue and doubleValue, parsing them into numeric types during ingestion. Utilities and parser logic are adjusted accordingly, and new tests validate coercion for resource attributes with stringified integers. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant Client
participant in_opentelemetry as in_opentelemetry
participant Utils as flb_otel_utils_json_payload_get_wrapped_value
participant Parser as process_attribute
Client->>in_opentelemetry: POST /v1/logs (JSON)
in_opentelemetry->>Utils: Extract wrapped value (intValue/doubleValue)
alt value is number
Utils-->>in_opentelemetry: Return numeric msgpack object
else value is string
Utils-->>in_opentelemetry: Return string msgpack object
end
in_opentelemetry->>Parser: Process attribute (key, value)
alt integer path
opt value is string
Parser->>Parser: strtoll parse to int
end
Parser-->>in_opentelemetry: Set integer attribute
else float path
opt value is string
Parser->>Parser: strtod parse to double
end
Parser-->>in_opentelemetry: Set float attribute
else unsupported
Parser-->>in_opentelemetry: Error
end
in_opentelemetry-->>Client: 200 OK
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Assessment against linked issues
Assessment against linked issues: Out-of-scope changes
Poem
✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/opentelemetry/flb_opentelemetry_utils.c (1)
123-131: Allowing doubleValue as string is correct, but downstream conversion is missing.The unwrap path for FLOAT currently passes through strings unchanged, so {"doubleValue":"3.14"} will emit a string, not a double. Add symmetric parsing like the int branch.
Apply this diff in flb_otel_utils_json_payload_append_unwrapped_value:
- char temporary_buffer[33]; + char temporary_buffer[128]; @@ - else if (type == MSGPACK_OBJECT_FLOAT) { - unwrap_value = FLB_TRUE; - } + else if (type == MSGPACK_OBJECT_FLOAT) { + if (value->type == MSGPACK_OBJECT_STR) { + memset(temporary_buffer, 0, sizeof(temporary_buffer)); + + if (value->via.str.size < sizeof(temporary_buffer)) { + strncpy(temporary_buffer, + value->via.str.ptr, + value->via.str.size); + } + else { + strncpy(temporary_buffer, + value->via.str.ptr, + sizeof(temporary_buffer) - 1); + } + + result = flb_log_event_encoder_append_double( + encoder, + target_field, + strtod(temporary_buffer, NULL)); + } + else { + unwrap_value = FLB_TRUE; + } + }
🧹 Nitpick comments (5)
tests/internal/data/opentelemetry/test_cases.json (1)
1258-1280: Add doubleValue-as-string and negative intValue-as-string cases.To fully cover this PR’s goal, please add:
- resource attribute with {"doubleValue":"3.14"} → expected 3.14
- resource attribute with {"intValue":"-2"} → expected -2
Apply this minimal fixture extension:
@@ "resource_attribute_intvalue_string": { @@ } }, + "resource_attribute_doublevalue_string": { + "input": { + "resourceLogs": [{ + "resource": { + "attributes": [ + {"key": "resourceThreshold", "value": {"doubleValue": "3.14"}} + ] + }, + "scopeLogs": [{ + "logRecords": [{ + "timeUnixNano": "1640995200000000000", + "body": {"stringValue": "test log"} + }] + }] + }] + }, + "expected": { + "group_metadata": {"schema":"otlp","resource_id":0,"scope_id":0}, + "group_body": {"resource":{"attributes":{"resourceThreshold":3.14}}}, + "log_metadata": {"otlp":{}}, + "log_body": {"log": "test log"} + } + }, + "resource_attribute_intvalue_string_negative": { + "input": { + "resourceLogs": [{ + "resource": { + "attributes": [ + {"key": "resourceVersion", "value": {"intValue": "-2"}} + ] + }, + "scopeLogs": [{ + "logRecords": [{ + "timeUnixNano": "1640995200000000000", + "body": {"stringValue": "test log"} + }] + }] + }] + }, + "expected": { + "group_metadata": {"schema":"otlp","resource_id":0,"scope_id":0}, + "group_body": {"resource":{"attributes":{"resourceVersion":-2}}}, + "log_metadata": {"otlp":{}}, + "log_body": {"log": "test log"} + } + },src/opentelemetry/flb_opentelemetry_utils.c (1)
301-323: Consider stricter validation for integer strings.Current strtoll() usage accepts prefixes like "123abc". If you need tighter conformance, parse with endptr and ensure full consumption.
- result = flb_log_event_encoder_append_int64( + char *endp = NULL; + long long parsed = strtoll(temporary_buffer, &endp, 10); + if (endp == temporary_buffer || *endp != '\0') { + /* fall back to original string to avoid silent misparse */ + return -2; + } + result = flb_log_event_encoder_append_int64( encoder, target_field, - strtoll(temporary_buffer, NULL, 10)); + (int64_t) parsed);tests/internal/opentelemetry.c (1)
479-499: Add a symmetric test for doubleValue-as-string.Covers the broadened acceptance and guards future regressions.
@@ void test_json_payload_get_wrapped_value() - msgpack_sbuffer_destroy(&sbuf); - msgpack_unpacked_destroy(&up); + msgpack_sbuffer_destroy(&sbuf); + msgpack_unpacked_destroy(&up); + + /* Test double value provided as a string */ + msgpack_sbuffer_init(&sbuf); + msgpack_packer_init(&pck, &sbuf, msgpack_sbuffer_write); + + msgpack_pack_map(&pck, 1); + msgpack_pack_str(&pck, 11); + msgpack_pack_str_body(&pck, "doubleValue", 11); + msgpack_pack_str(&pck, 4); + msgpack_pack_str_body(&pck, "3.14", 4); + + msgpack_unpacked_init(&up); + msgpack_unpack_next(&up, sbuf.data, sbuf.size, NULL); + + ret = flb_otel_utils_json_payload_get_wrapped_value(&up.data, &val, &type); + TEST_CHECK(ret == 0); + TEST_CHECK(type == MSGPACK_OBJECT_FLOAT); + TEST_CHECK(val->type == MSGPACK_OBJECT_STR); + + msgpack_sbuffer_destroy(&sbuf); + msgpack_unpacked_destroy(&up);plugins/in_opentelemetry/opentelemetry_traces.c (2)
118-131: Float attribute accepts numeric strings — add minimal validation.strtod() will parse prefixes (e.g., "3.14xyz"). If stricter behavior is desired, validate with endptr and reject partials.
- value_double = strtod(value_str, NULL); + { + char *endp = NULL; + value_double = strtod(value_str, &endp); + if (endp == value_str || *endp != '\0') { + flb_sds_destroy(value_str); + flb_sds_destroy(key_str); + return -1; + } + }
96-109: Optional: factor out string→number parsing to a helper.Avoids duplication and centralizes validation (int64/double). I can provide a small static utility if useful.
Also applies to: 118-131
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (4)
plugins/in_opentelemetry/opentelemetry_traces.c(1 hunks)src/opentelemetry/flb_opentelemetry_utils.c(1 hunks)tests/internal/data/opentelemetry/test_cases.json(1 hunks)tests/internal/opentelemetry.c(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (2)
plugins/in_opentelemetry/opentelemetry_traces.c (2)
src/flb_sds.c (2)
flb_sds_create_len(58-76)flb_sds_destroy(389-399)lib/ctraces/src/ctr_attributes.c (1)
ctr_attributes_set_int64(68-71)
tests/internal/opentelemetry.c (1)
src/opentelemetry/flb_opentelemetry_utils.c (1)
flb_otel_utils_json_payload_get_wrapped_value(75-192)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (29)
- GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit (Arm64), amd64_arm64, -DCMAKE_SYSTEM_NAME=Windows -DCMA...
- GitHub Check: pr-windows-build / call-build-windows-package (Windows 32bit, x86, x86-windows-static, 3.31.6)
- GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit, x64, x64-windows-static, 3.31.6)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, gcc, g++)
- GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-22.04, clang-12)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_COVERAGE=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, gcc, g++)
- GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-24.04, clang-14)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_ARROW=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, clang, clang++)
- GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-22.04, clang-12)
- GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, clang, clang++)
- GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-24.04, clang-14)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, clang, clang++)
- GitHub Check: pr-compile-centos-7
- GitHub Check: PR - fuzzing test
🔇 Additional comments (4)
tests/internal/data/opentelemetry/test_cases.json (1)
1258-1280: Good regression case for intValue as string.Covers the resource attribute path and verifies numeric coercion to 1.
src/opentelemetry/flb_opentelemetry_utils.c (1)
114-122: Allowing intValue as string is correct and aligns with OTLP payloads.tests/internal/opentelemetry.c (1)
479-499: Unit test for intValue-as-string looks good.plugins/in_opentelemetry/opentelemetry_traces.c (1)
96-109: Nice: integer attribute accepts numeric strings.Good use of strtoll with i64 fallback when already numeric.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #10796
Updated OTLP JSON parsing to treat intValue and doubleValue wrappers containing either numeric or string values as numbers, ensuring attributes are unwrapped correctly during ingestion
Extended trace attribute processing to convert numeric strings into integers or doubles when constructing attributes, preventing malformed records for verbose OTLP payloads
Added regression tests and fixture cases validating integer attributes expressed as strings, guarding against regressions in wrapped attribute handling
Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.
Summary by CodeRabbit
New Features
Tests