Skip to content

sds: general fixes for sds_cat() usage - #11805

Merged
edsiper merged 7 commits into
masterfrom
checks-and-fixes
May 15, 2026
Merged

sds: general fixes for sds_cat() usage#11805
edsiper merged 7 commits into
masterfrom
checks-and-fixes

Conversation

@edsiper

@edsiper edsiper commented May 14, 2026

Copy link
Copy Markdown
Member

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Summary by CodeRabbit

  • Bug Fixes

    • Improved error handling and allocation-failure recovery across Elasticsearch, Azure, BigQuery, Stackdriver and other outputs, preventing crashes and ensuring cleaner cleanup.
    • Ensured bulk/response assembly emits consistent status entries, including an explicit unknown write-op status.
  • Tests

    • Broadened integration test expectations to accommodate additional exit codes in constrained buffer scenarios.

Review Change Stack

@coderabbitai

coderabbitai Bot commented May 14, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b05a63d1-eb1f-4d3b-8299-c3611df4d516

📥 Commits

Reviewing files that changed from the base of the PR and between 75f9507 and f9c3dcc.

📒 Files selected for processing (11)
  • plugins/in_elasticsearch/in_elasticsearch_bulk_prot.c
  • plugins/in_node_exporter_metrics/ne_meminfo_linux.c
  • plugins/out_azure/azure.c
  • plugins/out_azure/azure_conf.c
  • plugins/out_azure_kusto/azure_kusto.c
  • plugins/out_azure_kusto/azure_msiauth.c
  • plugins/out_bigquery/bigquery.c
  • plugins/out_stackdriver/gce_metadata.c
  • plugins/out_stackdriver/stackdriver.c
  • plugins/out_stackdriver/stackdriver_conf.c
  • tests/integration/scenarios/in_elasticsearch/tests/test_in_elasticsearch_001.py
🚧 Files skipped from review as they are similar to previous changes (10)
  • tests/integration/scenarios/in_elasticsearch/tests/test_in_elasticsearch_001.py
  • plugins/out_bigquery/bigquery.c
  • plugins/out_azure_kusto/azure_msiauth.c
  • plugins/out_stackdriver/gce_metadata.c
  • plugins/out_stackdriver/stackdriver.c
  • plugins/out_azure/azure_conf.c
  • plugins/out_azure/azure.c
  • plugins/out_azure_kusto/azure_kusto.c
  • plugins/in_elasticsearch/in_elasticsearch_bulk_prot.c
  • plugins/out_stackdriver/stackdriver_conf.c

📝 Walkthrough

Walkthrough

Protects meminfo metric-name construction by using safe SDS concatenation and explicit null/return checks, with proper cleanup of temporaries and lists on allocation/concat failure.

Changes

Node Exporter meminfo safe concatenation

Layer / File(s) Summary
Configure metric-name concatenation
plugins/in_node_exporter_metrics/ne_meminfo_linux.c
meminfo_configure now uses flb_sds_cat_safe and checks its return; on failure it destroys split_list and goes to the existing error path.
Update 3-part metric-name composition
plugins/in_node_exporter_metrics/ne_meminfo_linux.c
meminfo_update checks flb_sds_create_size and validates each flb_sds_cat_safe call when composing the base + "_bytes" metric name; failures clean up tmp, split_list, and list and return -1.

🎯 2 (Simple) | ⏱️ ~10 minutes

🐰 I stitched the names with gentle paws,
Checked every stitch and every clause,
If memory strings cannot be made,
I tidy up and softly fade,
Safe and neat in fluent claws.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 10.53% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: adding defensive checks and error handling around sds_cat() usage across multiple plugins.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch checks-and-fixes

Warning

Review ran into problems

🔥 Problems

Stopped waiting for pipeline failures after 30000ms. One of your pipelines takes longer than our 30000ms fetch window to run, so review may not consider pipeline-failure results for inline comments if any failures occurred after the fetch window. Increase the timeout if you want to wait longer or run a @coderabbit review after the pipeline has finished.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@edsiper edsiper added this to the Fluent Bit v5.0.6 milestone May 14, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
plugins/in_node_exporter_metrics/ne_meminfo_linux.c (1)

260-263: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Add return value checks to flb_sds_cat_safe calls in meminfo_update.

Lines 261-262 do not validate the return values from flb_sds_cat_safe, but identical concatenations in meminfo_configure (lines 100, 108, 134) check for errors. If either concatenation fails, tmp becomes invalid, causing undefined behavior when passed to flb_hash_table_get. Add error checks consistent with the rest of the file:

Suggested fix
         else if (parts == 3) {
             /* Compose new metric name */
             tmp = flb_sds_create_size(256);
-            flb_sds_cat_safe(&tmp, metric_name, flb_sds_len(metric_name) - 1);
-            flb_sds_cat_safe(&tmp, "_bytes", 6);
+            ret = flb_sds_cat_safe(&tmp, metric_name, flb_sds_len(metric_name) - 1);
+            if (ret != 0) {
+                flb_sds_destroy(tmp);
+                flb_slist_destroy(&split_list);
+                continue;
+            }
+            ret = flb_sds_cat_safe(&tmp, "_bytes", 6);
+            if (ret != 0) {
+                flb_sds_destroy(tmp);
+                flb_slist_destroy(&split_list);
+                continue;
+            }
 
             /* Get metric context */
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@plugins/in_node_exporter_metrics/ne_meminfo_linux.c` around lines 260 - 263,
In meminfo_update, the two flb_sds_cat_safe calls that append to tmp (the
flb_sds created with flb_sds_create_size) need their return values checked like
in meminfo_configure; if flb_sds_cat_safe returns FLB_SDS_ERROR (or NULL/invalid
per project convention) free tmp and return an error (or propagate failure)
instead of continuing to call flb_hash_table_get with an invalid tmp. Update the
flb_sds_cat_safe invocations in meminfo_update to validate the result, clean up
tmp on failure, and return early to match the error handling used in
meminfo_configure.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@plugins/in_elasticsearch/in_elasticsearch_bulk_prot.c`:
- Around line 173-180: Before appending a comma to bulk_statuses in the loop
(the call to bulk_statuses_cat when idx > 0 && idx % 2 == 0), ensure there is
enough space for both the comma and the minimum following status payload by
calling status_buffer_avail(ctx, *bulk_statuses, 51) (or equivalent threshold)
first; only call bulk_statuses_cat(...) if that pre-check returns FLB_TRUE,
otherwise break to avoid leaving a trailing comma. This change touches the
comma-insertion branch (the idx/%2 check and bulk_statuses_cat call) and the
subsequent status_buffer_avail check so order and threshold are updated to
prevent producing ",]" in constrained-buffer cases.

In `@plugins/out_bigquery/bigquery.c`:
- Around line 140-148: The code calls bigquery_jwt_base64_url_encode(..., &olen)
and immediately uses olen/buf without checking the helper's return value; modify
both call sites (the block using variables buf, buf_size, payload, len, olen,
tmp, out) to capture the helper's return (e.g., rc =
bigquery_jwt_base64_url_encode(..., &olen)) and if rc == -1 fail fast by jumping
to the existing error cleanup path (goto error) instead of using a stale olen or
appending buf; ensure the same change is applied to the other occurrence around
lines 185-197 so both paths consistently validate the encoder result before
calling flb_sds_cat.

In `@plugins/out_stackdriver/stackdriver.c`:
- Around line 293-303: The code calls jwt_base64_url_encode((unsigned char *)
buf, buf_size, (unsigned char *) payload, len, &olen) but ignores its return
value and proceeds to use olen when appending via flb_sds_cat, which can use a
stale olen on failure; update the blocks around jwt_base64_url_encode (both
occurrences) to check the function's return (e.g., rc or ret) for failure, jump
to the existing error cleanup label if encoding failed, and only call
flb_sds_cat(out, buf, olen) and assign tmp = ... when the encode call succeeded;
reference the jwt_base64_url_encode, olen, buf, buf_size, payload, flb_sds_cat,
out, tmp, and the error label to locate and implement the change.

---

Outside diff comments:
In `@plugins/in_node_exporter_metrics/ne_meminfo_linux.c`:
- Around line 260-263: In meminfo_update, the two flb_sds_cat_safe calls that
append to tmp (the flb_sds created with flb_sds_create_size) need their return
values checked like in meminfo_configure; if flb_sds_cat_safe returns
FLB_SDS_ERROR (or NULL/invalid per project convention) free tmp and return an
error (or propagate failure) instead of continuing to call flb_hash_table_get
with an invalid tmp. Update the flb_sds_cat_safe invocations in meminfo_update
to validate the result, clean up tmp on failure, and return early to match the
error handling used in meminfo_configure.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 8025c4f6-e3e2-4572-99ab-3dbec5fe115e

📥 Commits

Reviewing files that changed from the base of the PR and between 5a77b1c and 75f9507.

📒 Files selected for processing (11)
  • plugins/in_elasticsearch/in_elasticsearch_bulk_prot.c
  • plugins/in_node_exporter_metrics/ne_meminfo_linux.c
  • plugins/out_azure/azure.c
  • plugins/out_azure/azure_conf.c
  • plugins/out_azure_kusto/azure_kusto.c
  • plugins/out_azure_kusto/azure_msiauth.c
  • plugins/out_bigquery/bigquery.c
  • plugins/out_stackdriver/gce_metadata.c
  • plugins/out_stackdriver/stackdriver.c
  • plugins/out_stackdriver/stackdriver_conf.c
  • tests/integration/scenarios/in_elasticsearch/tests/test_in_elasticsearch_001.py

Comment thread plugins/in_elasticsearch/in_elasticsearch_bulk_prot.c
Comment thread plugins/out_bigquery/bigquery.c Outdated
Comment thread plugins/out_stackdriver/stackdriver.c
edsiper added 7 commits May 14, 2026 15:27
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
@edsiper
edsiper force-pushed the checks-and-fixes branch from 75f9507 to f9c3dcc Compare May 14, 2026 21:34
@edsiper
edsiper merged commit 8627815 into master May 15, 2026
47 of 50 checks passed
@edsiper
edsiper deleted the checks-and-fixes branch May 15, 2026 00:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant