Skip to content

Conversation

@nokute78
Copy link
Collaborator

This patch is to add missing releasing function.


Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • [N/A] Example configuration file for the change
  • [N/A] Debug log output from testing the change
  • [N/A] Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • [N/A] Run local packaging test showing all targets (including any new ones) build.
  • [N/A] Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • [N/A] Documentation required for this feature

Backporting

  • [N/A] Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

@nokute78 nokute78 temporarily deployed to pr August 27, 2023 11:42 — with GitHub Actions Inactive
@nokute78 nokute78 temporarily deployed to pr August 27, 2023 11:42 — with GitHub Actions Inactive
@nokute78 nokute78 temporarily deployed to pr August 27, 2023 11:42 — with GitHub Actions Inactive
@nokute78 nokute78 temporarily deployed to pr August 27, 2023 12:07 — with GitHub Actions Inactive
@edsiper edsiper added this to the Fluent Bit v2.1.9 milestone Aug 29, 2023
@edsiper
Copy link
Member

edsiper commented Aug 29, 2023

@nokute78 thanks for this patch.

I see that the context is registered right after creation here:

flb_input_set_context(ins, ctx);

but if the plugin cannot be initialized, in theory, it should be released, have you checked with Valgrind if is there a real leak ?

@nokute78
Copy link
Collaborator Author

nokute78 commented Sep 1, 2023

@edsiper I tested 2 cases.
Valgrind reported struct nginx_ctx is leaked.

1. After flb_input_set_context

I added following patch for v2.1.8 branch.

diff --git a/plugins/in_nginx_exporter_metrics/nginx.c b/plugins/in_nginx_exporter_metrics/nginx.c
index 9645fca8e..e5f985cfd 100644
--- a/plugins/in_nginx_exporter_metrics/nginx.c
+++ b/plugins/in_nginx_exporter_metrics/nginx.c
@@ -1595,6 +1595,8 @@ static int nginx_init(struct flb_input_instance *ins,
 
     flb_input_set_context(ins, ctx);
 
+    return -1;
+
     if (ctx->is_nginx_plus == FLB_FALSE) {
         /* These metrics follow the same format as those define here:
          * https://github.com/nginxinc/nginx-prometheus-exporter#metrics-for-nginx-oss

Valgrind output is

$ valgrind --leak-check=full bin/fluent-bit -i nginx_metrics -o stdout
==15380== Memcheck, a memory error detector
==15380== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==15380== Using Valgrind-3.18.1 and LibVEX; rerun with -h for copyright info
==15380== Command: bin/fluent-bit -i nginx_metrics -o stdout
==15380== 
Fluent Bit v2.1.8
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2023/09/02 07:29:59] [ info] [fluent bit] version=2.1.8, commit=1d83649441, pid=15380
[2023/09/02 07:29:59] [ info] [storage] ver=1.2.0, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2023/09/02 07:29:59] [ info] [cmetrics] version=0.6.3
[2023/09/02 07:29:59] [ info] [ctraces ] version=0.3.1
[2023/09/02 07:29:59] [ info] [input:nginx_metrics:nginx_metrics.0] initializing
[2023/09/02 07:29:59] [ info] [input:nginx_metrics:nginx_metrics.0] storage_strategy='memory' (memory only)
[2023/09/02 07:29:59] [error] failed initialize input nginx_metrics.0
[2023/09/02 07:29:59] [error] [engine] input initialization failed
[2023/09/02 07:29:59] [error] [lib] backend failed
==15380== 
==15380== HEAP SUMMARY:
==15380==     in use at exit: 713 bytes in 8 blocks
==15380==   total heap usage: 1,185 allocs, 1,177 frees, 222,107 bytes allocated
==15380== 
==15380== 7 bytes in 1 blocks are definitely lost in loss record 1 of 8
==15380==    at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==15380==    by 0x1C3E60: flb_malloc (flb_mem.h:80)
==15380==    by 0x1C414A: flb_strndup (flb_str.h:34)
==15380==    by 0x1C41B1: flb_strdup (flb_str.h:46)
==15380==    by 0x1C681C: flb_main (fluent-bit.c:928)
==15380==    by 0x1C7593: main (fluent-bit.c:1358)
==15380== 
==15380== 706 (192 direct, 514 indirect) bytes in 1 blocks are definitely lost in loss record 8 of 8
==15380==    at 0x484DA83: calloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==15380==    by 0x4D89F9: flb_calloc (flb_mem.h:95)
==15380==    by 0x4E3D3D: nginx_ctx_init (nginx.c:1519)
==15380==    by 0x4E401E: nginx_init (nginx.c:1590)
==15380==    by 0x1FD551: flb_input_instance_init (flb_input.c:1189)
==15380==    by 0x1FD714: flb_input_init_all (flb_input.c:1248)
==15380==    by 0x242B41: flb_engine_start (flb_engine.c:770)
==15380==    by 0x1DD944: flb_lib_worker (flb_lib.c:638)
==15380==    by 0x4FF3B42: start_thread (pthread_create.c:442)
==15380==    by 0x5084BB3: clone (clone.S:100)
==15380== 
==15380== LEAK SUMMARY:
==15380==    definitely lost: 199 bytes in 2 blocks
==15380==    indirectly lost: 514 bytes in 6 blocks
==15380==      possibly lost: 0 bytes in 0 blocks
==15380==    still reachable: 0 bytes in 0 blocks
==15380==         suppressed: 0 bytes in 0 blocks
==15380== 
==15380== For lists of detected and suppressed errors, rerun with: -s
==15380== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)

2. Failed at nginx_ctx_init

I added following patch for v2.1.8 branch.

diff --git a/plugins/in_nginx_exporter_metrics/nginx.c b/plugins/in_nginx_exporter_metrics/nginx.c
index 9645fca8e..57d49d2e4 100644
--- a/plugins/in_nginx_exporter_metrics/nginx.c
+++ b/plugins/in_nginx_exporter_metrics/nginx.c
@@ -1547,6 +1547,8 @@ struct nginx_ctx *nginx_ctx_init(struct flb_input_instance *ins,
 
     upstream = flb_upstream_create(config, ins->host.name, ins->host.port,
                                    upstream_flags, ins->tls);
+    flb_upstream_destroy(upstream);
+    return NULL;
 
     if (!upstream) {
         flb_plg_error(ins, "upstream initialization error");

Valgrind output is

$ valgrind --leak-check=full bin/fluent-bit -i nginx_metrics -o stdout
==18193== Memcheck, a memory error detector
==18193== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==18193== Using Valgrind-3.18.1 and LibVEX; rerun with -h for copyright info
==18193== Command: bin/fluent-bit -i nginx_metrics -o stdout
==18193== 
Fluent Bit v2.1.8
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2023/09/02 07:43:04] [ info] [fluent bit] version=2.1.8, commit=1d83649441, pid=18193
[2023/09/02 07:43:04] [ info] [storage] ver=1.2.0, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2023/09/02 07:43:04] [ info] [cmetrics] version=0.6.3
[2023/09/02 07:43:04] [ info] [ctraces ] version=0.3.1
[2023/09/02 07:43:04] [ info] [input:nginx_metrics:nginx_metrics.0] initializing
[2023/09/02 07:43:04] [ info] [input:nginx_metrics:nginx_metrics.0] storage_strategy='memory' (memory only)
[2023/09/02 07:43:04] [error] failed initialize input nginx_metrics.0
[2023/09/02 07:43:04] [error] [engine] input initialization failed
[2023/09/02 07:43:04] [error] [lib] backend failed
==18193== 
==18193== HEAP SUMMARY:
==18193==     in use at exit: 383 bytes in 6 blocks
==18193==   total heap usage: 1,185 allocs, 1,179 frees, 222,107 bytes allocated
==18193== 
==18193== 7 bytes in 1 blocks are definitely lost in loss record 1 of 6
==18193==    at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==18193==    by 0x1C3E60: flb_malloc (flb_mem.h:80)
==18193==    by 0x1C414A: flb_strndup (flb_str.h:34)
==18193==    by 0x1C41B1: flb_strdup (flb_str.h:46)
==18193==    by 0x1C681C: flb_main (fluent-bit.c:928)
==18193==    by 0x1C7593: main (fluent-bit.c:1358)
==18193== 
==18193== 376 (192 direct, 184 indirect) bytes in 1 blocks are definitely lost in loss record 6 of 6
==18193==    at 0x484DA83: calloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==18193==    by 0x4D89F9: flb_calloc (flb_mem.h:95)
==18193==    by 0x4E3D3D: nginx_ctx_init (nginx.c:1519)
==18193==    by 0x4E3F9A: nginx_init (nginx.c:1592)
==18193==    by 0x1FD551: flb_input_instance_init (flb_input.c:1189)
==18193==    by 0x1FD714: flb_input_init_all (flb_input.c:1248)
==18193==    by 0x242B41: flb_engine_start (flb_engine.c:770)
==18193==    by 0x1DD944: flb_lib_worker (flb_lib.c:638)
==18193==    by 0x4FF3B42: start_thread (pthread_create.c:442)
==18193==    by 0x5084BB3: clone (clone.S:100)
==18193== 
==18193== LEAK SUMMARY:
==18193==    definitely lost: 199 bytes in 2 blocks
==18193==    indirectly lost: 184 bytes in 4 blocks
==18193==      possibly lost: 0 bytes in 0 blocks
==18193==    still reachable: 0 bytes in 0 blocks
==18193==         suppressed: 0 bytes in 0 blocks
==18193== 
==18193== For lists of detected and suppressed errors, rerun with: -s
==18193== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)

@edsiper edsiper merged commit f7731f3 into fluent:master Sep 2, 2023
@edsiper
Copy link
Member

edsiper commented Sep 2, 2023

thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants