Skip to content

tests: runtime: processor_content_modifier: add test code#8591

Merged
edsiper merged 2 commits intofluent:masterfrom
nokute78:test_processor_content_modifer
Mar 17, 2024
Merged

tests: runtime: processor_content_modifier: add test code#8591
edsiper merged 2 commits intofluent:masterfrom
nokute78:test_processor_content_modifer

Conversation

@nokute78
Copy link
Contributor

@nokute78 nokute78 commented Mar 16, 2024

This patch is to

  • Add test code for "logs" using processor_content_modifier
  • Fix not freed memory issue for processor_content_modifier

Note: Test code for metrics and traces are not implemented.

Memory issue is

==66962== HEAP SUMMARY:
==66962==     in use at exit: 105 bytes in 2 blocks
==66962==   total heap usage: 12,619 allocs, 12,617 frees, 4,267,696 bytes allocated
==66962== 
==66962== 105 (24 direct, 81 indirect) bytes in 1 blocks are definitely lost in loss record 2 of 2
==66962==    at 0x484DA83: calloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==66962==    by 0x96CE2F: cfl_variant_create (cfl_variant.c:242)
==66962==    by 0x96CB1A: cfl_variant_create_from_string_s (cfl_variant.c:94)
==66962==    by 0x53D9A7: cfl_variant_convert (cm_logs.c:182)
==66962==    by 0x53D6AB: hash_transformer (cm_logs.c:79)
==66962==    by 0x53E234: run_action_hash (cm_logs.c:449)
==66962==    by 0x53E6A9: cm_logs_process (cm_logs.c:584)
==66962==    by 0x534BE2: cb_process_logs (cm.c:78)
==66962==    by 0x214BC4: flb_processor_run (flb_processor.c:570)
==66962==    by 0x28F7EC: input_log_append (flb_input_log.c:51)
==66962==    by 0x28F909: flb_input_log_append (flb_input_log.c:90)
==66962==    by 0x51511D: in_lib_collect (in_lib.c:162)
==66962== 
==66962== LEAK SUMMARY:
==66962==    definitely lost: 24 bytes in 1 blocks
==66962==    indirectly lost: 81 bytes in 1 blocks
==66962==      possibly lost: 0 bytes in 0 blocks
==66962==    still reachable: 0 bytes in 0 blocks
==66962==         suppressed: 0 bytes in 0 blocks
==66962== 

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • [N/A] Example configuration file for the change
  • Debug log output from testing the change
  • Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • [N/A] Run local packaging test showing all targets (including any new ones) build.
  • [N/A] Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • [N/A] Documentation required for this feature

Backporting

  • [N/A] Backport to latest stable release.

Debug/Valgrind output

$ valgrind --leak-check=full bin/flb-rt-processor_content_modifier 
==32959== Memcheck, a memory error detector
==32959== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==32959== Using Valgrind-3.18.1 and LibVEX; rerun with -h for copyright info
==32959== Command: bin/flb-rt-processor_content_modifier
==32959== 
Test logs.action.insert...                      [ OK ]
Test logs.action.delete...                      [ OK ]
Test logs.action.rename...                      [ OK ]
Test logs.action.upsert...                      [ OK ]
Test logs.action.hash...                        [ OK ]
Test logs.action.extract...                     [ OK ]
Test logs.action.convert_from_string_to_int...  [ OK ]
Test logs.action.convert_from_int_to_string...  [ OK ]
Test logs.action.convert_from_string_to_double... [ OK ]
Test logs.action.convert_from_double_to_string... [ OK ]
Test logs.action.convert_from_string_to_boolean... [ OK ]
SUCCESS: All unit tests have passed.
==32959== 
==32959== HEAP SUMMARY:
==32959==     in use at exit: 0 bytes in 0 blocks
==32959==   total heap usage: 21,979 allocs, 21,979 frees, 8,949,295 bytes allocated
==32959== 
==32959== All heap blocks were freed -- no leaks are possible
==32959== 
==32959== For lists of detected and suppressed errors, rerun with: -s
==32959== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

@nokute78
Copy link
Contributor Author

nokute78 commented Mar 16, 2024

Hmm processor_content_modifier doesn't support some conversion rules and can't test them now.
e.g. boolean -> string , positive integer -> string.

I think it is better to use/enhance flb_typecast.c to unify conversion rules.

I raised an issue #8592

Signed-off-by: Takahiro Yamashita <nokute78@gmail.com>
Signed-off-by: Takahiro Yamashita <nokute78@gmail.com>
@edsiper
Copy link
Member

edsiper commented Mar 17, 2024

thank you

@nokute78 nokute78 deleted the test_processor_content_modifer branch March 17, 2024 02:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants