sp: prevent converting from string to num if config is set

[nokute78]

This patch is to fix #2498 #5629
TODO: add test code.

This patch is to support following configuration.

Key	Description	Default
sp.convert_from_str_to_num	If enabled, Stream processor converts from number string to number type.	true

---

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

• Example configuration file for the change
• Debug log output from testing the change

• Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

• [N/A] Run local packaging test showing all targets (including any new ones) build.
• [N/A] Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

• Documentation required for this feature

Backporting

• [N/A] Backport to latest stable release.

Configuration

a.conf:

[SERVICE]
    Streams_File stream.conf
    Parsers_File parsers.conf
    sp.convert_from_str_to_num false

[INPUT]
    Name tail
    Path b.log
    Parser json
    Read_From_Head true

[OUTPUT]
    Name stdout
    Match test

stream.conf:

[STREAM_TASK]
    Name sample
    Exec CREATE STREAM test WITH (tag='test') AS SELECT word, num, COUNT(*) FROM STREAM:tail.0 WINDOW TUMBLING (1 SECOND) GROUP BY word, num;

b.log:

{"date": "22/abr/2019:12:43:51 -0600", "ip": "73.113.230.135", "word": "balsamine", "country": "Japan", "flag": false, "num": 96}
{"date": "22/abr/2019:12:43:52 -0600", "ip": "242.212.128.227", "word": "23456", "country": "Chile", "flag": false, "num": 15}
{"date": "22/abr/2019:12:43:52 -0600", "ip": "85.61.182.212", "word": "elicits", "country": "Argentina", "flag": true, "num": 73}

Debug/Valgrind output

Following error will be fixed by #7004

$ valgrind --leak-check=full bin/fluent-bit -c ~/git/WORKTREE/fix_5629/build/a.conf 
==47513== Memcheck, a memory error detector
==47513== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==47513== Using Valgrind-3.18.1 and LibVEX; rerun with -h for copyright info
==47513== Command: bin/fluent-bit -c /home/taka/git/WORKTREE/fix_5629/build/a.conf
==47513== 
Fluent Bit v2.1.0
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2023/03/12 14:18:40] [ info] [fluent bit] version=2.1.0, commit=e209745f5e, pid=47513
[2023/03/12 14:18:40] [ info] [storage] ver=1.4.0, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2023/03/12 14:18:40] [ info] [cmetrics] version=0.5.8
[2023/03/12 14:18:40] [ info] [ctraces ] version=0.3.0
[2023/03/12 14:18:40] [ info] [input:tail:tail.0] initializing
[2023/03/12 14:18:40] [ info] [input:tail:tail.0] storage_strategy='memory' (memory only)
[2023/03/12 14:18:40] [ info] [output:stdout:stdout.0] worker #0 started
[2023/03/12 14:18:40] [ info] [input:stream_processor:test] initializing
[2023/03/12 14:18:40] [ info] [input:stream_processor:test] storage_strategy=(null)
[2023/03/12 14:18:40] [ info] [sp] stream processor started
[2023/03/12 14:18:40] [ info] [sp] registered task: sample
[2023/03/12 14:18:40] [ info] [input:tail:tail.0] inotify_fs_add(): inode=4209349 watch_fd=1 name=b.log
[0] test: [1678598379.944006594, {""=>"balsamine", ""=>96, "COUNT(*)"=>1}]
[1] test: [1678598379.956341834, {""=>"23456", ""=>15, "COUNT(*)"=>1}]
[2] test: [1678598379.956385237, {""=>"elicits", ""=>73, "COUNT(*)"=>1}]
^C[2023/03/12 14:19:42] [engine] caught signal (SIGINT)
[2023/03/12 14:19:42] [ warn] [engine] service will shutdown in max 5 seconds
[2023/03/12 14:19:42] [ info] [input] pausing tail.0
[2023/03/12 14:19:42] [ info] [input] pausing test
[2023/03/12 14:19:42] [ info] [engine] service has stopped (0 pending tasks)
[2023/03/12 14:19:42] [ info] [input] pausing tail.0
[2023/03/12 14:19:42] [ info] [input] pausing test
[2023/03/12 14:19:42] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=4209349 watch_fd=1
[2023/03/12 14:19:43] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
[2023/03/12 14:19:43] [ info] [output:stdout:stdout.0] thread worker #0 stopped
==47513== 
==47513== HEAP SUMMARY:
==47513==     in use at exit: 172 bytes in 2 blocks
==47513==   total heap usage: 4,981 allocs, 4,979 frees, 3,419,150 bytes allocated
==47513== 
==47513== 23 bytes in 1 blocks are definitely lost in loss record 1 of 2
==47513==    at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==47513==    by 0x1B9657: flb_malloc (flb_mem.h:80)
==47513==    by 0x1B983A: sds_alloc (flb_sds.c:41)
==47513==    by 0x1B98C1: flb_sds_create_len (flb_sds.c:62)
==47513==    by 0x1B9967: flb_sds_create (flb_sds.c:88)
==47513==    by 0x242C87: flb_cf_section_property_get_string (flb_config_format.c:301)
==47513==    by 0xA5C272: sp_config_file (flb_sp.c:110)
==47513==    by 0xA5D787: flb_sp_create (flb_sp.c:704)
==47513==    by 0x2028C2: flb_engine_start (flb_engine.c:802)
==47513==    by 0x1A1B1F: flb_lib_worker (flb_lib.c:629)
==47513==    by 0x4FD7B42: start_thread (pthread_create.c:442)
==47513==    by 0x5068BB3: clone (clone.S:100)
==47513== 
==47513== 149 bytes in 1 blocks are definitely lost in loss record 2 of 2
==47513==    at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==47513==    by 0x1B9657: flb_malloc (flb_mem.h:80)
==47513==    by 0x1B983A: sds_alloc (flb_sds.c:41)
==47513==    by 0x1B98C1: flb_sds_create_len (flb_sds.c:62)
==47513==    by 0x1B9967: flb_sds_create (flb_sds.c:88)
==47513==    by 0x242C87: flb_cf_section_property_get_string (flb_config_format.c:301)
==47513==    by 0xA5C2E7: sp_config_file (flb_sp.c:117)
==47513==    by 0xA5D787: flb_sp_create (flb_sp.c:704)
==47513==    by 0x2028C2: flb_engine_start (flb_engine.c:802)
==47513==    by 0x1A1B1F: flb_lib_worker (flb_lib.c:629)
==47513==    by 0x4FD7B42: start_thread (pthread_create.c:442)
==47513==    by 0x5068BB3: clone (clone.S:100)
==47513== 
==47513== LEAK SUMMARY:
==47513==    definitely lost: 172 bytes in 2 blocks
==47513==    indirectly lost: 0 bytes in 0 blocks
==47513==      possibly lost: 0 bytes in 0 blocks
==47513==    still reachable: 0 bytes in 0 blocks
==47513==         suppressed: 0 bytes in 0 blocks
==47513== 
==47513== For lists of detected and suppressed errors, rerun with: -s
==47513== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)

---

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

[nokute78]

I added a test code and this PR is ready to review.

[koleini]

Curious to know what is the behavior of other stream processors in such cases (strings that may represent a number).

[nokute78]

I'm not familiar with other stream processors.
In my opinion, it would be better to temporarily convert the type to check condition, but eventually revert to the initial type.

[github-actions]

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.

[nokute78]

Doc PR is fluent/fluent-bit-docs#1204