Skip to content

tests: runtime: plugins: Enable to build and run runtime testcases on windows - #12092

Merged
edsiper merged 54 commits into
masterfrom
cosmo0920-enable-runtime-testcases-on-windows
Jul 13, 2026
Merged

tests: runtime: plugins: Enable to build and run runtime testcases on windows#12092
edsiper merged 54 commits into
masterfrom
cosmo0920-enable-runtime-testcases-on-windows

Conversation

@cosmo0920

@cosmo0920 cosmo0920 commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Verification

Runtime testing is enabled on Windows.

The complete runtime test suite was executed, rather than only the tests related
to the initial changes. This broader verification exposed that the
runtime_shell cases depended on Bash scripts and therefore could not run
natively on Windows.

The affected shell test runners were converted to PowerShell. The Bash and
PowerShell runners use the same external configuration files; Fluent Bit
configuration content is not embedded in the PowerShell scripts.

Final results:

  • Runtime test binaries: 84/84 passed.
  • Runtime shell tests: 7/7 passed.
  • flb-rt-in_event_test: passed, including five consecutive executions.
  • out_exit: built and tested with its Windows default enabled.
  • git diff --check: passed.
  • Valgrind was not run because it is unavailable for native Windows binaries.
  • YAML runtime-shell cases remained skipped because libyaml was unavailable on separated testing build directory.
  • Unix-domain-socket cases remained excluded because they are not supported by
    the Windows test path.

Log

PS> ctest -j20 . --output-on-failure
Test project C:/Users/cosmo/Documents/GitHub/fluent-bit/build
        Start  36: flb-rt-filter_kubernetes
        Start  71: flb-rt-out_stackdriver
        Start  76: flb-rt-out_s3
        Start  19: flb-rt-in_tail
        Start  33: flb-rt-filter_throttle_size
        Start  65: flb-rt-out_loki
        Start  38: flb-rt-filter_modify
        Start  69: flb-rt-out_syslog
        Start  12: flb-rt-in_http
        Start  11: flb-rt-in_dummy
        Start  73: flb-rt-out_firehose
        Start  74: flb-rt-out_kinesis
        Start  42: flb-rt-filter_multiline
        Start  58: flb-rt-out_opensearch
        Start  17: flb-rt-in_splunk
        Start  82: flb-rt-processor_content_modifier
        Start  39: flb-rt-filter_lua
        Start  72: flb-rt-out_cloudwatch
        Start  13: flb-rt-in_elasticsearch
        Start   2: flb-rt-custom_calyptia_registration_retry_test
  1/164 Test  #13: flb-rt-in_elasticsearch ..........................   Passed   43.73 sec
        Start  61: flb-rt-out_http
  2/164 Test  #76: flb-rt-out_s3 ....................................   Passed   55.49 sec
  3/164 Test   #2: flb-rt-custom_calyptia_registration_retry_test ...   Passed   43.83 sec
        Start  27: flb-rt-filter_aws
        Start  37: flb-rt-filter_parser
  4/164 Test  #72: flb-rt-out_cloudwatch ............................   Passed   46.78 sec
  5/164 Test  #39: flb-rt-filter_lua ................................   Passed   47.49 sec
        Start  70: flb-rt-out_tcp
        Start  59: flb-rt-out_exit
  6/164 Test  #82: flb-rt-processor_content_modifier ................   Passed   49.86 sec
        Start  46: flb-rt-filter_log_to_metrics
  7/164 Test  #17: flb-rt-in_splunk .................................   Passed   51.18 sec
        Start  22: flb-rt-in_forward
  8/164 Test  #58: flb-rt-out_opensearch ............................   Passed   52.68 sec
        Start  64: flb-rt-out_logdna
  9/164 Test  #42: flb-rt-filter_multiline ..........................   Passed   53.90 sec
        Start  35: flb-rt-filter_rewrite_tag
 10/164 Test  #73: flb-rt-out_firehose ..............................   Passed   55.96 sec
        Start 154: flb-it-stream_processor
 11/164 Test  #74: flb-rt-out_kinesis ...............................   Passed   55.91 sec
        Start  18: flb-rt-in_syslog
 12/164 Test  #11: flb-rt-in_dummy ..................................   Passed   63.96 sec
        Start  31: flb-rt-filter_grep
 13/164 Test  #12: flb-rt-in_http ...................................   Passed   65.53 sec
        Start  14: flb-rt-in_opentelemetry
 14/164 Test  #69: flb-rt-out_syslog ................................   Passed   68.35 sec
        Start 162: flb-it-error_reporter
 15/164 Test  #38: flb-rt-filter_modify .............................   Passed   77.30 sec
        Start 131: flb-it-log
 16/164 Test #154: flb-it-stream_processor ..........................   Passed   19.59 sec
        Start  25: flb-rt-in_kubernetes_events
 17/164 Test  #65: flb-rt-out_loki ..................................   Passed   83.33 sec
        Start  20: flb-rt-in_udp
 18/164 Test  #64: flb-rt-out_logdna ................................   Passed   31.14 sec
        Start  77: flb-rt-out_s3_otlp_json
 19/164 Test  #22: flb-rt-in_forward ................................   Passed   32.65 sec
 20/164 Test  #35: flb-rt-filter_rewrite_tag ........................   Passed   31.42 sec
        Start  49: flb-rt-core_engine
        Start  45: flb-rt-filter_ecs
 21/164 Test  #18: flb-rt-in_syslog .................................   Passed   31.31 sec
 22/164 Test  #59: flb-rt-out_exit ..................................   Passed   35.20 sec
 23/164 Test  #46: flb-rt-filter_log_to_metrics .....................   Passed   34.45 sec
        Start  78: flb-rt-out_influxdb
        Start  80: flb-rt-processor_metrics_selector
        Start  28: flb-rt-filter_checklist
 24/164 Test #162: flb-it-error_reporter ............................   Passed   23.32 sec
        Start  16: flb-rt-in_statsd
 25/164 Test  #37: flb-rt-filter_parser .............................   Passed   39.31 sec
 26/164 Test  #27: flb-rt-filter_aws ................................   Passed   39.93 sec
 27/164 Test  #61: flb-rt-out_http ..................................   Passed   40.81 sec
        Start  75: flb-rt-group_counter_semantics
        Start  79: flb-rt-out_chronicle
        Start  41: flb-rt-filter_record_modifier
 28/164 Test  #14: flb-rt-in_opentelemetry ..........................   Passed   27.97 sec
        Start  60: flb-rt-out_flowcounter
 29/164 Test  #31: flb-rt-filter_grep ...............................   Passed   29.32 sec
        Start  29: flb-rt-filter_expect
 30/164 Test  #70: flb-rt-out_tcp ...................................   Passed   41.59 sec
        Start  21: flb-rt-in_tcp
 31/164 Test #131: flb-it-log .......................................   Passed   20.97 sec
        Start  40: flb-rt-filter_type_converter
 32/164 Test  #25: flb-rt-in_kubernetes_events ......................   Passed   23.32 sec
        Start  63: flb-rt-out_lib
 33/164 Test  #33: flb-rt-filter_throttle_size ......................   Passed  105.97 sec
        Start  48: flb-rt-counter_parity_e2e
 34/164 Test  #20: flb-rt-in_udp ....................................   Passed   23.18 sec
        Start  44: flb-rt-filter_wasm
 35/164 Test  #41: flb-rt-filter_record_modifier ....................   Passed   16.40 sec
        Start  81: flb-rt-processor_labels
 36/164 Test  #79: flb-rt-out_chronicle .............................   Passed   17.51 sec
 37/164 Test  #16: flb-rt-in_statsd .................................   Passed   18.74 sec
        Start   9: flb-rt-core_chunk_trace
        Start  92: processor_conditional.ps1
 38/164 Test  #60: flb-rt-out_flowcounter ...........................   Passed   16.98 sec
 39/164 Test  #75: flb-rt-group_counter_semantics ...................   Passed   18.79 sec
 40/164 Test  #77: flb-rt-out_s3_otlp_json ..........................   Passed   23.29 sec
 41/164 Test  #29: flb-rt-filter_expect .............................   Passed   16.40 sec
 42/164 Test  #28: flb-rt-filter_checklist ..........................   Passed   20.18 sec
 43/164 Test  #49: flb-rt-core_engine ...............................   Passed   22.49 sec
        Start   5: flb-rt-core-timeout
        Start  10: flb-rt-in_event_test
        Start  34: flb-rt-filter_nest
        Start 122: flb-it-flb_event_loop
        Start 144: flb-it-opentelemetry
        Start  83: flb-rt-processor_cumulative_to_delta
 44/164 Test  #45: flb-rt-filter_ecs ................................   Passed   25.40 sec
 45/164 Test  #80: flb-rt-processor_metrics_selector ................   Passed   24.13 sec
 46/164 Test  #78: flb-rt-out_influxdb ..............................   Passed   24.66 sec
 47/164 Test  #40: flb-rt-filter_type_converter .....................   Passed   16.84 sec
        Start  62: flb-rt-out_kafka
        Start  50: flb-rt-core_log
        Start  30: flb-rt-filter_stdout
        Start  43: flb-rt-filter_sysinfo
 48/164 Test #144: flb-it-opentelemetry .............................   Passed    3.32 sec
 49/164 Test  #36: flb-rt-filter_kubernetes .........................   Passed  120.99 sec
 50/164 Test  #63: flb-rt-out_lib ...................................   Passed   15.76 sec
        Start  32: flb-rt-filter_throttle
        Start  68: flb-rt-out_stdout
        Start  55: flb-rt-out_azure_kusto
 51/164 Test  #48: flb-rt-counter_parity_e2e ........................   Passed   13.80 sec
        Start  88: in_http_tls_expect.ps1
 52/164 Test  #30: flb-rt-filter_stdout .............................   Passed    3.59 sec
        Start  67: flb-rt-out_splunk
 53/164 Test  #44: flb-rt-filter_wasm ...............................   Passed   14.57 sec
        Start  66: flb-rt-out_null
 54/164 Test #122: flb-it-flb_event_loop ............................   Passed    9.20 sec
        Start  53: flb-rt-out_counter
 55/164 Test   #5: flb-rt-core-timeout ..............................   Passed   11.57 sec
        Start  89: in_syslog_tcp_tls_expect.ps1
 56/164 Test  #43: flb-rt-filter_sysinfo ............................   Passed    6.74 sec
        Start  56: flb-rt-out_datadog
 57/164 Test  #34: flb-rt-filter_nest ...............................   Passed   11.07 sec
 58/164 Test  #62: flb-rt-out_kafka .................................   Passed    8.63 sec
 59/164 Test  #81: flb-rt-processor_labels ..........................   Passed   13.38 sec
 60/164 Test  #32: flb-rt-filter_throttle ...........................   Passed    6.61 sec
 61/164 Test   #9: flb-rt-core_chunk_trace ..........................   Passed   12.81 sec
        Start  57: flb-rt-out_skywalking
        Start 149: flb-it-oauth2
        Start  86: in_syslog_tcp_plaintext_expect.ps1
        Start  87: in_syslog_udp_plaintext_expect.ps1
        Start  23: flb-rt-in_fluentbit_metrics
 62/164 Test  #92: processor_conditional.ps1 ........................   Passed   13.92 sec
 63/164 Test  #88: in_http_tls_expect.ps1 ...........................   Passed    7.01 sec
        Start  15: flb-rt-in_random
        Start 130: flb-it-env
 64/164 Test  #10: flb-rt-in_event_test .............................   Passed   14.33 sec
 65/164 Test  #83: flb-rt-processor_cumulative_to_delta .............   Passed   12.24 sec
 66/164 Test  #50: flb-rt-core_log ..................................   Passed   10.96 sec
 67/164 Test  #68: flb-rt-out_stdout ................................   Passed    8.94 sec
 68/164 Test  #55: flb-rt-out_azure_kusto ...........................   Passed    8.49 sec
        Start  26: flb-rt-in_opentelemetry_routing
        Start 118: flb-it-multiline
        Start 158: flb-it-cumulative_to_delta
        Start  85: in_tail_expect.ps1
        Start  47: flb-rt-filter_counter_semantics
 69/164 Test  #67: flb-rt-out_splunk ................................   Passed    9.07 sec
 70/164 Test  #66: flb-rt-out_null ..................................   Passed    8.30 sec
        Start  84: in_dummy_expect.ps1
        Start  24: flb-rt-in_prometheus_textfile
 71/164 Test  #56: flb-rt-out_datadog ...............................   Passed    6.18 sec
 72/164 Test #149: flb-it-oauth2 ....................................   Passed    5.21 sec
 73/164 Test  #57: flb-rt-out_skywalking ............................   Passed    5.69 sec
 74/164 Test  #53: flb-rt-out_counter ...............................   Passed    7.04 sec
 75/164 Test  #21: flb-rt-in_tcp ....................................   Passed   33.46 sec
 76/164 Test  #89: in_syslog_tcp_tls_expect.ps1 .....................   Passed    6.19 sec
 77/164 Test  #86: in_syslog_tcp_plaintext_expect.ps1 ...............   Passed    4.66 sec
        Start 159: flb-it-http_server
        Start  51: flb-rt-core_routes
        Start 155: flb-it-record_accessor
        Start 143: flb-it-unicode
        Start 107: flb-it-gzip
        Start  99: flb-it-hash
        Start 138: flb-it-conditionals
 78/164 Test #130: flb-it-env .......................................   Passed    7.63 sec
 79/164 Test  #23: flb-rt-in_fluentbit_metrics ......................   Passed    8.74 sec
 80/164 Test  #87: in_syslog_udp_plaintext_expect.ps1 ...............   Passed    8.79 sec
        Start 119: flb-it-typecast
        Start  94: flb-it-pipe
        Start 105: flb-it-http_client
 81/164 Test  #24: flb-rt-in_prometheus_textfile ....................   Passed    6.51 sec
 82/164 Test  #85: in_tail_expect.ps1 ...............................   Passed    7.09 sec
 83/164 Test  #15: flb-rt-in_random .................................   Passed    9.67 sec
 84/164 Test #159: flb-it-http_server ...............................   Passed    5.78 sec
 85/164 Test #158: flb-it-cumulative_to_delta .......................   Passed    7.70 sec
 86/164 Test  #51: flb-rt-core_routes ...............................   Passed    5.13 sec
 87/164 Test  #84: in_dummy_expect.ps1 ..............................   Passed    6.52 sec
 88/164 Test #155: flb-it-record_accessor ...........................   Passed    4.64 sec
 89/164 Test  #47: flb-rt-filter_counter_semantics ..................   Passed    7.08 sec
 90/164 Test #143: flb-it-unicode ...................................   Passed    4.10 sec
 91/164 Test #107: flb-it-gzip ......................................   Passed    3.47 sec
 92/164 Test  #99: flb-it-hash ......................................   Passed    2.84 sec
 93/164 Test  #26: flb-rt-in_opentelemetry_routing ..................   Passed    8.69 sec
        Start 136: flb-it-uri
        Start  96: flb-it-sds_list
        Start 101: flb-it-slist
        Start 106: flb-it-utils
        Start   6: flb-rt-core_internal_logger
        Start 103: flb-it-router_config
        Start 127: flb-it-parser_ltsv
        Start 140: flb-it-task_map
        Start 116: flb-it-file
        Start 121: flb-it-bucket_queue
        Start 153: flb-it-parser
        Start 100: flb-it-kv
        Start 108: flb-it-zstd
 94/164 Test #118: flb-it-multiline .................................   Passed   15.74 sec
 95/164 Test #138: flb-it-conditionals ..............................   Passed    9.62 sec
 96/164 Test #119: flb-it-typecast ..................................   Passed    9.15 sec
 97/164 Test  #94: flb-it-pipe ......................................   Passed    8.57 sec
        Start 135: flb-it-processor_conditional
        Start 109: flb-it-random
        Start 141: flb-it-strptime
        Start  97: flb-it-hmac
 98/164 Test #105: flb-it-http_client ...............................   Passed   10.31 sec
 99/164 Test #136: flb-it-uri .......................................   Passed    9.84 sec
100/164 Test  #19: flb-rt-in_tail ...................................   Passed  147.04 sec
101/164 Test  #96: flb-it-sds_list ..................................   Passed    9.24 sec
102/164 Test #101: flb-it-slist .....................................   Passed    8.65 sec
103/164 Test #106: flb-it-utils .....................................   Passed    8.10 sec
104/164 Test   #6: flb-rt-core_internal_logger ......................   Passed    7.54 sec
105/164 Test #103: flb-it-router_config .............................   Passed    7.04 sec
106/164 Test #127: flb-it-parser_ltsv ...............................   Passed    6.60 sec
107/164 Test #140: flb-it-task_map ..................................   Passed    5.95 sec
108/164 Test #116: flb-it-file ......................................   Passed    5.24 sec
109/164 Test #121: flb-it-bucket_queue ..............................   Passed    4.62 sec
110/164 Test #153: flb-it-parser ....................................   Passed    4.02 sec
111/164 Test #100: flb-it-kv ........................................   Passed    3.51 sec
        Start 156: flb-it-cfl_record_accessor
        Start 104: flb-it-hashtable
        Start 148: flb-it-oauth2_jwt
        Start 102: flb-it-router
        Start 123: flb-it-ring_buffer
        Start 114: flb-it-input_chunk_routes
        Start   3: flb-rt-custom_calyptia_input_test
        Start 112: flb-it-mp
        Start  95: flb-it-sds
        Start 117: flb-it-csv
        Start 128: flb-it-parser_regex
        Start   1: flb-rt-custom_calyptia_test
        Start 139: flb-it-endianness
        Start  98: flb-it-crypto
112/164 Test #108: flb-it-zstd ......................................   Passed   10.67 sec
113/164 Test #135: flb-it-processor_conditional .....................   Passed    9.99 sec
114/164 Test #109: flb-it-random ....................................   Passed    9.50 sec
115/164 Test #141: flb-it-strptime ..................................   Passed    8.96 sec
        Start 137: flb-it-msgpack_append_message
        Start 145: flb-it-storage_dlq
        Start 142: flb-it-storage_inherit
        Start 124: flb-it-regex
116/164 Test  #97: flb-it-hmac ......................................   Passed   10.96 sec
117/164 Test #156: flb-it-cfl_record_accessor .......................   Passed   10.21 sec
118/164 Test  #71: flb-rt-out_stackdriver ...........................   Passed  158.51 sec
119/164 Test #104: flb-it-hashtable .................................   Passed    9.74 sec
120/164 Test #148: flb-it-oauth2_jwt ................................   Passed    9.26 sec
121/164 Test #102: flb-it-router ....................................   Passed    8.68 sec
122/164 Test #123: flb-it-ring_buffer ...............................   Passed    8.12 sec
123/164 Test #114: flb-it-input_chunk_routes ........................   Passed    7.56 sec
124/164 Test   #3: flb-rt-custom_calyptia_input_test ................   Passed    6.99 sec
125/164 Test #112: flb-it-mp ........................................   Passed    6.58 sec
126/164 Test  #95: flb-it-sds .......................................   Passed    6.08 sec
127/164 Test #117: flb-it-csv .......................................   Passed    5.54 sec
128/164 Test #128: flb-it-parser_regex ..............................   Passed    4.93 sec
129/164 Test   #1: flb-rt-custom_calyptia_test ......................   Passed    4.34 sec
130/164 Test #139: flb-it-endianness ................................   Passed    3.84 sec
        Start   8: flb-rt-http_client_chunked
        Start 120: flb-it-base64
        Start 157: flb-it-metrics
        Start 160: flb-it-aws_compress
        Start   4: flb-rt-in_calyptia_fleet_test
        Start 129: flb-it-parser_logfmt
        Start 126: flb-it-parser_json
        Start 163: flb-it-lua
        Start 161: flb-it-aws_aggregation
        Start 113: flb-it-mp_chunk_cobj
        Start  52: flb-rt-config_map_opts
        Start  54: flb-rt-out_azure_blob_compression
        Start   7: flb-rt-core_accept_timeout
        Start 164: flb-it-ripser
        Start 110: flb-it-thread_storage
131/164 Test  #98: flb-it-crypto ....................................   Passed   12.05 sec
132/164 Test #137: flb-it-msgpack_append_message ....................   Passed   11.33 sec
133/164 Test #145: flb-it-storage_dlq ...............................   Passed   10.82 sec
134/164 Test #142: flb-it-storage_inherit ...........................   Passed   10.07 sec
        Start 134: flb-it-processor
        Start 125: flb-it-scheduler
        Start  93: processor_invalid.ps1
        Start  91: dry_run_invalid_property.ps1
135/164 Test #124: flb-it-regex .....................................   Passed   10.76 sec
136/164 Test   #8: flb-rt-http_client_chunked .......................   Passed   10.22 sec
137/164 Test #120: flb-it-base64 ....................................   Passed    9.74 sec
138/164 Test #157: flb-it-metrics ...................................   Passed    9.21 sec
139/164 Test #160: flb-it-aws_compress ..............................   Passed    8.65 sec
140/164 Test   #4: flb-rt-in_calyptia_fleet_test ....................   Passed    8.08 sec
141/164 Test #129: flb-it-parser_logfmt .............................   Passed    7.64 sec
142/164 Test #126: flb-it-parser_json ...............................   Passed    7.02 sec
143/164 Test #163: flb-it-lua .......................................   Passed    6.37 sec
144/164 Test #161: flb-it-aws_aggregation ...........................   Passed    5.79 sec
145/164 Test #113: flb-it-mp_chunk_cobj .............................   Passed    5.27 sec
146/164 Test  #52: flb-rt-config_map_opts ...........................   Passed    4.65 sec
147/164 Test  #54: flb-rt-out_azure_blob_compression ................   Passed    3.76 sec
148/164 Test   #7: flb-rt-core_accept_timeout .......................   Passed    3.28 sec
149/164 Test #164: flb-it-ripser ....................................   Passed    2.74 sec
        Start 151: flb-it-config_format_fluentbit
        Start 152: flb-it-config_format_yaml
        Start 147: flb-it-upstream_tls
        Start 111: flb-it-config_map
        Start 146: flb-it-azure_blob_path
        Start 150: flb-it-config_format
        Start 115: flb-it-flb_time
        Start 133: flb-it-log_event_encoder
        Start  90: custom_calyptia.ps1
        Start 132: flb-it-log_event_decoder
150/164 Test #110: flb-it-thread_storage ............................   Passed    7.21 sec
151/164 Test #134: flb-it-processor .................................   Passed    6.61 sec
152/164 Test #125: flb-it-scheduler .................................   Passed    5.82 sec
153/164 Test  #93: processor_invalid.ps1 ............................   Passed    5.19 sec
154/164 Test  #91: dry_run_invalid_property.ps1 .....................   Passed    5.18 sec
155/164 Test #147: flb-it-upstream_tls ..............................   Passed    4.06 sec
156/164 Test #111: flb-it-config_map ................................   Passed    3.46 sec
157/164 Test #146: flb-it-azure_blob_path ...........................   Passed    2.88 sec
158/164 Test #150: flb-it-config_format .............................   Passed    2.30 sec
159/164 Test #115: flb-it-flb_time ..................................   Passed    1.72 sec
160/164 Test #133: flb-it-log_event_encoder .........................   Passed    1.20 sec
161/164 Test  #90: custom_calyptia.ps1 ..............................   Passed    0.63 sec
162/164 Test #152: flb-it-config_format_yaml ........................   Passed    4.57 sec
163/164 Test #151: flb-it-config_format_fluentbit ...................   Passed    5.18 sec
164/164 Test #132: flb-it-log_event_decoder .........................   Passed    0.81 sec

100% tests passed, 0 tests failed out of 164

Label Time Summary:
internal         = 537.83 sec*proc (71 tests)
runtime          = 2509.09 sec*proc (83 tests)
runtime_shell    =  65.19 sec*proc (10 tests)

Total Test time (real) = 174.75 sec

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
  • Debug log output from testing the change
  • Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • Run local packaging test showing all targets (including any new ones) build.
  • Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • Documentation required for this feature

Backporting

  • Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Summary by CodeRabbit

  • Bug Fixes

    • Improved Windows compatibility for file handling, socket operations, path matching, environment variables, and temporary storage.
    • Corrected S3 buffering paths on Windows and improved upload timing during testing.
    • Improved cancellation behavior for throttled processing and safer syslog socket cleanup.
    • Made Kubernetes metadata and retention-time handling more reliable across platforms.
  • Tests

    • Expanded runtime and shell test coverage, including PowerShell support on Windows.
    • Improved test synchronization by replacing fixed delays with bounded polling and completion checks.

cosmo0920 added 30 commits July 13, 2026 21:11
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 911c18a5bf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tests/runtime/in_dummy.c

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (2)
tests/runtime_shell/custom_calyptia.ps1 (1)

24-28: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Replace fixed 30-second sleep with bounded polling.

Start-Sleep -Seconds 30 always waits the full duration even if YAML files appear immediately, and delays failure detection when Fluent Bit crashes early. The PR objectives explicitly call out "replacing fixed test delays with bounded polling or completion synchronization," and common.ps1 already provides Wait-ForFile for this pattern. A similar polling loop checking for YAML files in $env:CALYPTIA_FLEET_DIR would return sooner on success and detect early exit faster.

♻️ Proposed bounded-polling replacement for lines 24-28
 $process = Start-FluentBit $config
-Start-Sleep -Seconds 30
-if ($process.HasExited -and $process.ExitCode -ne 0) {
-    throw "Fluent Bit exited with code $($process.ExitCode)"
-}
+
+$deadline = [DateTime]::UtcNow.AddSeconds(30)
+while ([DateTime]::UtcNow -lt $deadline) {
+    $yamlFiles = @(Get-ChildItem -LiteralPath $env:CALYPTIA_FLEET_DIR `
+        -Filter "*.yaml" -File -Recurse -ErrorAction SilentlyContinue)
+    if ($yamlFiles.Count -gt 0) {
+        break
+    }
+    if ($process.HasExited) {
+        throw "Fluent Bit exited before generating YAML files (code $($process.ExitCode))"
+    }
+    Start-Sleep -Milliseconds 100
+}

 $yamlFiles = @(Get-ChildItem -LiteralPath $env:CALYPTIA_FLEET_DIR `
     -Filter "*.yaml" -File -Recurse -ErrorAction SilentlyContinue)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/runtime_shell/custom_calyptia.ps1` around lines 24 - 28, Replace the
fixed Start-Sleep call in the Start-FluentBit flow with bounded polling for YAML
files under $env:CALYPTIA_FLEET_DIR, reusing Wait-ForFile from common.ps1 where
applicable. Poll until the expected files appear or the timeout expires,
checking $process.HasExited during polling to fail promptly with its exit code,
while preserving the existing failure handling.
tests/runtime/out_s3.c (1)

108-144: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add timeout diagnostics to S3 polling helpers for consistency.

wait_for_s3_call_count and wait_for_file_count silently return on timeout without any log, unlike wait_with_timeout in filter_kubernetes.c which emits flb_warn("[timeout] elapsed_time: %ld", ...). A timeout warning would help distinguish "condition never met" from "logical assertion failure" during test debugging.

♻️ Proposed timeout warning for both helpers
 static void wait_for_s3_call_count(const char *api, int expected)
 {
     uint64_t elapsed_ms;
     struct flb_time start_time;
     struct flb_time end_time;
     struct flb_time diff_time;

     elapsed_ms = 0;
     flb_time_get(&start_time);

     while (get_s3_call_count(api) < expected &&
            elapsed_ms < S3_TEST_WAIT_TIMEOUT_MS) {
         flb_time_msleep(S3_TEST_WAIT_STEP_MS);
         flb_time_get(&end_time);
         flb_time_diff(&end_time, &start_time, &diff_time);
         elapsed_ms = flb_time_to_nanosec(&diff_time) / 1000000;
     }
+    if (elapsed_ms >= S3_TEST_WAIT_TIMEOUT_MS) {
+        flb_warn("[timeout] wait_for_s3_call_count(%s, %d) elapsed: %llu ms",
+                  api, expected, (unsigned long long) elapsed_ms);
+    }
 }

 static void wait_for_file_count(const char *path, int expected)
 {
     uint64_t elapsed_ms;
     struct flb_time start_time;
     struct flb_time end_time;
     struct flb_time diff_time;

     elapsed_ms = 0;
     flb_time_get(&start_time);

     while (count_files_recursive(path) < expected &&
            elapsed_ms < S3_TEST_WAIT_TIMEOUT_MS) {
         flb_time_msleep(S3_TEST_WAIT_STEP_MS);
         flb_time_get(&end_time);
         flb_time_diff(&end_time, &start_time, &diff_time);
         elapsed_ms = flb_time_to_nanosec(&diff_time) / 1000000;
     }
+    if (elapsed_ms >= S3_TEST_WAIT_TIMEOUT_MS) {
+        flb_warn("[timeout] wait_for_file_count(%s, %d) elapsed: %llu ms",
+                  path, expected, (unsigned long long) elapsed_ms);
+    }
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/runtime/out_s3.c` around lines 108 - 144, Add timeout warnings to both
wait_for_s3_call_count and wait_for_file_count after their polling loops when
the expected condition remains unmet, including the elapsed timeout duration in
the log. Keep the existing polling and success behavior unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@plugins/filter_throttle/throttle.c`:
- Around line 95-97: Update the Windows sleep path in the ticker loop around
pthread_testcancel() to divide long sleeps into shorter intervals, invoking
pthread_testcancel() between each interval and preserving the total configured
delay. Keep cb_throttle_exit shutdown behavior unchanged while ensuring
cancellation is observed promptly during the sleep.

In `@tests/runtime_shell/common.ps1`:
- Around line 63-70: Update Stop-FluentBit to tolerate the process exiting
between the HasExited check and Stop-Process invocation. Handle the expected
Stop-Process failure from concurrent exit locally so the global Stop error
policy cannot mask the test result, while preserving the existing cleanup and
wait behavior.

In `@tests/runtime_shell/processor_conditional.ps1`:
- Around line 13-17: Replace fixed sleeps with bounded polling in all three
sites: in tests/runtime_shell/processor_conditional.ps1 lines 13-17, poll
captured output for the expected records before stopping Fluent Bit; in
tests/runtime_shell/processor_conditional.sh lines 20-21, poll for the GET
marker while verifying the POST marker remains absent; and in
tests/runtime_shell/processor_conditional.sh lines 56-57, poll until both
accepted endpoints appear or the timeout/process exit condition occurs.

In `@tests/runtime_shell/processor_conditional.sh`:
- Around line 23-36: Update every Fluent Bit termination path in the test,
including the missing-output branch and the cleanup around both cases, to wait
for the process after sending SIGTERM. Preserve the existing tolerant kill
behavior, then invoke the appropriate wait on FLB_PID before proceeding,
exiting, or starting the next case so the process has fully terminated.

---

Nitpick comments:
In `@tests/runtime_shell/custom_calyptia.ps1`:
- Around line 24-28: Replace the fixed Start-Sleep call in the Start-FluentBit
flow with bounded polling for YAML files under $env:CALYPTIA_FLEET_DIR, reusing
Wait-ForFile from common.ps1 where applicable. Poll until the expected files
appear or the timeout expires, checking $process.HasExited during polling to
fail promptly with its exit code, while preserving the existing failure
handling.

In `@tests/runtime/out_s3.c`:
- Around line 108-144: Add timeout warnings to both wait_for_s3_call_count and
wait_for_file_count after their polling loops when the expected condition
remains unmet, including the elapsed timeout duration in the log. Keep the
existing polling and success behavior unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 06b5fc6d-f0e5-4c0f-90d2-66cf5fcee9a8

📥 Commits

Reviewing files that changed from the base of the PR and between 0896e9c and 911c18a.

📒 Files selected for processing (68)
  • CMakeLists.txt
  • cmake/windows-setup.cmake
  • plugins/filter_kubernetes/kube_meta.c
  • plugins/filter_throttle/throttle.c
  • plugins/in_event_test/event_test.c
  • plugins/in_kubernetes_events/kubernetes_events.c
  • plugins/in_tail/tail_scan_win32.c
  • plugins/out_loki/loki.c
  • plugins/out_s3/s3.c
  • plugins/out_syslog/syslog.c
  • tests/include/aws_client_mock.h
  • tests/runtime/CMakeLists.txt
  • tests/runtime/core-timeout.c
  • tests/runtime/core_accept_timeout.c
  • tests/runtime/core_chunk_trace.c
  • tests/runtime/core_engine.c
  • tests/runtime/counter_parity_e2e.c
  • tests/runtime/custom_calyptia_input_test.c
  • tests/runtime/filter_kubernetes.c
  • tests/runtime/filter_parser.c
  • tests/runtime/filter_throttle_size.c
  • tests/runtime/flb_tests_runtime.h.in
  • tests/runtime/group_counter_semantics.c
  • tests/runtime/http_client_chunked.c
  • tests/runtime/in_calyptia_fleet_test.c
  • tests/runtime/in_dummy.c
  • tests/runtime/in_event_test.c
  • tests/runtime/in_forward.c
  • tests/runtime/in_http.c
  • tests/runtime/in_kubernetes_events.c
  • tests/runtime/in_opentelemetry_routing.c
  • tests/runtime/in_prometheus_textfile.c
  • tests/runtime/in_random.c
  • tests/runtime/in_simple_systems.c
  • tests/runtime/in_syslog.c
  • tests/runtime/in_tail.c
  • tests/runtime/in_tcp.c
  • tests/runtime/in_udp.c
  • tests/runtime/out_counter.c
  • tests/runtime/out_firehose.c
  • tests/runtime/out_http.c
  • tests/runtime/out_logdna.c
  • tests/runtime/out_s3.c
  • tests/runtime/out_s3_otlp_json.c
  • tests/runtime/out_stackdriver.c
  • tests/runtime/out_syslog.c
  • tests/runtime/processor_cumulative_to_delta.c
  • tests/runtime_shell/CMakeLists.txt
  • tests/runtime_shell/common.ps1
  • tests/runtime_shell/conf/dry_run_invalid_property.yaml
  • tests/runtime_shell/conf/in_tail_expect.conf
  • tests/runtime_shell/conf/processor_conditional.yaml
  • tests/runtime_shell/conf/processor_conditional_grep.yaml
  • tests/runtime_shell/conf/processor_invalid.yaml
  • tests/runtime_shell/custom_calyptia.ps1
  • tests/runtime_shell/dry_run_invalid_property.ps1
  • tests/runtime_shell/dry_run_invalid_property.sh
  • tests/runtime_shell/in_dummy_expect.ps1
  • tests/runtime_shell/in_http_tls_expect.ps1
  • tests/runtime_shell/in_syslog_tcp_plaintext_expect.ps1
  • tests/runtime_shell/in_syslog_tcp_tls_expect.ps1
  • tests/runtime_shell/in_syslog_udp_plaintext_expect.ps1
  • tests/runtime_shell/in_tail_expect.ps1
  • tests/runtime_shell/in_tail_expect.sh
  • tests/runtime_shell/processor_conditional.ps1
  • tests/runtime_shell/processor_conditional.sh
  • tests/runtime_shell/processor_invalid.ps1
  • tests/runtime_shell/processor_invalid.sh

Comment on lines +95 to +97
/* Windows sleep is not a pthread cancellation point. */
sleep(ctx->ticker_data.seconds);
pthread_testcancel();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Shutdown may block for the full sleep duration on Windows.

pthread_testcancel() after sleep() correctly enables cancellation, but pthread_join() in cb_throttle_exit (Line 285) will block until the sleep completes. If ctx->ticker_data.seconds is large, shutdown could be delayed significantly. Consider breaking the sleep into smaller increments with pthread_testcancel() between them for faster cancellation response.

♻️ Suggested improvement
     /* Windows sleep is not a pthread cancellation point. */
-    sleep(ctx->ticker_data.seconds);
-    pthread_testcancel();
+    {
+        int remaining = ctx->ticker_data.seconds;
+        while (remaining > 0) {
+            int chunk = remaining > 5 ? 5 : remaining;
+            sleep(chunk);
+            pthread_testcancel();
+            remaining -= chunk;
+        }
+    }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
/* Windows sleep is not a pthread cancellation point. */
sleep(ctx->ticker_data.seconds);
pthread_testcancel();
/* Windows sleep is not a pthread cancellation point. */
{
int remaining = ctx->ticker_data.seconds;
while (remaining > 0) {
int chunk = remaining > 5 ? 5 : remaining;
sleep(chunk);
pthread_testcancel();
remaining -= chunk;
}
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@plugins/filter_throttle/throttle.c` around lines 95 - 97, Update the Windows
sleep path in the ticker loop around pthread_testcancel() to divide long sleeps
into shorter intervals, invoking pthread_testcancel() between each interval and
preserving the total configured delay. Keep cb_throttle_exit shutdown behavior
unchanged while ensuring cancellation is observed promptly during the sleep.

Comment on lines +63 to +70
function Stop-FluentBit
{
param([System.Diagnostics.Process] $Process)

if ($null -ne $Process -and -not $Process.HasExited) {
Stop-Process -Id $Process.Id -Force
[void] $Process.WaitForExit(5000)
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Make process cleanup tolerant of concurrent exit.

The process can exit after HasExited is checked, causing Stop-Process to throw under the global Stop error policy and mask the real test result.

Proposed fix
     if ($null -ne $Process -and -not $Process.HasExited) {
-        Stop-Process -Id $Process.Id -Force
+        Stop-Process -Id $Process.Id -Force -ErrorAction SilentlyContinue
         [void] $Process.WaitForExit(5000)
     }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
function Stop-FluentBit
{
param([System.Diagnostics.Process] $Process)
if ($null -ne $Process -and -not $Process.HasExited) {
Stop-Process -Id $Process.Id -Force
[void] $Process.WaitForExit(5000)
}
function Stop-FluentBit
{
param([System.Diagnostics.Process] $Process)
if ($null -ne $Process -and -not $Process.HasExited) {
Stop-Process -Id $Process.Id -Force -ErrorAction SilentlyContinue
[void] $Process.WaitForExit(5000)
}
🧰 Tools
🪛 PSScriptAnalyzer (1.25.0)

[warning] 63-63: Function 'Stop-FluentBit' has verb that could change system state. Therefore, the function has to support 'ShouldProcess'.

(PSUseShouldProcessForStateChangingFunctions)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/runtime_shell/common.ps1` around lines 63 - 70, Update Stop-FluentBit
to tolerate the process exiting between the HasExited check and Stop-Process
invocation. Handle the expected Stop-Process failure from concurrent exit
locally so the global Stop error policy cannot mask the test result, while
preserving the existing cleanup and wait behavior.

Comment on lines +13 to +17
$process = Start-FluentBit $ConfigPath @("-o", "stdout") $OutputPath $ErrorPath
Start-Sleep -Seconds 5
if ($process.HasExited -and $process.ExitCode -ne 0) {
throw "Fluent Bit exited with code $($process.ExitCode)"
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Replace fixed delays with bounded completion polling across both implementations.

  • tests/runtime_shell/processor_conditional.ps1#L13-L17: poll captured output for the expected records before stopping Fluent Bit.
  • tests/runtime_shell/processor_conditional.sh#L20-L21: poll for the GET marker and ensure the POST marker remains absent.
  • tests/runtime_shell/processor_conditional.sh#L56-L57: poll until both accepted endpoints are present or timeout/process exit occurs.
📍 Affects 2 files
  • tests/runtime_shell/processor_conditional.ps1#L13-L17 (this comment)
  • tests/runtime_shell/processor_conditional.sh#L20-L21
  • tests/runtime_shell/processor_conditional.sh#L56-L57
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/runtime_shell/processor_conditional.ps1` around lines 13 - 17, Replace
fixed sleeps with bounded polling in all three sites: in
tests/runtime_shell/processor_conditional.ps1 lines 13-17, poll captured output
for the expected records before stopping Fluent Bit; in
tests/runtime_shell/processor_conditional.sh lines 20-21, poll for the GET
marker while verifying the POST marker remains absent; and in
tests/runtime_shell/processor_conditional.sh lines 56-57, poll until both
accepted endpoints appear or the timeout/process exit condition occurs.

Comment on lines 23 to +36
if [ ! -f "$OUTPUT_FILE" ]; then
echo "Output file not found"
kill -15 $FLB_PID || true
kill -15 "$FLB_PID" || true
exit 1
fi

echo "Output file content:"
cat $OUTPUT_FILE
cat "$OUTPUT_FILE"

# Verify that the GET condition was applied but not the POST condition
GET_FIELD=$(grep -c "modified_if_get" $OUTPUT_FILE)
POST_FIELD=$(grep -c "modified_if_post" $OUTPUT_FILE)
GET_FIELD=$(grep -c "modified_if_get" "$OUTPUT_FILE")
POST_FIELD=$(grep -c "modified_if_post" "$OUTPUT_FILE")

# Clean up
echo "Cleaning up..."
kill -15 $FLB_PID || true
rm -f /tmp/processor_conditional.yaml
rm -f $OUTPUT_FILE
kill -15 "$FLB_PID" || true
rm -f "$OUTPUT_FILE"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Wait for Fluent Bit after sending SIGTERM.

kill only requests termination. Without wait, the test can proceed or exit while Fluent Bit remains alive, and the first process may overlap the second case.

Proposed cleanup pattern
 kill -15 "$FLB_PID" || true
+wait "$FLB_PID" 2>/dev/null || true
 rm -f "$OUTPUT_FILE"

Apply the same sequence to every termination path.

Also applies to: 59-73

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/runtime_shell/processor_conditional.sh` around lines 23 - 36, Update
every Fluent Bit termination path in the test, including the missing-output
branch and the cleanup around both cases, to wait for the process after sending
SIGTERM. Preserve the existing tolerant kill behavior, then invoke the
appropriate wait on FLB_PID before proceeding, exiting, or starting the next
case so the process has fully terminated.

@edsiper
edsiper merged commit b7eb8cf into master Jul 13, 2026
93 of 100 checks passed
@edsiper
edsiper deleted the cosmo0920-enable-runtime-testcases-on-windows branch July 13, 2026 17:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants