Fluentd 1.10.0: Unable to parse

[mmoench]

Check CONTRIBUTING guideline first and here is the list to help us investigate the problem.

Describe the bug
After upgrading to td-agent 3.7.0 (fluentd 1.10.0) I am unable to parse my syslog messages (in rfc5424 format).

To Reproduce
Install td-agent 3.7.0 and forward rsyslog-messages to td-agent, using rfc5424 as format.

Expected behavior
The syslog-messages should get parsed correctly.

Your Environment

• td-agent 3.7.0
• fluentd 1.10.0
• Ubuntu 18.04
• Kernel 4.15.0-1063-aws

Your Configuration

<source>
    @type syslog
    port 42185
    bind 127.0.0.1
    tag syslog
    <parse>
        message_format rfc5424
        with_priority true
    </parse>
</source>

Your Error Log

2020-04-02 11:34:21 +0000 [error]: #0 invalid input data="<13>1 2020-04-02T11:34:21.730058+00:00 testhost.app-stage test - - -  testing3" error_class=NoMethodError error="undefined method `>>' for \"13\":String\nDid you mean?  >"
  2020-04-02 11:34:21 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.10.0/lib/fluent/plugin/in_syslog.rb:239:in `block in message_handler'
  2020-04-02 11:34:21 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.10.0/lib/fluent/plugin/parser_syslog.rb:244:in `parse_plain'
  2020-04-02 11:34:21 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.10.0/lib/fluent/plugin/parser_syslog.rb:214:in `parse_rfc5424_regex'
  2020-04-02 11:34:21 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.10.0/lib/fluent/plugin/in_syslog.rb:229:in `message_handler'
  2020-04-02 11:34:21 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.10.0/lib/fluent/plugin/in_syslog.rb:167:in `block in start_udp_server'
  2020-04-02 11:34:21 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.10.0/lib/fluent/plugin_helper/server.rb:543:in `on_readable_with_sock'
  2020-04-02 11:34:21 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/cool.io-1.6.0/lib/cool.io/io.rb:186:in `on_readable'
  2020-04-02 11:34:21 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/cool.io-1.6.0/lib/cool.io/loop.rb:88:in `run_once'
  2020-04-02 11:34:21 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/cool.io-1.6.0/lib/cool.io/loop.rb:88:in `run'
  2020-04-02 11:34:21 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.10.0/lib/fluent/plugin_helper/event_loop.rb:93:in `block in start'
  2020-04-02 11:34:21 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.10.0/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'

Additional context
I guess the function "parse_rfc5424_regex" is missing an "to_i", which did fix the issue for me:

--- /tmp/parser_syslog.rb       2020-04-02 12:00:13.476209187 +0000
+++ /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.10.0/lib/fluent/plugin/parser_syslog.rb   2020-04-02 11:57:45.311978982 +0000
@@ -175,7 +175,7 @@

         if @with_priority
           if (m = RFC5424_PRI_REGEXP.match(text))
-            record['pri'] = m['pri']
+            record['pri'] = m['pri'].to_i
             idx = m.end(0)
           else
             yield(nil, nil)

[repeatedly]

This has been fixed in the master: #2923