When using the same port number, the td-agent fails to bind to IPv4 alongside IPv6

[nmagnezi]

Describe the bug
When trying to configure the td-agent to bind both to IPv4 and IPv6 using the same port, The following error occurs:

2019-11-04 14:00:26 +0000 [error]: #0 unexpected error error_class=Errno::EADDRINUSE error="Address already in use - bind(2) for \"0.0.0.0\" port 8888"

To Reproduce
Use the below-mentioned configuration.

Expected behavior
fluentd should be able to use the same port number for both types of IP addresses.

Your Environment

• Fluentd or td-agent version: td-agent 1.7.4
• Operating system: Fedora 29
• Kernel version: 4.18.16-300.fc29.x86_64

Your Configuration

<source>
  @type http
  @id input_http_ipv4
  port 8888
</source>

<source>
  @type http
  @id input_http_ipv6
  bind ::
  port 8888
</source>

<match *.*>
  @type stdout
</match>```

Your Error Log

2019-11-04 14:00:26 +0000 [info]: gem 'fluent-plugin-elasticsearch' version '3.5.5'
2019-11-04 14:00:26 +0000 [info]: gem 'fluent-plugin-kafka' version '0.12.1'
2019-11-04 14:00:26 +0000 [info]: gem 'fluent-plugin-prometheus' version '1.7.0'
2019-11-04 14:00:26 +0000 [info]: gem 'fluent-plugin-record-modifier' version '2.0.1'
2019-11-04 14:00:26 +0000 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '2.2.0'
2019-11-04 14:00:26 +0000 [info]: gem 'fluent-plugin-s3' version '1.2.0'
2019-11-04 14:00:26 +0000 [info]: gem 'fluent-plugin-td' version '1.0.0'
2019-11-04 14:00:26 +0000 [info]: gem 'fluent-plugin-td-monitoring' version '0.2.4'
2019-11-04 14:00:26 +0000 [info]: gem 'fluent-plugin-webhdfs' version '1.2.4'
2019-11-04 14:00:26 +0000 [info]: gem 'fluentd' version '1.7.4'
2019-11-04 14:00:26 +0000 [info]: adding match pattern="*.*" type="stdout"
2019-11-04 14:00:26 +0000 [info]: adding source type="http"
2019-11-04 14:00:26 +0000 [info]: adding source type="http"
2019-11-04 14:00:26 +0000 [info]: #0 starting fluentd worker pid=5959 ppid=5930 worker=0
2019-11-04 14:00:26 +0000 [error]: #0 unexpected error error_class=Errno::EADDRINUSE error="Address already in use - bind(2) for \"0.0.0.0\" port 8888"
  2019-11-04 14:00:26 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/serverengine-2.1.1/lib/serverengine/socket_manager_unix.rb:53:in `initialize'
  2019-11-04 14:00:26 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/serverengine-2.1.1/lib/serverengine/socket_manager_unix.rb:53:in `new'
  2019-11-04 14:00:26 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/serverengine-2.1.1/lib/serverengine/socket_manager_unix.rb:53:in `listen_tcp_new'
  2019-11-04 14:00:26 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/serverengine-2.1.1/lib/serverengine/socket_manager.rb:121:in `block in listen'
  2019-11-04 14:00:26 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/serverengine-2.1.1/lib/serverengine/socket_manager.rb:119:in `synchronize'
  2019-11-04 14:00:26 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/serverengine-2.1.1/lib/serverengine/socket_manager.rb:119:in `listen'
  2019-11-04 14:00:26 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/serverengine-2.1.1/lib/serverengine/socket_manager.rb:128:in `listen_tcp'
  2019-11-04 14:00:26 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/serverengine-2.1.1/lib/serverengine/socket_manager_unix.rb:106:in `send_socket'
  2019-11-04 14:00:26 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/serverengine-2.1.1/lib/serverengine/socket_manager.rb:155:in `process_peer'
2019-11-04 14:00:26 +0000 [error]: #0 unexpected error error_class=Errno::EADDRINUSE error="Address already in use - bind(2) for \"0.0.0.0\" port 8888"
  2019-11-04 14:00:26 +0000 [error]: #0 suppressed same stacktrace

Additional context

Just to note that as a workaround, if you set two different port numbers, it binds to both IPv4 and IPv6.

netstat -ntpl | grep ruby
tcp        0      0 0.0.0.0:48888           0.0.0.0:*               LISTEN      6152/ruby
tcp6       0      0 :::8888                 :::*                    LISTEN      6152/ruby

[repeatedly]

This looks like Linux's dual-stack mode issue: https://stackoverflow.com/questions/51911564/cannot-bind-socket-to-port-in-both-ipv4-and-ipv6

[nmagnezi]

Hi @repeatedly, thanks for the prompt response.

I suspect this is not a Linux issue.
On that very same node, I was able to make rsyslogd to bind both IPv4 and IPv6 addresses using the same port.

netstat -ntpl  | grep rsyslogd
tcp        0      0 0.0.0.0:514             0.0.0.0:*               LISTEN      729/rsyslogd
tcp6       0      0 :::514                  :::*                    LISTEN      729/rsyslogd

[repeatedly]

Yeah, maybe rsyslog sets IPV6_V6ONLY flag explicitly on Linux.
I tested following code on Linux and Mac. Linux failed by same error.

require 'socket'

serv1 = TCPServer.new("0.0.0.0", 2300)
serv2 = TCPServer.new("::", 2300)

I will check how to apply IPV6_V6ONLY for TCPServer.new.

[repeatedly]

I checked the source code and TCPServer doesn't provide the way to set IPV6_V6ONLY flag.
Other socket utilities set IPV6_V6ONLY internally, so I will investigate how to use these functions with TCPServer.