Skip to content

listener: enable connection balancer to support multiple addresses listener#21774

Merged
mattklein123 merged 17 commits intoenvoyproxy:mainfrom
soulxu:multi_addrs_conn_balancer
Jun 23, 2022
Merged

listener: enable connection balancer to support multiple addresses listener#21774
mattklein123 merged 17 commits intoenvoyproxy:mainfrom
soulxu:multi_addrs_conn_balancer

Conversation

@soulxu
Copy link
Copy Markdown
Member

@soulxu soulxu commented Jun 20, 2022

Commit Message: listener: enable connection balancer to support multiple addresses listener
Additional Description:
This PR enables the ListenerImpl to create connection balancer for each listening address. And pass the specific connection balancer to ActiveTcpListener. Also enable the ConnectionHandlerImpl::getBalancedHandlerByTag to distinguish different address.

Please reference the full implementation of multiple addresses in listener at #19367

Risk Level: high
Testing: unittest
Docs Changes: n/a
Release Notes: n/a
Platform Specific Features: n/a
Part of #11184

soulxu added 6 commits June 20, 2022 03:12
Signed-off-by: He Jie Xu <hejie.xu@intel.com>
Signed-off-by: He Jie Xu <hejie.xu@intel.com>
Signed-off-by: He Jie Xu <hejie.xu@intel.com>
Signed-off-by: He Jie Xu <hejie.xu@intel.com>
…pports pipe

Signed-off-by: He Jie Xu <hejie.xu@intel.com>
Signed-off-by: He Jie Xu <hejie.xu@intel.com>
Signed-off-by: He Jie Xu <hejie.xu@intel.com>
@soulxu
Copy link
Copy Markdown
Member Author

soulxu commented Jun 20, 2022

/assign @adisuissa @mattklein123

Signed-off-by: He Jie Xu <hejie.xu@intel.com>
Copy link
Copy Markdown
Member

@mattklein123 mattklein123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Generally LGTM. Flushing out some comments.

/wait

std::atomic<uint64_t> num_listener_connections_{};

Network::ConnectionBalancer& connection_balancer_;
Network::Address::InstanceConstSharedPtr address_;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This listener must know it's address because it has a socket, right? It seems like this shouldn't be necessary?

Copy link
Copy Markdown
Member Author

@soulxu soulxu Jun 22, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have think about another two ways:

  • We can get the address from the accepted socket, but it won't work with the case of the connection is redirected by the iptable
  • The socket object is held by the socket listener, then we can add an interface to the listener to return the socket address.
    class Listener {

The second way works also. But let me know if the second way you think better.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

soulxu added 4 commits June 22, 2022 06:35
Signed-off-by: He Jie Xu <hejie.xu@intel.com>
Signed-off-by: He Jie Xu <hejie.xu@intel.com>
Signed-off-by: He Jie Xu <hejie.xu@intel.com>
Signed-off-by: He Jie Xu <hejie.xu@intel.com>
Signed-off-by: He Jie Xu <hejie.xu@intel.com>
Copy link
Copy Markdown
Member

@mattklein123 mattklein123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a few more questions and comments.

/wait

// This is the address of this listener is listening on. And used for get the correct listener
// when rebalancing. The accepted socket can't be used to get the listening address, since
// the accepted socket's remote address can be another address than the listening address.
Network::Address::InstanceConstSharedPtr address_;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Network::Address::InstanceConstSharedPtr address_;
Network::Address::InstanceConstSharedPtr listen_address_;

Can't you get this from the config though? Is't that stored in ActiveListenerImplBase in config_ or somewhere else?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After add multiple addresses support for ListenerImpl, then the ListenerImpl's address() will changes to addresses() (also need to move address() interface to base ListenerConfig), there still no way for ActiveListener knows which address is listening. I have tried to pass an index of addresses() to the ActiveListener, but I thought the index isn't readable in the end, hold a Network::Address::InstanceConstSharedPtr is more readable.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK this makes sense, thanks.

Signed-off-by: He Jie Xu <hejie.xu@intel.com>
soulxu added 3 commits June 22, 2022 23:22
Signed-off-by: He Jie Xu <hejie.xu@intel.com>
Signed-off-by: He Jie Xu <hejie.xu@intel.com>
Signed-off-by: He Jie Xu <hejie.xu@intel.com>
Copy link
Copy Markdown
Member

@mattklein123 mattklein123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@mattklein123 mattklein123 merged commit 3642562 into envoyproxy:main Jun 23, 2022
Amila-Rukshan pushed a commit to Amila-Rukshan/envoy that referenced this pull request Jun 28, 2022
…stener (envoyproxy#21774)

Signed-off-by: He Jie Xu <hejie.xu@intel.com>
Signed-off-by: Amila Senadheera <amila.15@cse.mrt.ac.lk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants