Skip to content

matchers: extend network inputs to HTTP requests #20796

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
snowp merged 6 commits into from
Apr 18, 2022
Merged

matchers: extend network inputs to HTTP requests #20796

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
fd4f8bb
matchers: extend network inputs to HTTP requests
kyessenov Apr 12, 2022
6464060
Merge remote-tracking branch 'upstream/main' into network_inputs_http
kyessenov Apr 12, 2022
200786d
merge race
kyessenov Apr 12, 2022
817ae12
add source type
kyessenov Apr 13, 2022
de264d3
move to header
kyessenov Apr 14, 2022
3d77c5d
inline
kyessenov Apr 14, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions docs/root/intro/arch_overview/advanced/matching/matching_api.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,18 +40,21 @@ These input functions are available for matching HTTP requests:
Network Input Functions
***********************

These input functions are available for matching both TCP connections and UDP datagrams:
These input functions are available for matching TCP connections, UDP datagrams, and HTTP requests:

* :ref:`Destination IP <extension_envoy.matching.inputs.destination_ip>`.
* :ref:`Destination port <extension_envoy.matching.inputs.destination_port>`.
* :ref:`Source IP <extension_envoy.matching.inputs.source_ip>`.
* :ref:`Source port <extension_envoy.matching.inputs.source_port>`.

These input functions are available for matching TCP connections:
These input functions are available for matching TCP connections and HTTP requests:

* :ref:`Direct source IP <extension_envoy.matching.inputs.direct_source_ip>`.
* :ref:`Source type <extension_envoy.matching.inputs.source_type>`.
* :ref:`Server name <extension_envoy.matching.inputs.server_name>`.

These input functions are available for matching TCP connections:

* :ref:`Transport protocol <extension_envoy.matching.inputs.transport_protocol>`.
* :ref:`Application protocol <extension_envoy.matching.inputs.application_protocol>`.

Expand Down
1 change: 1 addition & 0 deletions envoy/http/filter.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1001,6 +1001,7 @@ class HttpMatchingData {
virtual RequestTrailerMapOptConstRef requestTrailers() const PURE;
virtual ResponseHeaderMapOptConstRef responseHeaders() const PURE;
virtual ResponseTrailerMapOptConstRef responseTrailers() const PURE;
virtual const Network::ConnectionInfoProvider& connectionInfoProvider() const PURE;
};

/**
Expand Down
4 changes: 4 additions & 0 deletions envoy/network/filter.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -529,6 +529,10 @@ class MatchingData {
virtual ~MatchingData() = default;

virtual const ConnectionSocket& socket() const PURE;

const ConnectionInfoProvider& connectionInfoProvider() const {
return socket().connectionInfoProvider();
}
};

/**
Expand Down
9 changes: 6 additions & 3 deletions source/common/http/filter_manager.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -698,7 +698,8 @@ class FilterManager : public ScopeTrackedObject,
addStreamDecoderFilterWorker(
filter,
std::make_shared<FilterMatchState>(std::move(match_tree),
std::make_shared<Matching::HttpMatchingDataImpl>()),
std::make_shared<Matching::HttpMatchingDataImpl>(
stream_info_.downstreamAddressProvider())),
false);
return;
}
Expand All @@ -715,7 +716,8 @@ class FilterManager : public ScopeTrackedObject,
addStreamEncoderFilterWorker(
filter,
std::make_shared<FilterMatchState>(std::move(match_tree),
std::make_shared<Matching::HttpMatchingDataImpl>()),
std::make_shared<Matching::HttpMatchingDataImpl>(
stream_info_.downstreamAddressProvider())),
false);
return;
}
Expand All @@ -736,7 +738,8 @@ class FilterManager : public ScopeTrackedObject,
// the result to both filters after the first match evaluation.
if (match_tree) {
auto matching_state = std::make_shared<FilterMatchState>(
std::move(match_tree), std::make_shared<Matching::HttpMatchingDataImpl>());
std::move(match_tree), std::make_shared<Matching::HttpMatchingDataImpl>(
stream_info_.downstreamAddressProvider()));
addStreamDecoderFilterWorker(filter, matching_state, true);
addStreamEncoderFilterWorker(filter, std::move(matching_state), true);
return;
Expand Down
8 changes: 8 additions & 0 deletions source/common/http/matching/data_impl.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ namespace Matching {
*/
class HttpMatchingDataImpl : public HttpMatchingData {
public:
explicit HttpMatchingDataImpl(const Network::ConnectionInfoProvider& connection_info_provider)
: connection_info_provider_(connection_info_provider) {}

@zhxie zhxie Apr 13, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think it would be better to provide a StreamInfo directly for HTTP matching data, compared to ConnectionSocket for network matching data?

@kyessenov kyessenov Apr 13, 2022

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

StreamInfo is complicated, it can be nested for internal redirects so I wanted to avoid bringing all that complexity implicitly.
I think we'll probably end-up using it though.


static absl::string_view name() { return "http"; }

void onRequestHeaders(const RequestHeaderMap& request_headers) {
Expand Down Expand Up @@ -47,7 +50,12 @@ class HttpMatchingDataImpl : public HttpMatchingData {
return makeOptRefFromPtr(response_trailers_);
}

const Network::ConnectionInfoProvider& connectionInfoProvider() const override {
return connection_info_provider_;
}

private:
const Network::ConnectionInfoProvider& connection_info_provider_;
const RequestHeaderMap* request_headers_{};
const ResponseHeaderMap* response_headers_{};
const RequestTrailerMap* request_trailers_{};
Expand Down
1 change: 1 addition & 0 deletions source/common/network/matching/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ envoy_cc_library(
srcs = ["inputs.cc"],
hdrs = ["inputs.h"],
deps = [
"//envoy/http:filter_interface",
"//envoy/matcher:matcher_interface",
"//envoy/network:filter_interface",
"//envoy/registry",
Expand Down
85 changes: 66 additions & 19 deletions source/common/network/matching/inputs.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
#include "source/common/network/matching/inputs.h"

#include "envoy/http/filter.h"
#include "envoy/registry/registry.h"

#include "source/common/network/utility.h"
Expand All @@ -10,9 +11,10 @@ namespace Envoy {
namespace Network {
namespace Matching {

template <>
Matcher::DataInputGetResult DestinationIPInput<MatchingData>::get(const MatchingData& data) const {
const auto& address = data.socket().connectionInfoProvider().localAddress();
template <class MatchingDataType>
Matcher::DataInputGetResult
DestinationIPInput<MatchingDataType>::get(const MatchingDataType& data) const {

@zhxie zhxie Apr 13, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these template implementations should be inlined and moved to "inputs.h". Otherwise, we may face with linking issues.

@kyessenov kyessenov Apr 13, 2022

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you elaborate? I've seen primary templates in .cc files used, and that works as long as it's used only within the .cc file.

@zhxie zhxie Apr 14, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original contains a full template specialization implementation for MatchingData and another for UdpMatchingData. A template specialization implementation can be located in .cc, and the linker will recognize their symbols for linking. Now, we are going to change to a template member implementation, the implementation should be moved to ".h". We may come up with

// inputs.h
template <class MatchingDataType>
class SourceIPInput : public Matcher::DataInput<MatchingDataType> {
  // template member implementation
  Matcher::DataInputGetResult get(const MatchingDataType& data) const override {}
}

// template specialization definition for UdpMatchingData
template <>
Matcher::DataInputGetResult
DestinationIPInput<UdpMatchingData>::get(const UdpMatchingData& data) const;

// inputs.cc
// template specialization implementation for UdpMatchingData
template <>
Matcher::DataInputGetResult
DestinationIPInput<UdpMatchingData>::get(const UdpMatchingData& data) const {}

const auto& address = data.connectionInfoProvider().localAddress();
if (address->type() != Network::Address::Type::Ip) {
return {Matcher::DataInputGetResult::DataAvailability::AllDataAvailable, absl::nullopt};
}
Expand All @@ -31,10 +33,10 @@ DestinationIPInput<UdpMatchingData>::get(const UdpMatchingData& data) const {
address.ip()->addressAsString()};
}

template <>
template <class MatchingDataType>
Matcher::DataInputGetResult
DestinationPortInput<MatchingData>::get(const MatchingData& data) const {
const auto& address = data.socket().connectionInfoProvider().localAddress();
DestinationPortInput<MatchingDataType>::get(const MatchingDataType& data) const {
const auto& address = data.connectionInfoProvider().localAddress();
if (address->type() != Network::Address::Type::Ip) {
return {Matcher::DataInputGetResult::DataAvailability::AllDataAvailable, absl::nullopt};
}
Expand All @@ -53,9 +55,10 @@ DestinationPortInput<UdpMatchingData>::get(const UdpMatchingData& data) const {
absl::StrCat(address.ip()->port())};
}

template <>
Matcher::DataInputGetResult SourceIPInput<MatchingData>::get(const MatchingData& data) const {
const auto& address = data.socket().connectionInfoProvider().remoteAddress();
template <class MatchingDataType>
Matcher::DataInputGetResult
SourceIPInput<MatchingDataType>::get(const MatchingDataType& data) const {
const auto& address = data.connectionInfoProvider().remoteAddress();
if (address->type() != Network::Address::Type::Ip) {
return {Matcher::DataInputGetResult::DataAvailability::AllDataAvailable, absl::nullopt};
}
Expand All @@ -73,9 +76,10 @@ Matcher::DataInputGetResult SourceIPInput<UdpMatchingData>::get(const UdpMatchin
address.ip()->addressAsString()};
}

template <>
Matcher::DataInputGetResult SourcePortInput<MatchingData>::get(const MatchingData& data) const {
const auto& address = data.socket().connectionInfoProvider().remoteAddress();
template <class MatchingDataType>
Matcher::DataInputGetResult
SourcePortInput<MatchingDataType>::get(const MatchingDataType& data) const {
const auto& address = data.connectionInfoProvider().remoteAddress();
if (address->type() != Network::Address::Type::Ip) {
return {Matcher::DataInputGetResult::DataAvailability::AllDataAvailable, absl::nullopt};
}
Expand All @@ -94,25 +98,32 @@ SourcePortInput<UdpMatchingData>::get(const UdpMatchingData& data) const {
absl::StrCat(address.ip()->port())};
}

Matcher::DataInputGetResult DirectSourceIPInput::get(const MatchingData& data) const {
const auto& address = data.socket().connectionInfoProvider().directRemoteAddress();
template <class MatchingDataType>
Matcher::DataInputGetResult
DirectSourceIPInput<MatchingDataType>::get(const MatchingDataType& data) const {
const auto& address = data.connectionInfoProvider().directRemoteAddress();
if (address->type() != Network::Address::Type::Ip) {
return {Matcher::DataInputGetResult::DataAvailability::AllDataAvailable, absl::nullopt};
}
return {Matcher::DataInputGetResult::DataAvailability::AllDataAvailable,
address->ip()->addressAsString()};
}

Matcher::DataInputGetResult SourceTypeInput::get(const MatchingData& data) const {
const bool is_local_connection = Network::Utility::isSameIpOrLoopback(data.socket());
template <class MatchingDataType>
Matcher::DataInputGetResult
SourceTypeInput<MatchingDataType>::get(const MatchingDataType& data) const {
const bool is_local_connection =
Network::Utility::isSameIpOrLoopback(data.connectionInfoProvider());
if (is_local_connection) {
return {Matcher::DataInputGetResult::DataAvailability::AllDataAvailable, "local"};
}
return {Matcher::DataInputGetResult::DataAvailability::AllDataAvailable, absl::nullopt};
}

Matcher::DataInputGetResult ServerNameInput::get(const MatchingData& data) const {
const auto server_name = data.socket().requestedServerName();
template <class MatchingDataType>
Matcher::DataInputGetResult
ServerNameInput<MatchingDataType>::get(const MatchingDataType& data) const {
const auto server_name = data.connectionInfoProvider().requestedServerName();
if (!server_name.empty()) {
return {Matcher::DataInputGetResult::DataAvailability::AllDataAvailable,
std::string(server_name)};
Expand All @@ -138,17 +149,53 @@ Matcher::DataInputGetResult ApplicationProtocolInput::get(const MatchingData& da
return {Matcher::DataInputGetResult::DataAvailability::AllDataAvailable, absl::nullopt};
}

class DestinationIPInputFactory : public DestinationIPInputBaseFactory<MatchingData> {};
class UdpDestinationIPInputFactory : public DestinationIPInputBaseFactory<UdpMatchingData> {};
class HttpDestinationIPInputFactory : public DestinationIPInputBaseFactory<Http::HttpMatchingData> {
};
REGISTER_FACTORY(DestinationIPInputFactory, Matcher::DataInputFactory<MatchingData>);
REGISTER_FACTORY(UdpDestinationIPInputFactory, Matcher::DataInputFactory<UdpMatchingData>);
REGISTER_FACTORY(HttpDestinationIPInputFactory, Matcher::DataInputFactory<Http::HttpMatchingData>);

class DestinationPortInputFactory : public DestinationPortInputBaseFactory<MatchingData> {};
class UdpDestinationPortInputFactory : public DestinationPortInputBaseFactory<UdpMatchingData> {};
class HttpDestinationPortInputFactory
: public DestinationPortInputBaseFactory<Http::HttpMatchingData> {};
REGISTER_FACTORY(DestinationPortInputFactory, Matcher::DataInputFactory<MatchingData>);
REGISTER_FACTORY(UdpDestinationPortInputFactory, Matcher::DataInputFactory<UdpMatchingData>);
REGISTER_FACTORY(HttpDestinationPortInputFactory,
Matcher::DataInputFactory<Http::HttpMatchingData>);

class SourceIPInputFactory : public SourceIPInputBaseFactory<MatchingData> {};
class UdpSourceIPInputFactory : public SourceIPInputBaseFactory<UdpMatchingData> {};
class HttpSourceIPInputFactory : public SourceIPInputBaseFactory<Http::HttpMatchingData> {};
REGISTER_FACTORY(SourceIPInputFactory, Matcher::DataInputFactory<MatchingData>);
REGISTER_FACTORY(UdpSourceIPInputFactory, Matcher::DataInputFactory<UdpMatchingData>);
REGISTER_FACTORY(HttpSourceIPInputFactory, Matcher::DataInputFactory<Http::HttpMatchingData>);

class SourcePortInputFactory : public SourcePortInputBaseFactory<MatchingData> {};
class UdpSourcePortInputFactory : public SourcePortInputBaseFactory<UdpMatchingData> {};
class HttpSourcePortInputFactory : public SourcePortInputBaseFactory<Http::HttpMatchingData> {};
REGISTER_FACTORY(SourcePortInputFactory, Matcher::DataInputFactory<MatchingData>);
REGISTER_FACTORY(UdpSourcePortInputFactory, Matcher::DataInputFactory<UdpMatchingData>);
REGISTER_FACTORY(HttpSourcePortInputFactory, Matcher::DataInputFactory<Http::HttpMatchingData>);

class DirectSourceIPInputFactory : public DirectSourceIPInputBaseFactory<MatchingData> {};
class HttpDirectSourceIPInputFactory
: public DirectSourceIPInputBaseFactory<Http::HttpMatchingData> {};
REGISTER_FACTORY(DirectSourceIPInputFactory, Matcher::DataInputFactory<MatchingData>);
REGISTER_FACTORY(SourceTypeInputFactory, Matcher::DataInputFactory<MatchingData>);
REGISTER_FACTORY(HttpDirectSourceIPInputFactory, Matcher::DataInputFactory<Http::HttpMatchingData>);

class ServerNameInputFactory : public ServerNameInputBaseFactory<MatchingData> {};
class HttpServerNameInputFactory : public ServerNameInputBaseFactory<Http::HttpMatchingData> {};
REGISTER_FACTORY(ServerNameInputFactory, Matcher::DataInputFactory<MatchingData>);
REGISTER_FACTORY(HttpServerNameInputFactory, Matcher::DataInputFactory<Http::HttpMatchingData>);

class SourceTypeInputFactory : public SourceTypeInputBaseFactory<MatchingData> {};
class HttpSourceTypeInputFactory : public SourceTypeInputBaseFactory<Http::HttpMatchingData> {};
REGISTER_FACTORY(SourceTypeInputFactory, Matcher::DataInputFactory<MatchingData>);
REGISTER_FACTORY(HttpSourceTypeInputFactory, Matcher::DataInputFactory<Http::HttpMatchingData>);

REGISTER_FACTORY(TransportProtocolInputFactory, Matcher::DataInputFactory<MatchingData>);
REGISTER_FACTORY(ApplicationProtocolInputFactory, Matcher::DataInputFactory<MatchingData>);

Expand Down
67 changes: 33 additions & 34 deletions source/common/network/matching/inputs.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,10 +48,6 @@ class DestinationIPInputBaseFactory
MatchingDataType>("destination_ip") {}
};

class DestinationIPInputFactory : public DestinationIPInputBaseFactory<MatchingData> {};

class UdpDestinationIPInputFactory : public DestinationIPInputBaseFactory<UdpMatchingData> {};

template <class MatchingDataType>
class DestinationPortInput : public Matcher::DataInput<MatchingDataType> {
public:
Expand All @@ -71,10 +67,6 @@ class DestinationPortInputBaseFactory
MatchingDataType>("destination_port") {}
};

class DestinationPortInputFactory : public DestinationPortInputBaseFactory<MatchingData> {};

class UdpDestinationPortInputFactory : public DestinationPortInputBaseFactory<UdpMatchingData> {};

template <class MatchingDataType>
class SourceIPInput : public Matcher::DataInput<MatchingDataType> {
public:
Expand All @@ -93,10 +85,6 @@ class SourceIPInputBaseFactory
MatchingDataType>("source_ip") {}
};

class SourceIPInputFactory : public SourceIPInputBaseFactory<MatchingData> {};

class UdpSourceIPInputFactory : public SourceIPInputBaseFactory<UdpMatchingData> {};

template <class MatchingDataType>
class SourcePortInput : public Matcher::DataInput<MatchingDataType> {
public:
Expand All @@ -115,48 +103,59 @@ class SourcePortInputBaseFactory
MatchingDataType>("source_port") {}
};

class SourcePortInputFactory : public SourcePortInputBaseFactory<MatchingData> {};

class UdpSourcePortInputFactory : public SourcePortInputBaseFactory<UdpMatchingData> {};

class DirectSourceIPInput : public Matcher::DataInput<MatchingData> {
template <class MatchingDataType>
class DirectSourceIPInput : public Matcher::DataInput<MatchingDataType> {
public:
Matcher::DataInputGetResult get(const MatchingData& data) const override;
Matcher::DataInputGetResult get(const MatchingDataType& data) const override;
};

class DirectSourceIPInputFactory
template <class MatchingDataType>
class DirectSourceIPInputBaseFactory
: public BaseFactory<
DirectSourceIPInput,
DirectSourceIPInput<MatchingDataType>,
envoy::extensions::matching::common_inputs::network::v3::DirectSourceIPInput,
MatchingData> {
MatchingDataType> {
public:
DirectSourceIPInputFactory() : BaseFactory("direct_source_ip") {}
DirectSourceIPInputBaseFactory()
: BaseFactory<DirectSourceIPInput<MatchingDataType>,
envoy::extensions::matching::common_inputs::network::v3::DirectSourceIPInput,
MatchingDataType>("direct_source_ip") {}
};

class SourceTypeInput : public Matcher::DataInput<MatchingData> {
template <class MatchingDataType>
class SourceTypeInput : public Matcher::DataInput<MatchingDataType> {
public:
Matcher::DataInputGetResult get(const MatchingData& data) const override;
Matcher::DataInputGetResult get(const MatchingDataType& data) const override;
};

class SourceTypeInputFactory
: public BaseFactory<SourceTypeInput,
template <class MatchingDataType>
class SourceTypeInputBaseFactory
: public BaseFactory<SourceTypeInput<MatchingDataType>,
envoy::extensions::matching::common_inputs::network::v3::SourceTypeInput,
MatchingData> {
MatchingDataType> {
public:
SourceTypeInputFactory() : BaseFactory("source_type") {}
SourceTypeInputBaseFactory()
: BaseFactory<SourceTypeInput<MatchingDataType>,
envoy::extensions::matching::common_inputs::network::v3::SourceTypeInput,
MatchingDataType>("source_type") {}
};

class ServerNameInput : public Matcher::DataInput<MatchingData> {
template <class MatchingDataType>
class ServerNameInput : public Matcher::DataInput<MatchingDataType> {
public:
Matcher::DataInputGetResult get(const MatchingData& data) const override;
Matcher::DataInputGetResult get(const MatchingDataType& data) const override;
};

class ServerNameInputFactory
: public BaseFactory<ServerNameInput,
template <class MatchingDataType>
class ServerNameInputBaseFactory
: public BaseFactory<ServerNameInput<MatchingDataType>,
envoy::extensions::matching::common_inputs::network::v3::ServerNameInput,
MatchingData> {
MatchingDataType> {
public:
ServerNameInputFactory() : BaseFactory("server_name") {}
ServerNameInputBaseFactory()
: BaseFactory<ServerNameInput<MatchingDataType>,
envoy::extensions::matching::common_inputs::network::v3::ServerNameInput,
MatchingDataType>("server_name") {}
};

class TransportProtocolInput : public Matcher::DataInput<MatchingData> {
Expand Down
6 changes: 3 additions & 3 deletions source/common/network/utility.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -280,16 +280,16 @@ Address::InstanceConstSharedPtr Utility::getLocalAddress(const Address::IpVersio
return ret;
}

bool Utility::isSameIpOrLoopback(const ConnectionSocket& socket) {
bool Utility::isSameIpOrLoopback(const ConnectionInfoProvider& connection_info_provider) {
// These are local:
// - Pipes
// - Sockets to a loopback address
// - Sockets where the local and remote address (ignoring port) are the same
const auto& remote_address = socket.connectionInfoProvider().remoteAddress();
const auto& remote_address = connection_info_provider.remoteAddress();
if (remote_address->type() == Address::Type::Pipe || isLoopbackAddress(*remote_address)) {
return true;
}
const auto local_ip = socket.connectionInfoProvider().localAddress()->ip();
const auto local_ip = connection_info_provider.localAddress()->ip();
const auto remote_ip = remote_address->ip();
if (remote_ip != nullptr && local_ip != nullptr &&
remote_ip->addressAsString() == local_ip->addressAsString()) {
Expand Down
Loading