Skip to content

Add few more utility functions for address and http proto's. #2504

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
htuch merged 8 commits into from
Feb 2, 2018
Merged

Add few more utility functions for address and http proto's. #2504

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
da62afa
Add few more utility functions for address and http proto's.
Feb 1, 2018
858d842
Add more ut.
Feb 1, 2018
ab15cc3
Remove if check.
Feb 1, 2018
a519fb3
Consolidate usage.
Feb 1, 2018
57c6f26
Address review feedback.
Feb 1, 2018
e4156e8
Move protocol string to headers and access via utility.
Feb 2, 2018
49a3570
Review comments.
Feb 2, 2018
df0f0bf
Merge remote-tracking branch 'upstream/master' into ext-auth-review-g…
Feb 2, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions include/envoy/http/header_map.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,11 @@ class HeaderString {
*/
const char* c_str() const { return buffer_.ref_; }

/**
* @return a std::string.
*/
std::string getString() const { return std::string(buffer_.ref_, string_length_); }

/**
* Return the string to a default state. Reference strings are not touched. Both inline/dynamic
* strings are reset to zero size.
Expand Down
20 changes: 20 additions & 0 deletions include/envoy/http/protocol.h
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#pragma once
#include <string>

@alyssawilk alyssawilk Feb 2, 2018

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this include be removed?


namespace Envoy {
namespace Http {
Expand All @@ -10,5 +11,24 @@ namespace Http {
enum class Protocol { Http10, Http11, Http2 };
const size_t NumProtocols = 3;

static const std::string DefaultString = "";
static const std::string Http10String = "HTTP/1.0";

@htuch htuch Feb 1, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Static non-POD is not allowed in the Envoy style, see https://github.com/envoyproxy/envoy/blob/master/STYLE.md. This could probably just be const char DefaultString = "" etc.

@alyssawilk alyssawilk Feb 1, 2018

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also put these in a struct in headers.h
I was wavering on asking that anyway - they're not so much headers as "part of the firstline" but it's where I'd look for predefined header-type strings.

@mattklein123 mattklein123 Feb 1, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO we should continue to return string references for perf reasons. I still think that the non-POD rule for std::string is a little silly, but, the rules are the rules. I would just use the constant class pattern we use elsewhere like in Http::Headers.

@saumoh saumoh Feb 1, 2018

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattklein123 i think you are referring to https://github.com/envoyproxy/envoy/blob/master/source/common/http/headers.h
if so then i can certainly adopt that patter here as well.
@alyssawilk is it ok if i keep the new constant class here?

@alyssawilk alyssawilk Feb 1, 2018

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either way is fine with me

@saumoh saumoh Feb 1, 2018

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattklein123 @alyssawilk Last push addresses this. Adding NOT_REACHED in this file causes a circular dependency so i added a abort(). If that is not what we want then I can move both constants and the getProtocolString to the common/http/headers.h file.
please let me know if that would be preferable.

@mattklein123 mattklein123 Feb 2, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would recommend moving into common and using NOT_REACHED. You can either put in headers.h or feel free to just put in Http::Utility

@saumoh saumoh Feb 2, 2018

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool. moved. please review when u folks get a chance. thanks

static const std::string Http11String = "HTTP/1.1";
static const std::string Http2String = "HTTP/2";

inline const std::string& getProtocolString(const Protocol& p) {

@mattklein123 mattklein123 Feb 1, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Protocol should be passed by value: Protocol p

switch (p) {
case Protocol::Http10:
return Http10String;
case Protocol::Http11:
return Http11String;
case Protocol::Http2:
return Http2String;
default:

@alyssawilk alyssawilk Feb 1, 2018

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mild preference for no default - I'd rather have a compile fail if we add a new Protocol and forget to update this function.

@mattklein123 mattklein123 Feb 1, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 just return directly and avoid default. Add NOT_REACHED at end of function if necessary.

break;
}
return DefaultString;
}

} // namespace Http
} // namespace Envoy
1 change: 1 addition & 0 deletions source/common/access_log/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ envoy_cc_library(
"//include/envoy/thread_local:thread_local_interface",
"//include/envoy/upstream:cluster_manager_interface",
"//source/common/grpc:async_client_lib",
"//source/common/network:utility_lib",
"@envoy_api//envoy/api/v2/filter/accesslog:accesslog_cc",
"@envoy_api//envoy/config/accesslog/v2:als_cc",
"@envoy_api//envoy/service/accesslog/v2:als_cc",
Expand Down
13 changes: 1 addition & 12 deletions source/common/access_log/access_log_formatter.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,21 +26,10 @@ FormatterPtr AccessLogFormatUtils::defaultAccessLogFormatter() {
return FormatterPtr{new FormatterImpl(DEFAULT_FORMAT)};
}

static const std::string Http10String = "HTTP/1.0";
static const std::string Http11String = "HTTP/1.1";
static const std::string Http2String = "HTTP/2";

const std::string&
AccessLogFormatUtils::protocolToString(const Optional<Http::Protocol>& protocol) {
if (protocol.valid()) {
switch (protocol.value()) {
case Http::Protocol::Http10:
return Http10String;
case Http::Protocol::Http11:
return Http11String;
case Http::Protocol::Http2:
return Http2String;
}
return Http::getProtocolString(protocol.value());
} else {
return UnspecifiedValueString;
}
Expand Down
32 changes: 12 additions & 20 deletions source/common/access_log/grpc_access_log_impl.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

#include "common/common/assert.h"
#include "common/http/header_map_impl.h"
#include "common/network/utility.h"

namespace Envoy {
namespace AccessLog {
Expand Down Expand Up @@ -64,18 +65,6 @@ HttpGrpcAccessLog::HttpGrpcAccessLog(
: filter_(std::move(filter)), config_(config),
grpc_access_log_streamer_(grpc_access_log_streamer) {}

void HttpGrpcAccessLog::addressToAccessLogAddress(
envoy::api::v2::Address& proto_address, const Network::Address::Instance& network_address) {
if (network_address.type() == Network::Address::Type::Pipe) {
proto_address.mutable_pipe()->set_path(network_address.asString());
} else {
ASSERT(network_address.type() == Network::Address::Type::Ip);
auto* socket_address = proto_address.mutable_socket_address();
socket_address->set_address(network_address.ip()->addressAsString());
socket_address->set_port_value(network_address.ip()->port());
}
}

void HttpGrpcAccessLog::responseFlagsToAccessLogResponseFlags(
envoy::api::v2::filter::accesslog::AccessLogCommon& common_access_log,
const RequestInfo::RequestInfo& request_info) {
Expand Down Expand Up @@ -161,10 +150,12 @@ void HttpGrpcAccessLog::log(const Http::HeaderMap* request_headers,
// TODO(mattklein123): Populate time_to_first_downstream_tx_byte field.
// TODO(mattklein123): Populate metadata field and wire up to filters.
auto* common_properties = log_entry->mutable_common_properties();
addressToAccessLogAddress(*common_properties->mutable_downstream_remote_address(),
*request_info.downstreamRemoteAddress());
addressToAccessLogAddress(*common_properties->mutable_downstream_local_address(),
*request_info.downstreamLocalAddress());
Network::Utility::addressToProtobufAddress(
*request_info.downstreamRemoteAddress(),
*common_properties->mutable_downstream_remote_address());
Network::Utility::addressToProtobufAddress(
*request_info.downstreamLocalAddress(),
*common_properties->mutable_downstream_local_address());
common_properties->mutable_start_time()->MergeFrom(
Protobuf::util::TimeUtil::MicrosecondsToTimestamp(
std::chrono::duration_cast<std::chrono::microseconds>(
Expand All @@ -183,13 +174,14 @@ void HttpGrpcAccessLog::log(const Http::HeaderMap* request_headers,
common_properties->mutable_time_to_last_downstream_tx_byte()->MergeFrom(
Protobuf::util::TimeUtil::MicrosecondsToDuration(request_info.duration().count()));
if (request_info.upstreamHost() != nullptr) {
addressToAccessLogAddress(*common_properties->mutable_upstream_remote_address(),
*request_info.upstreamHost()->address());
Network::Utility::addressToProtobufAddress(
*request_info.upstreamHost()->address(),
*common_properties->mutable_upstream_remote_address());
common_properties->set_upstream_cluster(request_info.upstreamHost()->cluster().name());
}
if (request_info.upstreamLocalAddress() != nullptr) {
addressToAccessLogAddress(*common_properties->mutable_upstream_local_address(),
*request_info.upstreamLocalAddress());
Network::Utility::addressToProtobufAddress(
*request_info.upstreamLocalAddress(), *common_properties->mutable_upstream_local_address());
}
responseFlagsToAccessLogResponseFlags(*common_properties, request_info);

Expand Down
2 changes: 0 additions & 2 deletions source/common/access_log/grpc_access_log_impl.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,8 +114,6 @@ class HttpGrpcAccessLog : public Instance {
const envoy::config::accesslog::v2::HttpGrpcAccessLogConfig& config,
GrpcAccessLogStreamerSharedPtr grpc_access_log_streamer);

static void addressToAccessLogAddress(envoy::api::v2::Address& proto_address,
const Network::Address::Instance& network_address);
static void responseFlagsToAccessLogResponseFlags(
envoy::api::v2::filter::accesslog::AccessLogCommon& common_access_log,
const RequestInfo::RequestInfo& request_info);
Expand Down
1 change: 1 addition & 0 deletions source/common/network/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,7 @@ envoy_cc_library(
"//source/common/common:assert_lib",
"//source/common/common:utility_lib",
"//source/common/protobuf",
"@envoy_api//envoy/api/v2:address_cc",
"@envoy_api//envoy/api/v2:base_cc",
],
)
12 changes: 12 additions & 0 deletions source/common/network/utility.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -371,5 +371,17 @@ absl::uint128 Utility::flipOrder(const absl::uint128& input) {
return result;
}

void Utility::addressToProtobufAddress(const Address::Instance& address,

@mattklein123 mattklein123 Feb 1, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already implemented here: https://github.com/envoyproxy/envoy/blob/master/source/common/access_log/grpc_access_log_impl.cc#L67 please consolidate

Also, a dedicated function like this needs dedicated tests.

@saumoh saumoh Feb 1, 2018 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added tests and consolidated. thanks.

envoy::api::v2::Address& proto_address) {
if (address.type() == Address::Type::Pipe) {
proto_address.mutable_pipe()->set_path(address.asString());
} else {
ASSERT(address.type() == Address::Type::Ip);
auto* socket_address = proto_address.mutable_socket_address();
socket_address->set_address(address.ip()->addressAsString());
socket_address->set_port_value(address.ip()->port());
}
}

} // namespace Network
} // namespace Envoy
8 changes: 8 additions & 0 deletions source/common/network/utility.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,14 @@ class Utility {
*/
static absl::uint128 Ip6htonl(const absl::uint128& address);

/**
* Copies the address instance into the protobuf representation of an address.
* @param address is the address to be copied into the protobuf representation of this address.
* @param proto_address is the protobuf address to which the address instance is copied into.
*/
static void addressToProtobufAddress(const Address::Instance& address,
envoy::api::v2::Address& proto_address);

private:
static void throwWithMalformedIp(const std::string& ip_address);

Expand Down
13 changes: 13 additions & 0 deletions test/common/http/header_map_impl_test.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -283,6 +283,19 @@ TEST(HeaderStringTest, All) {
EXPECT_FALSE(string.caseInsensitiveContains("keep-alive"));
EXPECT_FALSE(string.caseInsensitiveContains(""));
}

// getString
{
std::string static_string("HELLO");
HeaderString headerString1(static_string);
std::string retString1 = headerString1.getString();
EXPECT_EQ("HELLO", retString1);
EXPECT_EQ(5U, retString1.size());

HeaderString headerString2;
std::string retString2 = headerString2.getString();
EXPECT_EQ(0U, retString2.size());
}
}

TEST(HeaderMapImplTest, InlineInsert) {
Expand Down
15 changes: 15 additions & 0 deletions test/common/network/utility_test.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,21 @@ TEST(NetworkUtility, AnyAddress) {
}
}

TEST(NetworkUtility, AddressToProtobufAddress) {
{
envoy::api::v2::Address proto_address;
Address::Ipv4Instance address("127.0.0.1");
Utility::addressToProtobufAddress(address, proto_address);
EXPECT_EQ(true, proto_address.has_socket_address());

@alyssawilk alyssawilk Feb 1, 2018

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For completeness, mind checking the value and port here, and adding a test case for pipe?

}
{
envoy::api::v2::Address proto_address;
Address::PipeInstance address("/hello");
Utility::addressToProtobufAddress(address, proto_address);
EXPECT_EQ(false, proto_address.has_socket_address());
}
}

TEST(PortRangeListTest, Errors) {
{
std::string port_range_str = "a1";
Expand Down