Skip to content
Closed
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions include/envoy/common/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -91,3 +91,11 @@ envoy_cc_library(
name = "scope_tracker_interface",
hdrs = ["scope_tracker.h"],
)

envoy_cc_library(
name = "case_insensitive",
hdrs = ["case_insensitive.h"],
deps = [
"//source/common/common:assert_lib",
],
)
71 changes: 71 additions & 0 deletions include/envoy/common/case_insensitive.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
#pragma once

#include <string>

#include "common/common/assert.h"

#include "absl/strings/ascii.h"
#include "absl/strings/string_view.h"

namespace Envoy {

/**
* Wrapper for case insensitive string to generally avoid needless case insensitive compares. The
* wrapper will uniformly convert the string content to lowercase.
*/
class LowerCaseStrBase {
public:
explicit LowerCaseStrBase(absl::string_view new_string) : string_(new_string) { lower(); }

LowerCaseStrBase(LowerCaseStrBase&&) = default;
LowerCaseStrBase& operator=(LowerCaseStrBase&&) = default;

LowerCaseStrBase(const LowerCaseStrBase&) = default;
LowerCaseStrBase& operator=(const LowerCaseStrBase&) = default;

const std::string& get() const { return string_; }
std::string& get() { return string_; }

bool operator==(const LowerCaseStrBase& rhs) const { return string_ == rhs.string_; }
bool operator!=(const LowerCaseStrBase& rhs) const { return string_ != rhs.string_; }
bool operator<(const LowerCaseStrBase& rhs) const { return string_.compare(rhs.string_) < 0; }

friend std::ostream& operator<<(std::ostream& os, const LowerCaseStrBase& lower_case_string) {
return os << lower_case_string.string_;
}

protected:
void lower() {
std::transform(string_.begin(), string_.end(), string_.begin(), absl::ascii_tolower);
}

std::string string_;
};

template <bool (*V)(absl::string_view)> class ValidatedLowerCaseStr : public LowerCaseStrBase {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure why we need this to be a template.

@wbpcode wbpcode May 27, 2021

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lizan Because I want to get a protocol-independent lower case string and be able to set a different validator. The template is used so that the validator can be pluggable.

Here are some discussions about this question in original PR #16049.
@jmarantz #16049 (comment)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK I see, then this should be commented why it is template. and in Http, it should be named like validateHttpLowerCaseString.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH my recollection from the.other PR is that there are a few jumbled concepts here and I want to just quickly mention my line of thinking.

IMO if you really need a validator, forget whether there's a compelling reason to prefer

  • template
  • virtual method
  • function-pointer
    But if we do use a function-pointer then in IMO it could be a std::function rather than a templatized function pointer.

Also I think once we are plugging in validation, there's no compelling reason to use LowerCaseString just for Tracing's sake, unless there's something about Tracing that wants strings to be lower-cased. IDK whether that's the case.

That said, I have no fundamental objection to moving LowerCaseString to common as long as it doesn't make anything slower for HTTP headers, which are performance critical.

I don't understand why LowerCaseString has to be paired with a validator though. Can't you just leave LowerCaseString exactly like it is, and move it out of http into common? That seems like a legitimate concept on its own. You could make another layer that adds validation if this is helpful in some way.

@wbpcode wbpcode May 28, 2021

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lizan I will add some new comments to explain why we need this template. 👌
In Http, I kept the original LowerCaseString name so as not to affect the existing code as much as possible. After all, the name Http::LowerCaseString is already widely used. 🤔

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jmarantz

  • First, about the performance: This modification will not affect the performance of Http::LowerCaseString since there is no new logic. The introduction of templates may increase the compilation overhead slightly.
  • Second, about the relationship between validator and LowerCaseString: I provide a class LowerCaseStrBase as a common implementation of LowerCaseString which with no any validation and can be used directly. Then I provide ValidatedLowerCaseStr template just as a helper for who wants to add validation to LowerCaseString, for example, Http.
    So that means I've got two layers as you think. One layer is the plain LowerCaseString, and the second layer is the ValidatedLowerCaseStr that can mount a different validator.

@wbpcode wbpcode May 28, 2021

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jmarantz At the source code level, it is move. I just kept the previous name LowerCaseString instead of renaming it to ValidatedHttpLowerCaseString or something like that to make PR clean.

using LowerCaseString = Envoy::ValidatedLowerCaseStr<validHeaderString>;

One of the key reasons Trace needs LowerCaseString is performance. But that can be discussed in the new PR.

This PR doesn't really have anything to do with Trace so far. Although the cause is Trace, for now this PR can be seen as a PR that simply strips LowerCaseString out of Http to make it more generic.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK...sorry, TBH I'm skeptical about the overall strategy and also concerned there may be overhead added to http headers.

But for the moment maybe it'd be better to focus on strategy. Do you want to describe the high level of what you are trying to achieve in a bug or a google doc? It might be easier to iterate there.

@wbpcode wbpcode May 28, 2021

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for you comments and patience. @jmarantz

In the original PR #16049, there is the high level design of the general tracing. And I remember I had a more detailed discussion with you in slack earlier.

For now, I think we can try to focus on this PR first. Because this PR don't involve the design of the new general tracing yet, just some pre-work. This is also the reason why I split the original PR into multiple PRs.

I think we can discuss the design of general tracing in detail in the new PR (One or two weeks after the completion of this PR).

For this PR I think the most important points are the following:

  • whether the PR is clean or not.
  • whether the PR is able to keep the existing functionality working.
  • whether the PR can guarantee that the existing performance is not affected.
  • Whether the PR achieves its purpose: to strip LowerCaseString from Http and get a general LowerCaseString.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think PR is probably OK but I'll let Lizan review the details. It'd be better not to do it unless it's needed though.

I am not understanding why you need lower-case semantics at all for the new tracers. There may be a good reason but it's not clear. Iterating in a doc may be easier, rather than this PR.

public:
ValidatedLowerCaseStr(LowerCaseStrBase&& rhs) noexcept : LowerCaseStrBase(std::move(rhs)) {
ASSERT(valid());
}
ValidatedLowerCaseStr& operator=(LowerCaseStrBase&& rhs) noexcept {
string_ = std::move(rhs.get());
ASSERT(valid());
return *this;
}

ValidatedLowerCaseStr(const LowerCaseStrBase& rhs) : LowerCaseStrBase(rhs) { ASSERT(valid()); }
ValidatedLowerCaseStr& operator=(const LowerCaseStrBase& rhs) {
string_ = rhs.get();
ASSERT(valid());
return *this;
}

explicit ValidatedLowerCaseStr(absl::string_view new_string) : LowerCaseStrBase(new_string) {
ASSERT(valid());
}

private:
bool valid() const { return V(string_); }
};

} // namespace Envoy
1 change: 1 addition & 0 deletions include/envoy/http/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,7 @@ envoy_cc_library(
],
deps = [
":header_formatter_interface",
"//include/envoy/common:case_insensitive",
"//source/common/common:assert_lib",
"//source/common/common:hash_lib",
],
Expand Down
47 changes: 4 additions & 43 deletions include/envoy/http/header_map.h
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
#include <string>
#include <vector>

#include "envoy/common/case_insensitive.h"
#include "envoy/common/optref.h"
#include "envoy/common/pure.h"
#include "envoy/http/header_formatter.h"
Expand All @@ -24,7 +25,7 @@ namespace Http {

// Used by ASSERTs to validate internal consistency. E.g. valid HTTP header keys/values should
// never contain embedded NULLs.
static inline bool validHeaderString(absl::string_view s) {
inline bool validHeaderString(absl::string_view s) {
// If you modify this list of illegal embedded characters you will probably
// want to change header_map_fuzz_impl_test at the same time.
for (const char c : s) {
Expand All @@ -43,49 +44,9 @@ static inline bool validHeaderString(absl::string_view s) {
}

/**
* Wrapper for a lower case string used in header operations to generally avoid needless case
* insensitive compares.
* Wrapper for a lower case string used in header operations.
*/
class LowerCaseString {
public:
LowerCaseString(LowerCaseString&& rhs) noexcept : string_(std::move(rhs.string_)) {
ASSERT(valid());
}
LowerCaseString& operator=(LowerCaseString&& rhs) noexcept {
string_ = std::move(rhs.string_);
ASSERT(valid());
return *this;
}

LowerCaseString(const LowerCaseString& rhs) : string_(rhs.string_) { ASSERT(valid()); }
LowerCaseString& operator=(const LowerCaseString& rhs) {
string_ = std::move(rhs.string_);
ASSERT(valid());
return *this;
}

explicit LowerCaseString(const std::string& new_string) : string_(new_string) {
ASSERT(valid());
lower();
}

const std::string& get() const { return string_; }
bool operator==(const LowerCaseString& rhs) const { return string_ == rhs.string_; }
bool operator!=(const LowerCaseString& rhs) const { return string_ != rhs.string_; }
bool operator<(const LowerCaseString& rhs) const { return string_.compare(rhs.string_) < 0; }

friend std::ostream& operator<<(std::ostream& os, const LowerCaseString& lower_case_string) {
return os << lower_case_string.string_;
}

private:
void lower() {
std::transform(string_.begin(), string_.end(), string_.begin(), absl::ascii_tolower);
}
bool valid() const { return validHeaderString(string_); }

std::string string_;
};
using LowerCaseString = Envoy::ValidatedLowerCaseStr<validHeaderString>;

/**
* Convenient type for a vector of lower case string and string pair.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ class OpenTracingHTTPHeadersWriter : public opentracing::HTTPHeadersWriter {
// opentracing::HTTPHeadersWriter
opentracing::expected<void> Set(opentracing::string_view key,
opentracing::string_view value) const override {
Http::LowerCaseString lowercase_key{key};
Http::LowerCaseString lowercase_key{{key.data(), key.size()}};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be moved to a different PR? Doesn't seem like this should be needed in this one.

request_headers_.remove(lowercase_key);
request_headers_.addCopy(std::move(lowercase_key), {value.data(), value.size()});
return {};
Expand All @@ -50,7 +50,7 @@ class OpenTracingHTTPHeadersReader : public opentracing::HTTPHeadersReader {
// opentracing::HTTPHeadersReader
opentracing::expected<opentracing::string_view>
LookupKey(opentracing::string_view key) const override {
const auto entry = request_headers_.get(Http::LowerCaseString{key});
const auto entry = request_headers_.get(Http::LowerCaseString{{key.data(), key.size()}});
if (!entry.empty()) {
// This is an implicitly untrusted header, so only the first value is used.
return opentracing::string_view{entry[0]->value().getStringView().data(),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@

#include "gtest/gtest.h"

using Envoy::Http::HeaderValueOf;
Comment thread
lizan marked this conversation as resolved.

namespace Envoy {
namespace Extensions {
namespace HttpFilters {
Expand Down