Skip to content
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
2b8bb98
Implement SpdyUnsafeArena using SpdySimpleArena.
danzh1989 Apr 16, 2019
4cdc39f
de-dup header file
danzh1989 Apr 16, 2019
9a925f0
fix macros, blocked by linked_hash_map
danzh1989 Apr 17, 2019
8d15493
update tar ball
danzh1989 Apr 30, 2019
3dd8ed9
Merge branch 'master' into spdy_arena
danzh1989 Apr 30, 2019
b5b2a94
plug in SimpleLinkedHashMap
danzh1989 May 2, 2019
05df3ee
format
danzh1989 May 2, 2019
171d832
Merge branch 'master' into spdy_arena
danzh1989 May 2, 2019
9e1bb66
add visibility
danzh1989 May 2, 2019
af85c9c
fix depenedency
danzh1989 May 2, 2019
55665ca
add visibility
danzh1989 May 2, 2019
d2f9641
add quic_containers_test
danzh1989 May 2, 2019
5fce010
inline some impl functions
danzh1989 May 3, 2019
7f1e153
break down spdy_core_lib
danzh1989 May 6, 2019
3f84116
Merge branch 'master' into spdy_arena
danzh1989 May 6, 2019
ca59554
fix #include
danzh1989 May 6, 2019
f2c37c7
fix format
danzh1989 May 7, 2019
6dedd7e
Merge branch 'master' into spdy_arena
danzh1989 May 8, 2019
fb42607
fix test deps
danzh1989 May 8, 2019
60b306a
format
danzh1989 May 8, 2019
e50d1a7
fix deps agains
danzh1989 May 8, 2019
67ce36b
fix deps
danzh1989 May 9, 2019
a4608c3
add dummy .cc file
danzh1989 May 15, 2019
5e2d064
add dummy function to logging_impl.h
danzh1989 May 17, 2019
0db82cb
rename dummy
danzh1989 May 17, 2019
52a9269
inline dummy
danzh1989 May 17, 2019
0cacc00
take out quic_logging
danzh1989 May 20, 2019
8de6228
Merge branch 'master' into spdy_arena
danzh1989 May 20, 2019
b15ec53
remove quiche_logging_impl.cc
danzh1989 May 20, 2019
60f982e
move quic_logging into base
danzh1989 May 21, 2019
b6056c2
break circular deps
danzh1989 May 21, 2019
5bcde48
format
danzh1989 May 21, 2019
c7e16e6
Merge branch 'master' into spdy_arena
danzh1989 May 21, 2019
09f9f7a
remove out-dated comment
danzh1989 May 21, 2019
792282d
format
danzh1989 May 21, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 72 additions & 0 deletions bazel/external/quiche.BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,7 @@ envoy_cc_library(
"quiche/spdy/platform/api/spdy_estimate_memory_usage.h",
"quiche/spdy/platform/api/spdy_export.h",
"quiche/spdy/platform/api/spdy_flags.h",
"quiche/spdy/platform/api/spdy_macros.h",
"quiche/spdy/platform/api/spdy_mem_slice.h",
"quiche/spdy/platform/api/spdy_ptr_util.h",
"quiche/spdy/platform/api/spdy_string.h",
Expand Down Expand Up @@ -147,6 +148,29 @@ envoy_cc_library(
deps = ["@envoy//source/extensions/quic_listeners/quiche/platform:spdy_platform_unsafe_arena_impl_lib"],
)

envoy_cc_library(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we discussed offline, can you split this library into smaller ones, mirroring the internal libraries?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LG modulo a super small nit: Can we remove the "spdy" in the middle of the library name? i.e. change to "spdy_core_alt_svc_wire_format_lib". Same for "spdy_core_spdy_header_block_lib", "spdy_core_spdy_headers_handler_interface", "spdy_core_spdy_protocol_lib" and "spdy_core_spdy_test_utils_lib".

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

name = "quiche_spdy_core_lib",
srcs = [

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: fold "srcs" and "hdrs" into a single line.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

"quiche/spdy/core/spdy_header_block.cc",
"quiche/spdy/core/spdy_test_utils.cc",
],
hdrs = [
"quiche/spdy/core/spdy_alt_svc_wire_format.h",
"quiche/spdy/core/spdy_bitmasks.h",
"quiche/spdy/core/spdy_header_block.h",
"quiche/spdy/core/spdy_headers_handler_interface.h",
"quiche/spdy/core/spdy_protocol.h",
"quiche/spdy/core/spdy_test_utils.h",
],
copts = quiche_copt,
repository = "@envoy",
visibility = ["//visibility:public"],
deps = [
":spdy_platform",
":spdy_platform_unsafe_arena_lib",
],
)

envoy_cc_library(
name = "quic_platform",
srcs = ["quiche/quic/platform/api/quic_mutex.cc"] + envoy_select_quiche(
Expand Down Expand Up @@ -365,6 +389,32 @@ envoy_cc_test_library(
deps = ["@envoy//test/extensions/quic_listeners/quiche/platform:quic_platform_epoll_impl_lib"],
)

envoy_cc_library(
name = "quiche_common_platform",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I'd prefer "quiche_platform", for similarity with (quic|spdy|http2)_platform.

Your call. (If you do change it to quiche_platform, please also change quiche_common_platform_test to quiche_platform_test, and quiche_common_platform_impl_lib to quiche_platform_impl_lib)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My concern is that if there are other directory added under quiche in the future, it will be hard to name those without specifying "common" here.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM!

hdrs = [
"quiche/common/platform/api/quiche_logging.h",
"quiche/common/platform/api/quiche_ptr_util.h",
"quiche/common/platform/api/quiche_unordered_containers.h",
],
repository = "@envoy",
deps = ["@envoy//source/extensions/quic_listeners/quiche/platform:quiche_common_platform_impl_lib"],
)

envoy_cc_test_library(
name = "quiche_common_platform_test",
hdrs = ["quiche/common/platform/api/quiche_test.h"],
repository = "@envoy",
deps = ["@envoy//test/extensions/quic_listeners/quiche/platform:quiche_common_platform_test_impl_lib"],
)

envoy_cc_library(
name = "quiche_common_lib",
hdrs = ["quiche/common/simple_linked_hash_map.h"],
repository = "@envoy",
visibility = ["//visibility:public"],
deps = [":quiche_common_platform"],
)

envoy_cc_test(
name = "epoll_server_test",
srcs = ["quiche/epoll_server/simple_epoll_server_test.cc"],
Expand All @@ -373,6 +423,17 @@ envoy_cc_test(
deps = [":epoll_server_lib"],
)

envoy_cc_test(
name = "quiche_common_test",
srcs = ["quiche/common/simple_linked_hash_map_test.cc"],
copts = quiche_copt,
repository = "@envoy",
deps = [
":quiche_common_lib",
":quiche_common_platform_test",
],
)

envoy_cc_test(
name = "http2_platform_api_test",
srcs = [
Expand All @@ -396,6 +457,17 @@ envoy_cc_test(
deps = [":spdy_platform"],
)

envoy_cc_test(
name = "quiche_spdy_core_test",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: For spdy test, please name it spdy_${something}_test.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

srcs = ["quiche/spdy/core/spdy_header_block_test.cc"],
copts = quiche_copt,
repository = "@envoy",
deps = [
":quiche_spdy_core_lib",
":spdy_platform",
],
)

envoy_cc_test(
name = "quic_platform_api_test",
srcs = envoy_select_quiche(
Expand Down
3 changes: 2 additions & 1 deletion bazel/external/quiche.genrule_cmd
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,11 @@ cat <<EOF >sed_commands
/^#include/ s!net/quic/platform/impl/quic_test_impl.h!test/extensions/quic_listeners/quiche/platform/quic_test_impl.h!
/^#include/ s!net/quic/platform/impl/quic_test_output_impl.h!test/extensions/quic_listeners/quiche/platform/quic_test_output_impl.h!
/^#include/ s!net/quic/platform/impl/quic_thread_impl.h!test/extensions/quic_listeners/quiche/platform/quic_thread_impl.h!
/^#include/ s!net/quiche/common/platform/impl/quiche_test_impl.h!test/extensions/quic_listeners/quiche/platform/quiche_test_impl.h!
/^#include/ s!net/spdy/platform/impl/spdy_test_helpers_impl.h!test/extensions/quic_listeners/quiche/platform/spdy_test_helpers_impl.h!

# Rewrite include directives for platform impl files.
/^#include/ s!net/(http2|spdy|quic)/platform/impl/!extensions/quic_listeners/quiche/platform/!
/^#include/ s!net/(http2|spdy|quic|quiche/common)/platform/impl/!extensions/quic_listeners/quiche/platform/!

# Rewrite include directives for epoll_server platform impl files.
/^#include/ s!net/tools/epoll_server/platform/impl!test/extensions/quic_listeners/quiche/platform/!
Expand Down
6 changes: 3 additions & 3 deletions bazel/repository_locations.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -240,8 +240,8 @@ REPOSITORY_LOCATIONS = dict(
urls = ["https://github.com/google/subpar/archive/1.3.0.tar.gz"],
),
com_googlesource_quiche = dict(
# Static snapshot of https://quiche.googlesource.com/quiche/+archive/43a1c0f10f2855c3cd142f500e8d19ac6d6f5a8c.tar.gz
sha256 = "7ee437b5b0f64290760cef43b93790122c751f24508e93393484ddb80c1f8bfe",
urls = ["https://storage.googleapis.com/quiche-envoy-integration/43a1c0f10f2855c3cd142f500e8d19ac6d6f5a8c.tar.gz"],
# Static snapshot of https://quiche.googlesource.com/quiche/+archive/7bf7c3c358eb954e463bde14ea27444f4bd8ea05.tar.gz
sha256 = "36fe180d532a9ccb18cd32328af5231636c7408104523f9ed5eebbad75f1e039",
urls = ["https://storage.googleapis.com/quiche-envoy-integration/7bf7c3c358eb954e463bde14ea27444f4bd8ea05.tar.gz"],
),
)
13 changes: 13 additions & 0 deletions source/extensions/quic_listeners/quiche/platform/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,18 @@ envoy_cc_library(
]),
)

envoy_cc_library(
name = "quiche_common_platform_impl_lib",
hdrs = [
"quiche_logging_impl.h",
"quiche_ptr_util_impl.h",
"quiche_unordered_containers_impl.h",
],
external_deps = ["abseil_node_hash_map"],
visibility = ["//visibility:public"],
deps = [":quic_platform_base_impl_lib"],

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add :quic_platform_logging_impl_lib to fix clang-tidy, bazel doesn't build header files only target explicitly but clang-tidy checks whether you added dependencies correctly.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Woops, just realized another problem that quic_logging_impl.h is duplicated in both quic_platform_logging_impl_lib and quic_platform_base_impl_lib.

Fixed. Thanks for looking into this problem!

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still not fixed. This is really weird. Looking at the errors a few lines above:
Error while processing /source/source/extensions/quic_listeners/quiche/platform/quic_containers_impl.h.
10294 warnings and 1 error generated.
Error while processing /source/test/extensions/quic_listeners/quiche/platform/quiche_test_impl.h.
16998 warnings and 1 error generated.
Error while processing /source/source/extensions/quic_listeners/quiche/platform/quiche_logging_impl.h.
23702 warnings and 1 error generated.
Error while processing /source/source/extensions/quic_listeners/quiche/platform/quiche_logging_impl.cc.
31722 warnings and 1 error generated.
Error while processing /source/source/extensions/quic_listeners/quiche/platform/spdy_containers_impl.h.
37223 warnings and 1 error generated.
Error while processing /source/source/extensions/quic_listeners/quiche/platform/spdy_string_utils_impl.h.
39075 warnings and 1 error generated.
Error while processing /source/source/extensions/quic_listeners/quiche/platform/quiche_ptr_util_impl.h.
43844 warnings and 1 error generated.
Error while processing /source/source/extensions/quic_listeners/quiche/platform/quiche_unordered_containers_impl.h.
warning: do not reference any 'internal' namespaces; those implementation details are reserved to Abseil [abseil-no-internal-dependencies]
bazel-out/k8-fastbuild/genfiles/external/com_googlesource_quiche/quiche/common/platform/api/quiche_logging.h:4:10: error: 'extensions/quic_listeners/quiche/platform/quiche_logging_impl.h' file not found [clang-diagnostic-error]
#include "extensions/quic_listeners/quiche/platform/quiche_logging_impl.h"

The file path is /source/source/extensions/quic_listeners/quiche/platform/... or /source/test/extensions/quic_listeners/quiche/platform/.... There is a /source added to the actual file path.

Why would clang_tidy CI do that?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI image checkout repository to /source so the path is correct.

)

envoy_cc_library(
name = "quic_platform_sleep_impl_lib",
hdrs = ["quic_sleep_impl.h"],
Expand Down Expand Up @@ -215,6 +227,7 @@ envoy_cc_library(
":quic_platform_logging_impl_lib",
":string_utils_lib",
"//source/common/common:assert_lib",
"@com_googlesource_quiche//:quiche_common_lib",
]),
)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
#include "absl/container/inlined_vector.h"
#include "absl/container/node_hash_map.h"
#include "absl/container/node_hash_set.h"
#include "quiche/common/simple_linked_hash_map.h"

// NOLINT(namespace-envoy)

Expand All @@ -26,14 +27,11 @@ using QuicUnorderedMapImpl = absl::node_hash_map<Key, Value, Hash>;

template <typename Key, typename Hash> using QuicUnorderedSetImpl = absl::node_hash_set<Key, Hash>;

// TODO: implement
template <typename Key, typename Value, typename Hash> class QuicLinkedHashMapImpl {};

// TODO: implement
template <typename Key, typename Value, int Size> class QuicSmallMapImpl {};
template <typename Key, typename Value, typename Hash>
using QuicLinkedHashMapImpl = quiche::SimpleLinkedHashMap<Key, Value, Hash>;

// TODO: implement
template <typename T> class QuicIntervalSetImpl;
template <typename Key, typename Value, int Size>
using QuicSmallMapImpl = std::unordered_map<Key, Value>;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about absl::flat_hash_map, which is typically faster?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


template <typename T> using QuicQueueImpl = std::queue<T>;

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
#pragma once

// NOLINT(namespace-envoy)

// This file is part of the QUICHE platform implementation, and is not to be
// consumed or referenced directly by other Envoy code. It serves purely as a
// porting layer for QUICHE.

#include "extensions/quic_listeners/quiche/platform/quic_logging_impl.h"
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
#pragma once

// NOLINT(namespace-envoy)

// This file is part of the QUICHE platform implementation, and is not to be
// consumed or referenced directly by other Envoy code. It serves purely as a
// porting layer for QUICHE.

#include <memory>

namespace quiche {

template <typename T, typename... Args> std::unique_ptr<T> QuicheMakeUniqueImpl(Args&&... args) {
return std::make_unique<T>(std::forward<Args>(args)...);
}

} // namespace quiche
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#pragma once

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is already in test/extensions/..., which is correct since it's test only.
Can you remove this file?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, yes, this is not supposed to be here.


// NOLINT(namespace-envoy)

// This file is part of the QUICHE platform implementation, and is not to be
// consumed or referenced directly by other Envoy code. It serves purely as a
// porting layer for QUICHE.

#include "gmock/gmock.h"
#include "gtest/gtest.h"
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
#pragma once

// NOLINT(namespace-envoy)

// This file is part of the QUICHE platform implementation, and is not to be
// consumed or referenced directly by other Envoy code. It serves purely as a
// porting layer for QUICHE.

#include "absl/container/node_hash_map.h"

namespace quiche {

// The default hasher used by hash tables.
template <typename Key> using QuicheDefaultHasherImpl = absl::Hash<Key>;

// Similar to std::unordered_map, but with better performance and memory usage.
template <typename Key, typename Value, typename Hash>
using QuicheUnorderedMapImpl = absl::node_hash_map<Key, Value, Hash>;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a particular reason to use absl::node_hash_map instead of absl::flat_hash_map?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

flat_hash_map doesn't support key stability. Since this impl will probably replace QuicUnorderedMap which is used in several places as replacement of std::unordered_map. It will take some extra effort to investigate if those places requires key stability or not to switch to absl::flat_hash_map. And as we use node_hash_map in upstream, it's better to use the same class here to make sure future upstream change can be smoothly imported.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM. In the long term we should probably rename it to QuicheNodeHashMap and add a QuicheFlatHashMap, or directly use the absl ones if that's allowed.


} // namespace quiche
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
#pragma once

// NOLINT(namespace-envoy)

// This file is part of the QUICHE platform implementation, and is not to be
// consumed or referenced directly by other Envoy code. It serves purely as a
// porting layer for QUICHE.

#include "absl/container/flat_hash_map.h"
#include "absl/container/flat_hash_set.h"
#include "absl/container/inlined_vector.h"
Expand All @@ -14,12 +20,7 @@
// TODO(mpw): fix includes to use full paths.
#include "spdy_string_impl.h"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This TODO seems easy to fix, can you do it while we are here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

#include "spdy_string_piece_impl.h"

// NOLINT(namespace-envoy)

// This file is part of the QUICHE platform implementation, and is not to be
// consumed or referenced directly by other Envoy code. It serves purely as a
// porting layer for QUICHE.
#include "quiche/common/simple_linked_hash_map.h"

namespace spdy {

Expand All @@ -31,8 +32,8 @@ using SpdyHashMapImpl = absl::flat_hash_map<KeyType, ValueType, Hash>;
template <typename ElementType, typename Hasher, typename Eq>
using SpdyHashSetImpl = absl::flat_hash_set<ElementType, Hasher, Eq>;

// TODO: implement
template <typename Key, typename Value, typename Hash> class SpdyLinkedHashMapImpl {};
template <typename Key, typename Value, typename Hash>
using SpdyLinkedHashMapImpl = quiche::SimpleLinkedHashMap<Key, Value, Hash>;

template <typename T, size_t N, typename A = std::allocator<T>>
using SpdyInlinedVectorImpl = absl::InlinedVector<T, N, A>;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,3 @@

#define SPDY_MUST_USE_RESULT_IMPL ABSL_MUST_USE_RESULT
#define SPDY_UNUSED_IMPL ABSL_ATTRIBUTE_UNUSED

// TODO: implement
#define SPDY_DVLOG_IF_IMPL 0
5 changes: 5 additions & 0 deletions test/extensions/quic_listeners/quiche/platform/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,11 @@ envoy_cc_test_library(
],
)

envoy_cc_test_library(
name = "quiche_common_platform_test_impl_lib",
hdrs = ["quiche_test_impl.h"],
)

envoy_cc_test_library(
name = "spdy_platform_test_helpers_impl_lib",
hdrs = ["spdy_test_helpers_impl.h"],
Expand Down
10 changes: 10 additions & 0 deletions test/extensions/quic_listeners/quiche/platform/quiche_test_impl.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#pragma once

// NOLINT(namespace-envoy)

// This file is part of the QUICHE platform implementation, and is not to be
// consumed or referenced directly by other Envoy code. It serves purely as a
// porting layer for QUICHE.

#include "gmock/gmock.h"
#include "gtest/gtest.h"