Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
818ec8d
Add ability to filter ConfigDump.
paul-r-gall Jun 2, 2021
c9712c3
Merge remote-tracking branch 'upstream/main' into FilterConfigDump
paul-r-gall Jun 8, 2021
56fd0f9
Revert "Add ability to filter ConfigDump."
paul-r-gall Jun 8, 2021
34f1993
Add ConfigDumpFilter setup with default filter extension.
paul-r-gall Jun 9, 2021
74606de
Merge remote-tracking branch 'upstream/main' into FilterConfigDump
paul-r-gall Jun 9, 2021
56627a4
Fix PURE virtual fxns
paul-r-gall Jun 14, 2021
d2105e5
simple name matching only
paul-r-gall Jun 17, 2021
65ad9a3
Merge remote-tracking branch 'upstream/main' into FilterConfigDump
paul-r-gall Jun 17, 2021
57ec7e4
move universal matcher to common lib
paul-r-gall Jun 17, 2021
30957dd
catch RE2 failure exception
paul-r-gall Jun 18, 2021
f51e4de
fix build errors
paul-r-gall Jun 18, 2021
4319328
add testing
paul-r-gall Jun 18, 2021
10252d1
Add documentation
paul-r-gall Jun 18, 2021
52fe34f
fix whitespace issues
paul-r-gall Jun 22, 2021
5741f4d
remove offending log
paul-r-gall Jun 22, 2021
7a8b47f
fix coverage
paul-r-gall Jun 22, 2021
f12d0b8
additional coverage fixes
paul-r-gall Jun 23, 2021
2bb4801
Merge remote-tracking branch 'upstream/main' into FilterConfigDump
paul-r-gall Jun 23, 2021
2f92553
fix formatting
paul-r-gall Jun 23, 2021
64a720b
Merge branch 'main' into FilterConfigDump
paul-r-gall Jun 28, 2021
2b7b069
docs and exception fix
paul-r-gall Jun 28, 2021
3c46827
Merge remote-tracking branch 'upstream/main' into FilterConfigDump
paul-r-gall Jun 29, 2021
1e2c4fc
Merge remote-tracking branch 'upstream/main' into FilterConfigDump
paul-r-gall Jul 1, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions docs/root/operations/admin.rst
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,25 @@ modify different aspects of the server:
field, use the mask query parameter documented above. If you want only a subset of fields from the repeated
resource, use both as documented below.

.. _operations_admin_interface_config_dump_by_name_regex:

.. http:get:: /config_dump?name_regex={}

Dump only the currently loaded configurations whose names match the specified regex. Can be used with
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doc nit: Can you ref link some example name fields for the user so it's more clear what name we are referring to. Separately, this means that effectively we need a name field in all xDS protos to make this consistent, right? (Just trying to make sure I understand.)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So currently all xDS protos have a name-like field; most of those name-like fields are actually called name, except for endpoints, which have cluster_name. Only the protos whose name-like field matches the given regexp will be added to the dump. E.G. putting in name_regex=prefix.* will return routes such that route.name() matches, and endpoints such that endpoint.cluster_name() matches.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup that makes sense. I would just ref link some fields or just list them in a bullet list to be more clear to the user. Right now I think it might not be clear to the user what this will do.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, for some reason clang-format is complaining about this file and tools/code_format/check_format.py fix isn't making any changes -- do you know what that's about?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, not sure how it would be complaining about the RST file. What is the error? cc @phlax

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wound up forcing with --no-verify, but the error was

~/git/envoy$ git push origin FilterConfigDump
Running pre-push check; to skip this step use 'push --no-verify'
  Checking format for docs/root/operations/admin.rst - ERROR: From ./docs/root/operations/admin.rst
ERROR: Unable to find Envoy namespace or NOLINT(namespace-envoy) for file: ./docs/root/operations/admin.rst
ERROR: clang-format check failed for file: ./docs/root/operations/admin.rst
ERROR: check format failed. run 'tools/code_format/check_format.py fix'
error: failed to push some refs to 'github.com:paul-r-gall/envoy.git

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm this sounds like some type of check format regression. cc @phlax to help out.

both `resource` and `mask` query parameters.

For example, ``/config_dump?name_regex=.*substring.*`` would return all resource types
whose name field matches the given regex.

Per resource, the matched name field is:

- :ref:`envoy.config.listener.v3.Listener.name <envoy_v3_api_field_config.listener.v3.Listener.name>`
- :ref:`envoy.config.route.v3.RouteConfiguration.name <envoy_v3_api_field_config.route.v3.RouteConfiguration.name>`
- :ref:`envoy.config.route.v3.ScopedRouteConfiguration.name <envoy_v3_api_field_config.route.v3.ScopedRouteConfiguration.name>`
- :ref:`envoy.config.cluster.v3.Cluster.name <envoy_v3_api_field_config.cluster.v3.Cluster.name>`
- :ref:`envoy.extensions.transport_sockets.tls.v3.Secret <envoy_v3_api_field_extensions.transport_sockets.tls.v3.Secret.name>`
- :ref:`envoy.config.endpoint.v3.ClusterLoadAssignment <envoy_v3_api_field_config.endpoint.v3.ClusterLoadAssignment.cluster_name>`

.. _operations_admin_interface_config_dump_by_resource_and_mask:

.. http:get:: /config_dump?resource={}&mask={}
Expand Down
1 change: 1 addition & 0 deletions envoy/server/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ envoy_cc_library(
name = "config_tracker_interface",
hdrs = ["config_tracker.h"],
deps = [
"//envoy/common:matchers_interface",
"//source/common/common:non_copyable",
"//source/common/protobuf",
],
Expand Down
5 changes: 4 additions & 1 deletion envoy/server/config_tracker.h
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
#include <map>
#include <memory>

#include "envoy/common/matchers.h"
#include "envoy/common/pure.h"

#include "source/common/common/non_copyable.h"
Expand All @@ -21,7 +22,9 @@ namespace Server {
*/
class ConfigTracker {
public:
using Cb = std::function<ProtobufTypes::MessagePtr()>;
// The passed StringMatcher will be matched against the `name` field of whatever
// proto is returned.
using Cb = std::function<ProtobufTypes::MessagePtr(const Matchers::StringMatcher&)>;
using CbsMap = std::map<std::string, Cb>;

/**
Expand Down
5 changes: 5 additions & 0 deletions source/common/common/matchers.h
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,11 @@ class DoubleMatcher : public ValueMatcher {
const envoy::type::matcher::v3::DoubleMatcher matcher_;
};

class UniversalStringMatcher : public StringMatcher {
public:
bool match(absl::string_view) const override { return true; }
};

class StringMatcherImpl : public ValueMatcher, public StringMatcher {
public:
explicit StringMatcherImpl(const envoy::type::matcher::v3::StringMatcher& matcher);
Expand Down
5 changes: 3 additions & 2 deletions source/common/config/config_provider_impl.cc
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,9 @@ bool ConfigSubscriptionInstance::checkAndApplyConfigUpdate(const Protobuf::Messa

ConfigProviderManagerImplBase::ConfigProviderManagerImplBase(Server::Admin& admin,
const std::string& config_name) {
config_tracker_entry_ =
admin.getConfigTracker().add(config_name, [this] { return dumpConfigs(); });
config_tracker_entry_ = admin.getConfigTracker().add(
config_name,
[this](const Matchers::StringMatcher& name_matcher) { return dumpConfigs(name_matcher); });
// ConfigTracker keys must be unique. We are asserting that no one has stolen the key
// from us, since the returned entry will be nullptr if the key already exists.
RELEASE_ASSERT(config_tracker_entry_, "");
Expand Down
3 changes: 2 additions & 1 deletion source/common/config/config_provider_impl.h
Original file line number Diff line number Diff line change
Expand Up @@ -383,7 +383,8 @@ class ConfigProviderManagerImplBase : public ConfigProviderManager, public Singl
* @return ProtobufTypes::MessagePtr the config dump proto corresponding to the associated
* config providers.
*/
virtual ProtobufTypes::MessagePtr dumpConfigs() const PURE;
virtual ProtobufTypes::MessagePtr
dumpConfigs(const Matchers::StringMatcher& name_matcher) const PURE;

protected:
// Ordered set for deterministic config dump output.
Expand Down
13 changes: 11 additions & 2 deletions source/common/router/rds_impl.cc
Original file line number Diff line number Diff line change
Expand Up @@ -328,7 +328,9 @@ void RdsRouteConfigProviderImpl::requestVirtualHostsUpdate(

RouteConfigProviderManagerImpl::RouteConfigProviderManagerImpl(Server::Admin& admin) {
config_tracker_entry_ =
admin.getConfigTracker().add("routes", [this] { return dumpRouteConfigs(); });
admin.getConfigTracker().add("routes", [this](const Matchers::StringMatcher& matcher) {
return dumpRouteConfigs(matcher);
});
// ConfigTracker keys must be unique. We are asserting that no one has stolen the "routes" key
// from us, since the returned entry will be nullptr if the key already exists.
RELEASE_ASSERT(config_tracker_entry_, "");
Expand Down Expand Up @@ -379,7 +381,8 @@ RouteConfigProviderPtr RouteConfigProviderManagerImpl::createStaticRouteConfigPr
}

std::unique_ptr<envoy::admin::v3::RoutesConfigDump>
RouteConfigProviderManagerImpl::dumpRouteConfigs() const {
RouteConfigProviderManagerImpl::dumpRouteConfigs(
const Matchers::StringMatcher& name_matcher) const {
auto config_dump = std::make_unique<envoy::admin::v3::RoutesConfigDump>();

for (const auto& element : dynamic_route_config_providers_) {
Expand All @@ -391,6 +394,9 @@ RouteConfigProviderManagerImpl::dumpRouteConfigs() const {
ASSERT(subscription->route_config_provider_opt_.has_value());

if (subscription->routeConfigUpdate()->configInfo()) {
if (!name_matcher.match(subscription->routeConfigUpdate()->protobufConfiguration().name())) {
continue;
}
auto* dynamic_config = config_dump->mutable_dynamic_route_configs()->Add();
dynamic_config->set_version_info(subscription->routeConfigUpdate()->configVersion());
dynamic_config->mutable_route_config()->PackFrom(
Expand All @@ -402,6 +408,9 @@ RouteConfigProviderManagerImpl::dumpRouteConfigs() const {

for (const auto& provider : static_route_config_providers_) {
ASSERT(provider->configInfo());
if (!name_matcher.match(provider->configInfo().value().config_.name())) {
continue;
}
auto* static_config = config_dump->mutable_static_route_configs()->Add();
static_config->mutable_route_config()->PackFrom(
API_RECOVER_ORIGINAL(provider->configInfo().value().config_));
Expand Down
3 changes: 2 additions & 1 deletion source/common/router/rds_impl.h
Original file line number Diff line number Diff line change
Expand Up @@ -242,7 +242,8 @@ class RouteConfigProviderManagerImpl : public RouteConfigProviderManager,
public:
RouteConfigProviderManagerImpl(Server::Admin& admin);

std::unique_ptr<envoy::admin::v3::RoutesConfigDump> dumpRouteConfigs() const;
std::unique_ptr<envoy::admin::v3::RoutesConfigDump>
dumpRouteConfigs(const Matchers::StringMatcher& name_matcher) const;

// RouteConfigProviderManager
RouteConfigProviderSharedPtr createRdsRouteConfigProvider(
Expand Down
9 changes: 8 additions & 1 deletion source/common/router/scoped_rds.cc
Original file line number Diff line number Diff line change
Expand Up @@ -521,7 +521,8 @@ ScopedRdsConfigProvider::ScopedRdsConfigProvider(
ScopedRdsConfigSubscriptionSharedPtr&& subscription)
: MutableConfigProviderCommonBase(std::move(subscription), ConfigProvider::ApiType::Delta) {}

ProtobufTypes::MessagePtr ScopedRoutesConfigProviderManager::dumpConfigs() const {
ProtobufTypes::MessagePtr
ScopedRoutesConfigProviderManager::dumpConfigs(const Matchers::StringMatcher& name_matcher) const {
auto config_dump = std::make_unique<envoy::admin::v3::ScopedRoutesConfigDump>();
for (const auto& element : configSubscriptions()) {
auto subscription = element.second.lock();
Expand All @@ -535,6 +536,9 @@ ProtobufTypes::MessagePtr ScopedRoutesConfigProviderManager::dumpConfigs() const
dynamic_config->set_name(typed_subscription->name());
const ScopedRouteMap& scoped_route_map = typed_subscription->scopedRouteMap();
for (const auto& it : scoped_route_map) {
if (!name_matcher.match(it.second->configProto().name())) {
continue;
}
dynamic_config->mutable_scoped_route_configs()->Add()->PackFrom(
API_RECOVER_ORIGINAL(it.second->configProto()));
}
Expand All @@ -550,6 +554,9 @@ ProtobufTypes::MessagePtr ScopedRoutesConfigProviderManager::dumpConfigs() const
auto* inline_config = config_dump->mutable_inline_scoped_route_configs()->Add();
inline_config->set_name(static_cast<InlineScopedRoutesConfigProvider*>(provider)->name());
for (const auto& config_proto : protos_info.value().config_protos_) {
if (!name_matcher.match(config_proto->name())) {
continue;
}
inline_config->mutable_scoped_route_configs()->Add()->PackFrom(
API_RECOVER_ORIGINAL(*config_proto));
}
Expand Down
2 changes: 1 addition & 1 deletion source/common/router/scoped_rds.h
Original file line number Diff line number Diff line change
Expand Up @@ -275,7 +275,7 @@ class ScopedRoutesConfigProviderManager : public Envoy::Config::ConfigProviderMa
~ScopedRoutesConfigProviderManager() override = default;

// Envoy::Config::ConfigProviderManagerImplBase
ProtobufTypes::MessagePtr dumpConfigs() const override;
ProtobufTypes::MessagePtr dumpConfigs(const Matchers::StringMatcher& name_matcher) const override;

// Envoy::Config::ConfigProviderManager
Envoy::Config::ConfigProviderPtr
Expand Down
Loading