Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
c4101e1
initial patch
jmarantz Jan 14, 2022
29cf776
get tests working
jmarantz Jan 14, 2022
94c97c0
Add parameters for other admin endpoints. Still need runtime_modify.
jmarantz Jan 14, 2022
aa00670
add coverage for the HTML generator
jmarantz Jan 15, 2022
34c7607
coverage
jmarantz Jan 15, 2022
4e821b4
coverage and doc
jmarantz Jan 15, 2022
8cbcf36
cover html rendering
jmarantz Jan 16, 2022
ff2aaf9
revert pointless name-change and rewrite comment.
jmarantz Jan 16, 2022
1e4d606
remove unused styles
jmarantz Jan 16, 2022
256b6c9
test cleanup
jmarantz Jan 16, 2022
49212f6
Merge branch 'main' into admin-params
jmarantz Jan 17, 2022
ba5a1dc
Remove the intermediate vectors for stats by using forEach for insert…
jmarantz Jan 17, 2022
c78b6a4
Merge branch 'main' into admin-params
jmarantz Jan 19, 2022
33b2f22
Go back to sorting by strings as it's somewhat faster than sorting by…
jmarantz Jan 19, 2022
efe142e
add in ifdef'd version where we do the sort in StaName space.
jmarantz Jan 20, 2022
78091d1
Merge branch 'main' into admin-params
jmarantz Feb 26, 2022
119270c
get build working
jmarantz Feb 26, 2022
68a89d2
fix respones code expectation
jmarantz Feb 26, 2022
70ce924
cleanup
jmarantz Feb 26, 2022
71c1647
Merge branch 'main' into admin-params
jmarantz Mar 2, 2022
d04a05c
get compile working again
jmarantz Mar 2, 2022
85d33f7
Merge branch 'main' into admin-params
jmarantz Apr 6, 2022
d482473
compiles and links tests, but tests fail
jmarantz Apr 7, 2022
d42f01a
All tests working and formatted.
jmarantz Apr 8, 2022
2c4422a
reduce allow-list for std::regex and cleanup commented-out code.
jmarantz Apr 8, 2022
594b618
Merge branch 'main' into admin-params
jmarantz Apr 8, 2022
102d95c
fix signature
jmarantz Apr 8, 2022
a5706e1
fix naming inconsistency
jmarantz Apr 8, 2022
a4de093
add missing function
jmarantz Apr 8, 2022
246af3a
Merge branch 'main' into admin-params
jmarantz Apr 10, 2022
987e764
cleanup
jmarantz Apr 10, 2022
5f5ae3c
use the correct UrlHandler block for /stats.
jmarantz Apr 10, 2022
c9041e6
add histogram buckets into UI params.
jmarantz Apr 11, 2022
afbcf1a
finalize before returning when we are not covering all types.
jmarantz Apr 11, 2022
3fc02f0
format
jmarantz Apr 11, 2022
fc5ac30
Merge branch 'main' into admin-params
jmarantz Apr 11, 2022
7bb3a00
Merge branch 'main' into admin-params
jmarantz Apr 20, 2022
6dcac05
format
jmarantz Apr 20, 2022
81f2763
API change for StatsParams
jmarantz Apr 20, 2022
329281b
Merge branch 'main' into admin-params
jmarantz Apr 22, 2022
c537b80
Merge branch 'main' into admin-params
jmarantz May 11, 2022
25c871e
try to get it to compile
jmarantz May 12, 2022
685e307
get tests working
jmarantz May 12, 2022
182c26d
Merge branch 'main' into admin-params
jmarantz Jun 24, 2022
fecece3
fix tests
jmarantz Jun 24, 2022
f187c65
get tests working in html-disabled mode.
jmarantz Jun 24, 2022
d16f773
cleanup
jmarantz Jun 25, 2022
2ec28de
emit the parameter info in the /help response.
jmarantz Jul 1, 2022
e23c884
format
jmarantz Jul 1, 2022
4f59d62
unbreak
jmarantz Jul 1, 2022
c2350a2
Merge branch 'main' into admin-params
jmarantz Jul 1, 2022
c9a0467
get tests woring with and without html.
jmarantz Jul 2, 2022
e8abc92
format
jmarantz Jul 2, 2022
bfb734a
cover some missing cases
jmarantz Jul 2, 2022
752ae63
add pass-thru overrides for Html rendering to fix gcc builds.
jmarantz Jul 2, 2022
e8d16fd
cover html histogram rendering.
jmarantz Jul 2, 2022
abe2049
add coverage
jmarantz Jul 2, 2022
ab028cf
format
jmarantz Jul 2, 2022
d9d03c8
Merge branch 'main' into admin-params
jmarantz Jul 4, 2022
b28dd55
add comments
jmarantz Jul 11, 2022
1b95d3b
properly sanitize the filter and quote it.
jmarantz Jul 12, 2022
048d239
split out the html rendered into its own files.
jmarantz Jul 12, 2022
93fe3f3
format
jmarantz Jul 12, 2022
5c0fd6c
Merge branch 'main' into admin-params
jmarantz Jul 12, 2022
85cde04
refactor to remove extra html implementation files
jmarantz Jul 12, 2022
b626917
tighten up the logic.
jmarantz Jul 13, 2022
ddda5d7
Merge branch 'main' into admin-params
jmarantz Jul 15, 2022
2a62570
Merge branch 'main' into admin-params
jmarantz Jul 17, 2022
03be332
add accessibility annotations.
jmarantz Jul 17, 2022
ce6e904
fix html validation errors.
jmarantz Jul 17, 2022
26f200d
fix unit tests and prepopulated values.
jmarantz Jul 17, 2022
50dbe5a
Render endpoints using GET that don't have params as simple a tags.
jmarantz Jul 18, 2022
0859243
review comments
jmarantz Jul 22, 2022
603f0f8
Merge branch 'main' into admin-params
jmarantz Jul 22, 2022
1cc67ca
Merge branch 'main' into admin-params
jmarantz Jul 25, 2022
ee21f74
add missing params (round 1)
jmarantz Jul 25, 2022
379b316
add remaining missing admin query params.
jmarantz Jul 26, 2022
a7b3565
back out post support for now
jmarantz Jul 26, 2022
c456954
back out more post params
jmarantz Jul 26, 2022
8c3af8b
format
jmarantz Jul 26, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/root/operations/admin.rst
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,9 @@ modify different aspects of the server:

.. http:get:: /

Render an HTML home page with a table of links to all available options.
Render an HTML home page with a table of links to all available options. This can be
disabled by compiling Envoy with `--define=admin_html=disabled` in which case an error
message is printed. Disabling the HTML mode reduces the Envoy binary size.

.. http:get:: /help

Expand Down
44 changes: 37 additions & 7 deletions envoy/server/admin.h
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,20 @@ class Admin {
public:
virtual ~Admin() = default;

// Describes a parameter for an endpoint. This structure is used when
// admin-html has not been disabled to populate an HTML form to enable a
// visitor to the admin console to intuitively specify query-parameters for
// each endpoint. The parameter descriptions also appear in the /help
// endpoint, independent of how Envoy is compiled.
struct ParamDescriptor {
enum class Type { Boolean, String, Enum };
const Type type_;
const std::string id_; // HTML form ID and query-param name (JS var name rules).
const std::string help_; // Help text rendered into UI.
std::vector<absl::string_view> enum_choices_{};
};
using ParamDescriptorVec = std::vector<ParamDescriptor>;

// Represents a request for admin endpoints, enabling streamed responses.
class Request {
public:
Expand Down Expand Up @@ -127,6 +141,23 @@ class Admin {
};
using RequestPtr = std::unique_ptr<Request>;

/**
* Lambda to generate a Request.
*/
using GenRequestFn = std::function<RequestPtr(absl::string_view path, AdminStream&)>;

/**
* Individual admin handler including prefix, help text, and callback.
*/
struct UrlHandler {
const std::string prefix_;
const std::string help_text_;
const GenRequestFn handler_;
const bool removable_;
const bool mutates_server_state_;
const ParamDescriptorVec params_{};
};

/**
* Callback for admin URL handlers.
* @param path_and_query supplies the path and query of the request URL.
Expand All @@ -141,11 +172,6 @@ class Admin {
absl::string_view path_and_query, Http::ResponseHeaderMap& response_headers,
Buffer::Instance& response, AdminStream& admin_stream)>;

/**
* Lambda to generate a Request.
*/
using GenRequestFn = std::function<RequestPtr(absl::string_view path, AdminStream&)>;

/**
* Add a legacy admin handler where the entire response is written in
* one chunk.
Expand All @@ -155,10 +181,12 @@ class Admin {
* @param callback supplies the callback to invoke when the prefix matches.
* @param removable if true allows the handler to be removed via removeHandler.
* @param mutates_server_state indicates whether callback will mutate server state.
* @param params command parameter descriptors.
* @return bool true if the handler was added, false if it was not added.
*/
virtual bool addHandler(const std::string& prefix, const std::string& help_text,
HandlerCb callback, bool removable, bool mutates_server_state) PURE;
HandlerCb callback, bool removable, bool mutates_server_state,
const ParamDescriptorVec& params = {}) PURE;

/**
* Adds a an chunked admin handler.
Expand All @@ -168,11 +196,13 @@ class Admin {
* @param gen_request supplies the callback to generate a Request.
* @param removable if true allows the handler to be removed via removeHandler.
* @param mutates_server_state indicates whether callback will mutate server state.
* @param params command parameter descriptors.
* @return bool true if the handler was added, false if it was not added.
*/
virtual bool addStreamingHandler(const std::string& prefix, const std::string& help_text,
GenRequestFn gen_request, bool removable,
bool mutates_server_state) PURE;
bool mutates_server_state,
const ParamDescriptorVec& params = {}) PURE;

/**
* Remove an admin handler if it is removable.
Expand Down
32 changes: 27 additions & 5 deletions source/server/admin/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,19 @@ envoy_cc_library(
],
)

genrule(
name = "generate_admin_html",
srcs = [
"admin_head_start.html",
"admin.css",
],
outs = ["admin_html_gen.h"],
cmd = "./$(location :generate_admin_html.sh) \
$(location admin_head_start.html) $(location admin.css) > $@",
tools = [":generate_admin_html.sh"],
visibility = ["//visibility:private"],
)

envoy_cc_library(
name = "handler_ctx_lib",
hdrs = ["handler_ctx.h"],
Expand All @@ -103,6 +116,8 @@ envoy_cc_library(
deps = [
":utils_lib",
"//envoy/buffer:buffer_interface",
"//envoy/http:codes_interface",
"//envoy/server:admin_interface",
"//source/common/stats:histogram_lib",
],
)
Expand All @@ -114,10 +129,10 @@ envoy_cc_library(
deps = [
":handler_ctx_lib",
":prometheus_stats_lib",
":stats_params_lib",
":stats_render_lib",
":utils_lib",
"//envoy/http:codes_interface",
"//envoy/server:admin_interface",
"//source/common/http:codes_lib",
"//source/common/http:header_map_lib",
"//source/common/stats:histogram_lib",
Expand All @@ -127,16 +142,23 @@ envoy_cc_library(

envoy_cc_library(
name = "stats_render_lib",
srcs = ["stats_render.cc"],
hdrs = ["stats_render.h"],
srcs = ["stats_render.cc"] + envoy_select_admin_html([
":generate_admin_html",
"stats_html_render.cc",
]),
hdrs = ["stats_render.h"] + envoy_select_admin_html([
"stats_html_render.h",
]),
deps = [
":stats_params_lib",
":utils_lib",
"//envoy/server:admin_interface",
"//source/common/buffer:buffer_lib",
"//source/common/html:utility_lib",
"//source/common/json:json_sanitizer_lib",
"//source/common/stats:histogram_lib",
],
] + envoy_select_admin_html([
"//source/common/html:utility_lib",
]),
)

envoy_cc_library(
Expand Down
84 changes: 70 additions & 14 deletions source/server/admin/admin.cc
Original file line number Diff line number Diff line change
Expand Up @@ -99,10 +99,26 @@ AdminImpl::AdminImpl(const std::string& profile_path, Server::Instance& server,
MAKE_ADMIN_HANDLER(server_info_handler_.handlerCerts), false, false),
makeHandler("/clusters", "upstream cluster status",
MAKE_ADMIN_HANDLER(clusters_handler_.handlerClusters), false, false),
makeHandler("/config_dump", "dump current Envoy configs (experimental)",
MAKE_ADMIN_HANDLER(config_dump_handler_.handlerConfigDump), false, false),
makeHandler(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to have an assert in the constructor body that there are no duplicate prefixes in handlers_ ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

"/config_dump", "dump current Envoy configs (experimental)",
MAKE_ADMIN_HANDLER(config_dump_handler_.handlerConfigDump), false, false,
{{Admin::ParamDescriptor::Type::String, "resource", "The resource to dump"},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is missing certain parameters listed on the doc page:
https://www.envoyproxy.io/docs/envoy/latest/operations/admin#get--config_dump?include_eds

Copy link
Contributor Author

@jmarantz jmarantz Jul 26, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realized that and was thinking of filling in all the new parameters that were added since I first wrote this code as a follow up. However I'm glad you asked about this because this brings a new complexity. /logging is a mutating operation, and thus must be issued as a POST. POST params are transmitted from the browser to the server as POST params in the request body rather than query-params, and there's not currently a path to get get them into the admin handlers. I'm going to have to iterate on this a bit.

I'll get that sorted before asking for the next round of comments.

/wait

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I decided to back out my addition of POST params in this PR and will follow up. I got this working but it was too hacky. I need to make some infrastructural changes to get POST params to work well, and also to handling the "/logging" semantics for an optional enum param.

{Admin::ParamDescriptor::Type::String, "mask",
"The mask to apply. When both resource and mask are specified, "
"the mask is applied to every element in the desired repeated field so that only a "
"subset of fields are returned. The mask is parsed as a ProtobufWkt::FieldMask"},
{Admin::ParamDescriptor::Type::String, "name_regex",
"Dump only the currently loaded configurations whose names match the specified "
"regex. Can be used with both resource and mask query parameters."},
{Admin::ParamDescriptor::Type::Boolean, "include_eds",
"Dump currently loaded configuration including EDS. See the response definition "
"for more information"}}),
makeHandler("/init_dump", "dump current Envoy init manager information (experimental)",
MAKE_ADMIN_HANDLER(init_dump_handler_.handlerInitDump), false, false),
MAKE_ADMIN_HANDLER(init_dump_handler_.handlerInitDump), false, false,
{{Admin::ParamDescriptor::Type::String, "mask",
"The desired component to dump unready targets. The mask is parsed as "
"a ProtobufWkt::FieldMask. For example, get the unready targets of "
"all listeners with /init_dump?mask=listener`"}}),
makeHandler("/contention", "dump current Envoy mutex contention stats (if enabled)",
MAKE_ADMIN_HANDLER(stats_handler_.handlerContention), false, false),
makeHandler("/cpuprofiler", "enable/disable the CPU profiler",
Expand All @@ -118,8 +134,13 @@ AdminImpl::AdminImpl(const std::string& profile_path, Server::Instance& server,
makeHandler("/hot_restart_version", "print the hot restart compatibility version",
MAKE_ADMIN_HANDLER(server_info_handler_.handlerHotRestartVersion), false,
false),

// TODO(jmarantz): add support for param-passing through a POST. Browsers send
// those params as the post-body rather than query-params and that requires some
// re-plumbing through the admin callback API. See also drain_listeners.
makeHandler("/logging", "query/change logging levels",
MAKE_ADMIN_HANDLER(logs_handler_.handlerLogging), false, true),

makeHandler("/memory", "print current allocation/heap usage",
MAKE_ADMIN_HANDLER(server_info_handler_.handlerMemory), false, false),
makeHandler("/quitquitquit", "exit the server",
Expand All @@ -132,9 +153,16 @@ AdminImpl::AdminImpl(const std::string& profile_path, Server::Instance& server,
MAKE_ADMIN_HANDLER(server_info_handler_.handlerServerInfo), false, false),
makeHandler("/ready", "print server state, return 200 if LIVE, otherwise return 503",
MAKE_ADMIN_HANDLER(server_info_handler_.handlerReady), false, false),
makeStreamingHandler("/stats", "print server stats", stats_handler_, false, false),
stats_handler_.statsHandler(),
makeHandler("/stats/prometheus", "print server stats in prometheus format",
MAKE_ADMIN_HANDLER(stats_handler_.handlerPrometheusStats), false, false),
MAKE_ADMIN_HANDLER(stats_handler_.handlerPrometheusStats), false, false,
{{ParamDescriptor::Type::Boolean, "usedonly",
"Only include stats that have been written by system since restart"},
{ParamDescriptor::Type::Boolean, "text_readouts",
"Render text_readouts as new gaugues with value 0 (increases Prometheus "
"data size)"},
{ParamDescriptor::Type::String, "filter",
"Regular expression (ecmascript) for filtering stats"}}),
makeHandler("/stats/recentlookups", "Show recent stat-name lookups",
MAKE_ADMIN_HANDLER(stats_handler_.handlerStatsRecentLookups), false, false),
makeHandler("/stats/recentlookups/clear", "clear list of stat-name lookups and counter",
Expand All @@ -147,18 +175,35 @@ AdminImpl::AdminImpl(const std::string& profile_path, Server::Instance& server,
"/stats/recentlookups/enable", "enable recording of reset stat-name lookup names",
MAKE_ADMIN_HANDLER(stats_handler_.handlerStatsRecentLookupsEnable), false, true),
makeHandler("/listeners", "print listener info",
MAKE_ADMIN_HANDLER(listeners_handler_.handlerListenerInfo), false, false),
MAKE_ADMIN_HANDLER(listeners_handler_.handlerListenerInfo), false, false,
{{Admin::ParamDescriptor::Type::Enum,
"format",
"File format to use",
{"text", "json"}}}),
makeHandler("/runtime", "print runtime values",
MAKE_ADMIN_HANDLER(runtime_handler_.handlerRuntime), false, false),
makeHandler("/runtime_modify", "modify runtime values",
makeHandler("/runtime_modify",
"Adds or modifies runtime values as passed in query parameters. To delete a "
"previously added key, use an empty string as the value. Note that deletion "
"only applies to overrides added via this endpoint; values loaded from disk "
"can be modified via override but not deleted. E.g. "
"?key1=value1&key2=value2...",
MAKE_ADMIN_HANDLER(runtime_handler_.handlerRuntimeModify), false, true),
makeHandler("/reopen_logs", "reopen access logs",
MAKE_ADMIN_HANDLER(logs_handler_.handlerReopenLogs), false, true),
},
date_provider_(server.dispatcher().timeSource()),
admin_filter_chain_(std::make_shared<AdminFilterChain>()),
local_reply_(LocalReply::Factory::createDefault()),
ignore_global_conn_limit_(ignore_global_conn_limit) {}
ignore_global_conn_limit_(ignore_global_conn_limit) {
#ifndef NDEBUG
// Verify that no duplicate handlers exist.
absl::flat_hash_set<absl::string_view> handlers;
for (const UrlHandler& handler : handlers_) {
ASSERT(handlers.insert(handler.prefix_).second);
}
#endif
}

Http::ServerConnectionPtr AdminImpl::createCodec(Network::Connection& connection,
const Buffer::Instance& data,
Expand Down Expand Up @@ -322,6 +367,13 @@ void AdminImpl::getHelp(Buffer::Instance& response) {
// Prefix order is used during searching, but for printing do them in alpha order.
for (const UrlHandler* handler : sortedHandlers()) {
response.add(fmt::format(" {}: {}\n", handler->prefix_, handler->help_text_));
for (const ParamDescriptor& param : handler->params_) {
response.add(fmt::format(" {}: {}", param.id_, param.help_));
if (param.type_ == ParamDescriptor::Type::Enum) {
response.addFragments({"; One of (", absl::StrJoin(param.enum_choices_, ", "), ")"});
}
response.add("\n");
}
}
}

Expand All @@ -331,12 +383,15 @@ const Network::Address::Instance& AdminImpl::localAddress() {

AdminImpl::UrlHandler AdminImpl::makeHandler(const std::string& prefix,
const std::string& help_text, HandlerCb callback,
bool removable, bool mutates_state) {
return UrlHandler{prefix, help_text, RequestGasket::makeGen(callback), removable, mutates_state};
bool removable, bool mutates_state,
const ParamDescriptorVec& params) {
return UrlHandler{prefix, help_text, RequestGasket::makeGen(callback),
removable, mutates_state, params};
}

bool AdminImpl::addStreamingHandler(const std::string& prefix, const std::string& help_text,
GenRequestFn callback, bool removable, bool mutates_state) {
GenRequestFn callback, bool removable, bool mutates_state,
const ParamDescriptorVec& params) {
ASSERT(prefix.size() > 1);
ASSERT(prefix[0] == '/');

Expand All @@ -353,16 +408,17 @@ bool AdminImpl::addStreamingHandler(const std::string& prefix, const std::string
auto it = std::find_if(handlers_.cbegin(), handlers_.cend(),
[&prefix](const UrlHandler& entry) { return prefix == entry.prefix_; });
if (it == handlers_.end()) {
handlers_.push_back({prefix, help_text, callback, removable, mutates_state});
handlers_.push_back({prefix, help_text, callback, removable, mutates_state, params});
return true;
}
return false;
}

bool AdminImpl::addHandler(const std::string& prefix, const std::string& help_text,
HandlerCb callback, bool removable, bool mutates_state) {
HandlerCb callback, bool removable, bool mutates_state,
const ParamDescriptorVec& params) {
return addStreamingHandler(prefix, help_text, RequestGasket::makeGen(callback), removable,
mutates_state);
mutates_state, params);
}

bool AdminImpl::removeHandler(const std::string& prefix) {
Expand Down
42 changes: 42 additions & 0 deletions source/server/admin/admin.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
.home-table {
font-family: sans-serif;
font-size: medium;
border-collapse: collapse;
border-spacing: 0;
}

.home-data {
text-align: left;
padding: 4px;
}

.home-form {
margin-bottom: 0;
}

.button-as-link {
background: none!important;
border: none;
padding: 0!important;
font-family: sans-serif;
font-size: medium;
color: #069;
text-decoration: underline;
cursor: pointer;
}

.gray {
background-color: #dddddd;
}

.vert-space {
height: 4px;
}

.option {
padding-bottom: 4px;
padding-top: 4px;
padding-right: 4px;
padding-left: 20px;
text-align: right;
}
Loading