Skip to content

stats: adds metrics sink with ack#1176

Closed
jingwei99 wants to merge 1 commit intomainfrom
metrics_service
Closed

stats: adds metrics sink with ack#1176
jingwei99 wants to merge 1 commit intomainfrom
metrics_service

Conversation

@jingwei99
Copy link
Copy Markdown
Contributor

@jingwei99 jingwei99 commented Nov 11, 2020

Description: Replace the metrics sink from the envoy one to a custom one. The custom metrics sink comes with ack function on the grpc stream.
Risk Level: High
Testing: Local and ci unit tests and integration tests

Co-authored-by: Jose Nino jnino@lyft.com
Signed-off-by: Jingwei Hao jingwei.hao@gmail.com

Co-authored-by: Jose Nino <jnino@lyft.com>
Signed-off-by: Jingwei Hao <jingwei.hao@gmail.com>
Copy link
Copy Markdown
Member

@junr03 junr03 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great progress! First pass of comments.

#include "extensions/filters/http/dynamic_forward_proxy/config.h"
#include "extensions/filters/http/router/config.h"
#include "extensions/filters/network/http_connection_manager/config.h"
#include "extensions/stat_sinks/metrics_service/config.h"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would keep both around per my thought that it would be good, for a period of time, to emit metrics to both sinks and compare.

Additionally, we have not done the server work required to receive metrics from this sink, so we still need to emit using the old sink.

"envoy.filters.http.router": "//source/extensions/filters/http/router:config",
"envoy.filters.network.http_connection_manager": "//source/extensions/filters/network/http_connection_manager:config",
"envoy.stat_sinks.metrics_service": "//source/extensions/stat_sinks/metrics_service:config",
"envoymobile.stat_sinks.metrics_service": "@envoy_mobile//library/common/extensions/stat_sinks/metrics_service:config",
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"envoymobile.stat_sinks.metrics_service": "@envoy_mobile//library/common/extensions/stat_sinks/metrics_service:config",
"envoy.stat_sinks.metrics_service": "@envoy_mobile//library/common/extensions/stat_sinks/metrics_service:config",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have kept the envoy namespacefor the other extensions. In this case we changed toenvoymobile` and kept the rest of the name the same.

@goaway do you have an opinion about changing the namespace to envoymobile versus keeping it in envoy and changing the nested types (which is what was done in the actual c++ classes?

"envoy.transport_sockets.tls": "//source/extensions/transport_sockets/tls:config",
}
WINDOWS_EXTENSIONS = {}
WINDOWS_EXTENSIONS = {} No newline at end of file
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

newline

)

envoy_cc_library(
name = "metrics_service_with_response",
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
name = "metrics_service_with_response",
name = "metrics_service",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the with_response suffix is a little clunky. I'd rather keep the name shorter and write a good comment in the header file that explains what is different from the streamer here vs in envoy.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's also inconsistent with the class name chosen below EnvoyMobileMetricsService.

@@ -0,0 +1,34 @@
syntax = "proto3";

package envoymobile.extensions.StatSinks.MetricsService;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
package envoymobile.extensions.StatSinks.MetricsService;
package envoymobile.extensions.stat_sinks.metrics_service;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that this means we will have to change this type in other places (including the service_method).


void send(MetricsService::MetricsPtr&& metrics) override;

void onReceiveMessage(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move the method definition (body) to the cc file. This method will eventually be larger.

} // namespace EnvoyMobileMetricsService
} // namespace StatSinks
} // namespace Extensions
} // namespace Envoy No newline at end of file
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

newline. You can run the formaters locallly with ./tools/check_format.sh fix and also pre-commit run --all-files

// Send an empty response and end the stream. This should never happen but make sure nothing
// breaks and we make a new stream on a follow up request.
metrics_service_request_->startGrpcStream();
envoymobile::extensions::StatSinks::MetricsService::EnvoyMobileStreamMetricsResponse response_msg;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should set a batch id and verify it.

InSequence s;

// Start a stream and send first message.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

std::make_unique<Envoy::Protobuf::RepeatedPtrField<io::prometheus::client::MetricFamily>>();
streamer_->send(std::move(metrics));
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be a test for receiving a response message.

@jingwei99
Copy link
Copy Markdown
Contributor Author

jingwei99 commented Nov 14, 2020

opened #1183 instead

@jingwei99 jingwei99 closed this Nov 14, 2020
@junr03 junr03 deleted the metrics_service branch July 20, 2021 22:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants