Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The counter metrics about the number of sent/received events in streams #12319

Closed
loyd opened this issue Mar 21, 2024 · 5 comments
Closed

The counter metrics about the number of sent/received events in streams #12319

loyd opened this issue Mar 21, 2024 · 5 comments

Comments

@loyd
Copy link

loyd commented Mar 21, 2024

What problem are you trying to solve?

Hi, I'm choosing between linkerd (first of all because the proxy is written in Rust, which is lightweight on my tests) and envoy-based solutions for a cluster of GRPC services. My primary requirement is excellent telemetry.

We heavily rely on streaming between all our services. However, linkerd doesn't provide any event counter for long-lived streams:

route_request_total{direction="outbound",dst="lve-server.default.svc.cluster.local:4242",rt_route="Subscribe"} 1

(no response metrics until the subscription is ended)

2024-03-21_19-41-33

In the meantime, envoy provides such statistics, including in the response counter:

envoy_cluster_grpc_Subscribe_response_message_count{envoy_grpc_bridge_method="CounterService",envoy_grpc_bridge_service="counter",envoy_cluster_name="lve_server"} 327336

How should the problem be solved?

route_request_total (or another dedicated metric) should calculate the number of events in long-lived streaming

Any alternatives you've considered?

How would users interact with this feature?

No response

Would you like to work on this feature?

Maybe

@olix0r
Copy link
Member

olix0r commented Apr 3, 2024

It sounds like this is more generally asking for gRPC-specific metrics. As far as I understand, it wouldn't be possible for Linkerd to measure this sort of thing arbitrarily at the HTTP-level. We'd have to parse the response bodies to count the number of gRPC messages included in the response, for instance.

@loyd
Copy link
Author

loyd commented Apr 4, 2024

It seems that envoy count frames for this metric. I think it's a reasonable trade-off. Although, I'm not sure it's easy to implement with outdated hyper (0.14).

Anyway, will a patch with this metric be considered?

Copy link

stale bot commented Jul 3, 2024

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the wontfix label Jul 3, 2024
@loyd
Copy link
Author

loyd commented Jul 8, 2024

Linked PR is activity

@stale stale bot closed this as completed Jul 27, 2024
@loyd
Copy link
Author

loyd commented Aug 4, 2024

The bot's logic is ill-conceived =(. It should be reopened

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 4, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

2 participants