test: adding some debug logs to diagnose hang#6875
test: adding some debug logs to diagnose hang#6875alyssawilk merged 4 commits intoenvoyproxy:masterfrom
Conversation
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
|
I can't repro #6874 locally - this is my best idea is to try to get some more information. |
| initialize(); | ||
|
|
||
| // Debug logs for #6874 | ||
| std::cerr << "Waiting for load stats stream.\n"; |
There was a problem hiding this comment.
I think you'll need use std::endl as end of line so it flushes the buffer.
|
Hopefully not needed anymore via #6784 |
|
lizan merged commit 27b3e48 into envoyproxy:master 3 days ago Sat, May 11, 8:07 PM (2 days ago) |
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
mattklein123
left a comment
mattklein123
left a comment
There was a problem hiding this comment.
Agreed let's merge this and see if we can figure this out.
|
Thanks. One more hang last night |
For posterity #6875 got us the information it was a "waiting for stats" bug, which is then debuggable by lowering load stats interval to 1ms. This exposed two issues with merge. When request 1 arrived in a separate interval from request 2, the stats protos could arrive in the opposite order (fixed by ignoring order) and that if there was a stats-update with a request in progress it never resolved because we were always adding fields (oops!) I'm tempted to leave the interval low now that this is thoroughly debugged, but there's expectations on the time being within a range which wouldn't work if we lower it so meh? Risk Level: n/a (test only) Testing: yes. very much so. Docs Changes: n/a Release Notes: n/a Fixes #6874 Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
3 participants
Risk Level: n/a (test only)
Testing: yep
Docs Changes: no
Release Notes: no
#6874