[k8s] add more debug logs around k8s list_namespaced_pod #7820
Conversation
|
/quicktest-core |
cg505
left a comment
cg505
left a comment
There was a problem hiding this comment.
Is there any additional things (e.g. response headers, or request details) we can grab from the response object?
rohansonecha
force-pushed
the
query-logs
branch
from
c8ddab9 to
f6a18b5
Compare
I tried setting |
| logger.debug( | ||
| f'Query response for skypilot cluster {cluster_name_on_cloud}: ' | ||
| f'resource_version={response.metadata.resource_version}, ' | ||
| f'pod_count={len(pods)}, ' | ||
| f'continue_token={response.metadata._continue}') |
There was a problem hiding this comment.
Can we just print the entire response, instead of only the part of the members?
There was a problem hiding this comment.
Also, do we want to place this before pods = response.items so that if there is any error getting response.items the logger can print out the message?
This PR adds more debug logging around the
list_namespaced_podcall to the k8s api to help diagnose issues where the k8s api returns an empty list of pods. These debug logs can help us determine whether this case is a bug on our end (we are passing in incorrect filtering values) or in the k8s api.Tested (run the relevant ones):
bash format.sh/smoke-test(CI) orpytest tests/test_smoke.py(local)/smoke-test -k test_name(CI) orpytest tests/test_smoke.py::test_name(local)/quicktest-core(CI) orpytest tests/smoke_tests/test_backward_compat.py(local)