Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bug]: Error refused errors generate too many error logs #785

Closed
slacki123 opened this issue Jun 28, 2024 · 0 comments · Fixed by #794
Closed

[bug]: Error refused errors generate too many error logs #785

slacki123 opened this issue Jun 28, 2024 · 0 comments · Fixed by #794
Labels
bug Something isn't working

Comments

@slacki123
Copy link
Contributor

Describe the bug

When running the operator for the first time without any connection configuration, we quickly get an overwhelming spam of error connection logs on startup

This also happens if the operator loses connection somewhere in the midst of processing. The number of logs is a bit overwhelming and difficult to work with

To reproduce

  1. Create an operator
  2. run the operator without any connection config
  3. see there is an overwhelming number of logs

OR

  1. create an operator and run locally with a proxy to a kubernetes cluster pointing at the correct port kubectl proxy -p 8080
  2. let the operator do some processing
  3. kill the kubectl proxy -p 8080 command
  4. An overwhelming number of logs will be generated

Expected behavior

Perhaps a 1 or 2 second delay between errors when the connection has been refused

Screenshots

image

Additional Context

No response

@slacki123 slacki123 added the bug Something isn't working label Jun 28, 2024
buehler pushed a commit that referenced this issue Jul 25, 2024
…leadership (#794)

Fixes #785.

Introduce a retry delay before attempting to reacquire leadership,
fixing the log noise generated from the API server being unavailable.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
1 participant