Skip to content

Backport of Filter api-gateway cache logging to reduce log output on server disconnect into release/1.2.x#2893

Merged
hc-github-team-consul-core merged 2 commits intorelease/1.2.xfrom
backport/reduce-gateway-noise/reliably-vocal-foal
Sep 5, 2023
Merged

Backport of Filter api-gateway cache logging to reduce log output on server disconnect into release/1.2.x#2893
hc-github-team-consul-core merged 2 commits intorelease/1.2.xfrom
backport/reduce-gateway-noise/reliably-vocal-foal

Conversation

@hc-github-team-consul-core
Copy link
Collaborator

Backport

This PR is auto-generated from #2880 to be assessed for backporting due to the inclusion of the label backport/1.2.x.

The below text is copied from the body of the original PR.


Changes proposed in this PR:

  • I noticed this when I was working on a customer issue and it made debugging more difficult
  • I was working on an issue that was reproduced by deleting the consul-server-0 pod and the api-gateway logging spit out about 20k log lines in about 5 seconds
  • I don't think we need to log these errors out at the api-gateway level as other parts of the controller will log a similar error out but less frequently.
  • The customer will still see that the consul server is unreachable in the logs (see below)

Instead of 20k of these:

2023-08-31T14:33:54.861Z ERROR error fetching config entries for kind: http-route {"error": "Get "http://10.244.0.8:8500/v1/config/http-route?dc=dc1&index=10\": dial tcp 10.244.0.8:8500: connect: connection refused"}

They still see:

2023-08-31T14:41:43.031Z ERROR Reconciler error {"controller": "endpoints", "controllerGroup": "", "controllerKind": "Endpoints", "Endpoints": {"name":"consul-ui","namespace":"consul"}, "namespace": "consul", "name": "consul-ui", "reconcileID": "ce66b001-0173-40d1-95ab-a7a5363a6612", "error": "1 error occurred:\n\t* Get "http://:8500/v1/catalog/nodes?dc=dc1&filter=Meta%5B%22synthetic-node%22%5D+%3D%3D+%22true%22+&ns=%2A": dial tcp: lookup : no such host\n\n"}

How I've tested this PR:

  • build locally, delete consul-server-0 and look at the connect injector logs.

How I expect reviewers to test this PR:

👀

Checklist:


Overview of commits

@hc-github-team-consul-core hc-github-team-consul-core force-pushed the backport/reduce-gateway-noise/reliably-vocal-foal branch from 1e99716 to 62ab9cb Compare September 5, 2023 13:39
@hc-github-team-consul-core hc-github-team-consul-core force-pushed the backport/reduce-gateway-noise/reliably-vocal-foal branch from f0e43be to e43ef00 Compare September 5, 2023 13:39
@hc-github-team-consul-core hc-github-team-consul-core merged commit 653e7fd into release/1.2.x Sep 5, 2023
@hc-github-team-consul-core hc-github-team-consul-core deleted the backport/reduce-gateway-noise/reliably-vocal-foal branch September 5, 2023 14:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants