Backport of api-gateway: fix cache and service deletion issue into release/1.2.x

[hc-github-team-consul-core]

Backport

This PR is auto-generated from #2377 to be assessed for backporting due to the inclusion of the label backport/1.2.x.

🚨

Warning automatic cherry-pick of commits failed. If the first commit failed,
you will see a blank no-op commit below. If at least one commit succeeded, you
will see the cherry-picked commits up to, not including, the commit where
the merge conflict occurred.

The person who merged in the original PR is:
@t-eckert
This person should manually cherry-pick the original PR into a new backport PR,
and close this one when the manual backport PR is merged in.

merge conflict error: unable to process merge commit: "213a4d0ab489c04ddb502e2d5b5a3fe43d29c724", automatic backport requires rebase workflow

The below text is copied from the body of the original PR.

---

Changes proposed in this PR:

• Fixes a bug where deleting a Pod would not delete the corresponding service for a Gateway.

🐞 Bug
When a user would manually delete a Pod running Consul Dataplane for a gateway, the corresponding service in Consul was not getting deleted. This was due to a caching issue where the controller was not updating its store of active Consul services to diff against existing Consul Dataplane pods.

How I've tested this PR:

• Ran the test locally in Kind, manually deleting Pod and ensuring that stale services don't hang around.
• Ran the tests against all acceptance tests.
• Ran the test against a new acceptance test that catches this error (SO to @sarahalsmiller)

Checklist:

• Tests added
• CHANGELOG entry added

  HashiCorp engineers only, community PRs should not add a changelog entry.
  Entries should use present tense (e.g. Add support for...)

---

Overview of commits

• 896970c - a7dfa1c - 6edf812 - 213a4d0 - ef87e89 - b088e35 - 1dd8c81 - 7513cbc - 39d93a1 - 8681705 - 3178b1b - 979c906 - 02fe6d1

[t-eckert]

Closing this as the backport bot errored. Doing a manual backport.