Standby operator pod starts reconciling after namespace change

[gyfora]

Bug Report

Based on our production observations it seems to happen that in a HA setup with 2 operator pods, the standby (follower) starts reconciling resources in parallel with the leader after a namespace change event.

This is what we see:

1. Leader operator reconciles correctly for couple days
2. At a certain time we get the following log on the standby:

  Changing namespaces on 'flinkdeploymentcontroller' Controller to [...]...

In the Flink operator log this is triggered together with: controller.changeNamespaces(namespaces);

3. After this log (and changing the namespaces), both the standby and leader starts to reconcile the events, no LeaderElection related logs can be seen otherwise.

This causes all kinds of issues with the managed resources :)

Environment

EKS

[gyfora]

cc @csviri

[csviri]

thx @gyfora , we will take a look soon