Occasionally Ingress Controller introduces significant delays in process ingress events

[asridharan]

Describe the bug
Occasionally, when we modify the Ingress resource it takes to the order tens of minutes to update the Application Gateway configuration through ARM.

To Reproduce
Start a Kubernetes service. Expose it through the Application Gateway by creating an Ingress resource. Scale the service up or down, or remove the Ingress resource completely and observe the time it takes to apply this configuration on the Application Gateway. Most of the time the update would go through within a matter of seconds, but occasionally it will take to order tens of minutes to get applied.

Ingress Controller details

• Output of kubectl describe pod <ingress controller> . The pod name can be obtained by running helm list.
• Output of `kubectl logs .
  will add the logs here shortly.
• Any Azure support tickets associated with this issue.

[asridharan]

This issue could be related to:
Azure/AKS#522

[njzhangyifei]

This is somewhat related to the resync period. It happens when the informer misses events.

[asridharan]

This is somewhat related to the resync period. It happens when the informer misses events.

@njzhangyifei why would the informer miss events?

[njzhangyifei]

This is somewhat related to the resync period. It happens when the informer misses events.

@njzhangyifei why would the informer miss events?

I meant when they stop receiving events.

[asridharan]

Looks like the work around mentioned in this issue should resolve this problem till it fixed in AKS itself:
Azure/AKS#676

[asridharan]

@akshaysngupta could you take this on? This can be an extension of your helm work for enabling RBAC.