k8gb CoreDNS exposed over hostnetwork ingress - programmatic lookup

[k0da]

I'd like to discuss alternative approach on how to programmatically find CoreDNS IPs in a scenario it is exposed via ingress-controller running on hostNetwork (tcp/udp configmap)

Configuration for such case is not really controlled nor related to k8gb setup. Although k8gb needs a consistent how to discover CoreDNS IPs in order to build Zone delegation properly.

Current setup have following limitations:

• It assumes CoreDNS runs behind the same Ingress controller as Application. So it takes status.Loadbalancer despite the fact those can run in different IngressClasses.
• Zone is not created until really first GLSB/Ingress is created
• ZoneDelegation reconciles with GSLB which leads excessive Edge API access (Imagine 1500 Ingress/GSLB with re-queue every 30 seconds * by number of clusters forming a k8gb fleet)

Proposal:

• introduce annotation on CoreDNS service (ClusterIP) with information on how to access ingress object (could be fake object) to get status.LoadBalancer info. This way we ensure user maintains an object to reflect CoreDNS exposure not Application, IngressClass must be correct etc and status.LoadBalancer is set.
• Reconcile ZoneDelegation in a separate loop with separate re-queue time and separate watches Watch to CoreDNS svc for type LoadBalancer and given Ingress

[ytsarev]

introduce annotation on CoreDNS service (ClusterIP) with information on how to access ingress object (could be fake object) to get status.LoadBalancer info. This way we ensure user maintains an object to reflect CoreDNS exposure not Application, IngressClass must be correct etc and status.LoadBalancer is set.

It's not super clear how it will work. Can you provide draft manifest examples to represent the idea from a configuration standpoint?

Reconcile ZoneDelegation in a separate loop with separate re-queue time and separate watches Watch to CoreDNS svc for type LoadBalancer and given Ingress

Will it require a dedicated CR, or can we avoid it?

[k0da]

Yes. No additional CR required. On startup (separate reconciliation loop) we lookup for CordDNS service as we do now (COREDNS_EXPOSED flag is going away).

Based on the CoreDNS service type we do:

• LoadBalancer - get ip from status
• ClusterIP - do our best and check:
  • k8gb.io/coredns-exposure-type to be Ingress
  • k8gb.io/coredns-ingress-ref to contain namspaced obj reference. (A reference to standalone Ingress Object holding status.Loadbalancer for ingressclass coredns is exposed through)
    In case of error full stop.

What could be implemented later:
k8gb.io/coredns-exposure-type static_ips and k8gb.io/coredns-static-ips to be "1.2.3.4,4.3.2.1"

Then we simply install k8gb with following values:

coredns:
  service:
    annotations:
      k8gb.io/coredns-exposure-type: "ingress"
      k8gb.io/coredns-ingress-ref: "k8gb/coredns-ing"