Use service annotations to choose IPVS load balancing method

[murali-reddy]

Since Service manifest does not have support for choosing load balancing method, use service annotations to add meta data to service to specify load balancing method. Use this method details to configure ipvs service.

On side note how useful it is needs to be analyzed. Given that each node is making decision on the knowledge of connections it is aware of. Even if a node performs least connection load balancing it does not necessarily mean endpoint has least connection. As there can be connetions that are load balanced from across the cluster nodes. This is nature of distributed load balancer.

[bzub]

This is worth pursuing since work we do allowing users to customize their load-balancing setup for Kubernetes Ingress and beyond would then be usable here at the Service level and vice-versa. Letting users get more granular at the Service level would be nice.

[bzub]

You're right though that this isn't particularly useful with independent IPVS loadbalancers on each node and traditional schedulers. If we get #10 implemented then it becomes useful.