feat(operator): update pod definition and nifi reconciliation for istio compatibility

[nfoucha]

Q	A
Bug fix?	no
New feature?	yes
API breaks?	no
Deprecations?	no
Related tickets	partially #172
License	Apache 2.0

What's in this PR?

This pushes the zookeeper connectivity check into the pod command instead of the init container. Because the nifi image doesn't have netcat, the implementation was adjusted to use curl with a telnet address and a bad option so it immediately terminates the connection. This results in a specific exit code 48 when the connectivity check passes. However, the overall logic is unchanged.

The reconciliation checks were also adjusted to remove prometheus annotations and the liveness probe definition from consideration when checking for the difference between the currently running and the "desired" pod state when the pod has been affected by istio injection. Additional containers present on the current pod spec are carried over to the desired pod spec to ensure injected containers don't trigger reconciliation.

Additional context

There are a couple of caveats with this approach to Istio compatibility that are important to call out. I believe these can be addressed via an architecture change to utilize a StatefulSet over reconciling the individual pods directly, but that was out of scope for what I'm trying to accomplish with this PR.

• If an incoming spec change only targets the liveness probe definition, the operator will ignore the change and not reconcile the pods. This can be worked around by adding/removing annotations/labels.
• If an incoming spec change removes custom init containers and/or sidecars, the operator will ignore the change and not reconcile the pods. This can be worked around by adding/removing annotations/labels.

Checklist

• Implementation tested
• Error handling code meets the guideline
• Logging code meets the guideline
• User guide and development docs updated (if needed)
• Append changelog with changes

To Do

• Discuss pulling in changes from Draft: fix(nifikop): only use prefix of hostname in node DTO comparison #127 (issue Nodes constantly deleted and recreated in non-headless mode #130 not required, but encountered during development)

[nfoucha]

Worth mentioning that I tested this with zookeeper (1.28.0, 2.0.0) as well as the new kubernetes state management (2.0.0)

[juldrixx]

Can you update the CHANGELOG.md file to reflect your changes?

[juldrixx]

Can you explain me why these changes are necessary for Istio?
I'm pretty sure we were making it work when we wrote this https://konpyutaika.github.io/nifikop/docs/3_manage_nifi/1_manage_clusters/1_deploy_cluster/3_expose_cluster/2_istio_service_mesh but we moved from it so not sure if things change since then.

[juldrixx]

Can you update the CHANGELOG.md file to reflect your changes?

And can also rebase your PR and check if everything is still fine (we never know 😀).

[nfoucha]

When using an init container that requires network connectivity, and in an environment with restrictive NetworkPolicy objects, it will fail continuously because the istio-init container has not completed setting up the iptables for the pod. By moving the Zookeeper networking check to the main container instead it eliminates the race condition and waits until the istio networking setup is complete before attempting a connection.

[nfoucha]

The other changes relate to the mutations performed on the pod by istio injection. This causes the reconciler to incorrectly flag the pods as out of sync and constantly recreates them as a result. By adjusting the "desired" pod spec to include the mutated parts of the pod, this reconciliation infinite loop is fixed.

[juldrixx]

Thanks for the contribution. Will merge after the pipeline ends.