[Backport/v1.28] identity: reload CA root cert channel on file change

[jlojosnegros]

Backport of #1775 to release-1.28

Everything is almost directly backported.
Note: some dependencies added.

Introduces a modification over the master PR, a boolean env var CA_CERT_WATCHER (default: true) that allows operators to disable the CA root cert file watcher at runtime.
When set to false, no watcher thread is started and the gRPC channel is never rebuilt on cert rotation — the startup-time cert is retained permanently.

The flag is only effective when ca_root_cert is a file path; Static and Default certs never start a watcher regardless.

[istio-testing]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[fjglira]

Hey @jlojosnegros, the backports are automatically created from PR on the master branch by using the label cherry-pick/release-1.xx

[jlojosnegros]

Hi @fjglira
after this comment #1775 (comment)
I assume the backport should be done by hand and not using the usual labels.

Has the backport been approved?

[fjglira]

@jlojosnegros: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
test_ztunnel_release-1.28 88289cc link true /test test
Details

@jlojosnegros can you please check the errors?

[fjglira]

@ilrudie, what do you think about the CA_CERT_WATCHER flag? And from my point of view, I think the default state should be false? I mean, if there are users with already a ca_root_cert as a file path, this will change the current behaviour?

Another question: should we add a release note in the istio/istio to document the change?

Note: @jlojosnegros after we agree all this, you will need to create the backport also for the 1.29 release branch

[ilrudie]

I already did a 1.29 cherry-pick here. We can flip that flag there too if folks want.

The flag was requested here in slack for 1.29 where I had brought up the topic of cherry-picking more generally.

[fjglira]

CA_CERT_WATCHER

Great, approving

[fjglira]

/retest

[ilrudie]

@fjglira, the failure is a lint/clippy one not a flake. It'll need a code fix

[jlojosnegros]

Checking ...

clippy complains about the number of arguments in CaClient::new exceeding the max of 7 arguments ...

if we cannot use '#[allow]' to disable the warning here, we should think on grouping some params to reduce the number of arguments.

Maybe the boolean flags? Or something else with low impact in the code ...

WDYT @ilrudie could we disable the warning here ( as this is just a backport ) or should we try to group some arguments in a new type?

[ilrudie]

d10aa3e

I just suppressed the lint error. A refactor on a release branch to support a release-branch-only flag doesn't make sense to me.