feat: Adding ClusterAnalysisTemplate support for Stage verifications

[BenHesketh21]

closes #2481

ClusterAnalysisTemplate Support

This PR adds the ability to refer to cluster level analysis templates using the clusterScope option (inspired by Argo Rollouts).
The verification process finds all the templates and combines them in one run, in the same way as before but now will also find referenced ClusterAnalysisTemplates if clusterScope is set to true.
The UI has also been updated to show both scopes of analysis templates.

[netlify]

✅ Deploy Preview for docs-kargo-akuity-io ready!

Name	Link
🔨 Latest commit	c1fd414
🔍 Latest deploy log	https://app.netlify.com/sites/docs-kargo-akuity-io/deploys/670313a8391e19000817b21d
😎 Deploy Preview	https://deploy-preview-2673.kargo.akuity.io
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[netlify]

✅ Deploy Preview for docs-kargo-io ready!

Name	Link
🔨 Latest commit	1dea36c
🔍 Latest deploy log	https://app.netlify.com/sites/docs-kargo-io/deploys/6777fdeeca061e000850dfd7
😎 Deploy Preview	https://deploy-preview-2673.docs.kargo.io
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[codecov]

Codecov Report

Attention: Patch coverage is 81.71429% with 32 lines in your changes missing coverage. Please review.

Project coverage is 51.53%. Comparing base (bb95eb4) to head (1dea36c).

Files with missing lines	Patch %	Lines
internal/rollouts/analysis_run_builder.go	66.66%	13 Missing and 2 partials ⚠️
...ternal/api/get_clusteranalysistemplate_v1alpha1.go	82.00%	6 Missing and 3 partials ⚠️
...ntroller/rollouts/api/v1alpha1/analysis_helpers.go	72.22%	5 Missing ⚠️
...rnal/api/list_clusteranalysistemplates_v1alpha1.go	85.71%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2673      +/-   ##
==========================================
+ Coverage   51.40%   51.53%   +0.12%     
==========================================
  Files         288      291       +3     
  Lines       26108    26243     +135     
==========================================
+ Hits        13421    13524     +103     
- Misses      11961    11987      +26     
- Partials      726      732       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[krancour]

@BenHesketh21 thank you for your patience with this. I've finally started reviewing this in earnest. What I've looked at so far looks solid, but this is big and there's a lot more to still look at.

Due to the size and complexity, I'm going to ask @hiddeco and @Marvin9 for help with this. Group effort.

@hiddeco since this touches the Stages reconciler, which you are actively refactoring, controller changes here probably need to be on your radar.

@Marvin9 if you don't mind looking at the UI portions of this, it would be a huge help.

I am happy to review the remainder -- e.g. API server changes, docs, and such.

We will try to get this into v1.1.0, but it cannot be guaranteed.

[krancour]

Would we want to add a "Analysis Templates" section to the left side menu? or introduce a similar tab system to the project list page?

I would suggest, instead, a "Settings" item on the sidebar (with a gear as the icon) and have the UI for this be under a "Verifications" tab on that settings page. We currently lack any sort of UI for these other "cluster-scoped" things:

• Credentials (Really, these are Secrets in designated namespaces since there's no such thing as a cluster-scoped Secret.)
• ServiceAccounts. (Again, these are actually in designated namespaces.)
• ClusterRoles + ClusterRoleBindings for the aforementioned SAs

As we add those, a tabbed settings page would prevent the left sidebar from getting too noisy. And I think this is a fairly conventional approach anyway.

[Marvin9]

I agree with @krancour here. I initially thought put in sidebar and new page but couldn't find a reason to justify its place there (its the only cluster wide settings at the moment). I am fine under "settings" with "verification" tab. I would use Menu component from Antd to have vertical tab.

[BenHesketh21]

I'll get on that, thank you for the pointers

[BenHesketh21]

@Marvin9 One thing I'm not so sure on if you could clarify. I'm looking at a vertical tab using the Menu from Antd. I'm struggling to picture how it would look and where it would go? Here's my thinking so far.

1. It could be like a second sidebar on the left
2. We could move it to the right
3. The verification and others could be children items of the settings button on the left sidebar

Let me know your thoughts, thanks.

[BenHesketh21]

I've done the main parts of this, just waiting to see what the tabs should look like if they need to be vertical. I've put it in a horizontal one for now as it was the default. But the settings page with different tabs should be working.

[Marvin9]

Sure thing @BenHesketh21 here is how I imagine it would look like considering more settings incoming

Let me know if its feasible to you otherwise I would refactor the layout later this PR and for now we can keep the tab as in your screenshot. I will put my comments after we finalise what are we scoping for this PR. Thanks!

[BenHesketh21]

@Marvin9 Apologies for the delay, I had to re-do a lot of the controller work due to the refactor, I have moved the tab to be vertical. Let me know what you think.

[krancour]

We're trying to wrap up v1.2 work by 12/20 with plans to release after the holidays.

It might be too ambitious for us to get this merged before then. I'm tentatively moving this to the v1.3 milestone, but if it somehow becomes possible to sneak this into v1.2, I won't hesitate.

[Marvin9]

Sorry @BenHesketh21 do you mind if I change the design a bit? just adjusting color, spacing and layout.. Other than that I don't think there is anything that should block UI

[BenHesketh21]

@Marvin9 Please be my guest. UI isn't my strongest area.

[Marvin9]

Done with the UI and it LGTM! Thanks @BenHesketh21

@krancour it is now ready to review the rest of the things..