feat: Add sync-dependencies

[alexec]

Depended on by argo-cd#3517

This PR introduces a new feature: sync dependencies.

Sync dependencies give a new way to order sync operations that is influenced by sync waves and sync hooks (all of which I wrote, so I know it extremely well).

Like sync waves, a sync operation will only progress when all the dependents have completed.

Dependencies between objects is specified by the new argocd.argoproj.io/sync-dependencies annotation.

# The dependencies are specified as a comma-separated list of objects.
# The format of each object is <group>/<kind>/<namespace>/<name>
# The group and kind are optional. If not specified, they'll match any group or kind.
# The namespace is optional. If not specified, the namespace is assumed to be the same as the namespace of the object.
# The name is required.
argocd.argoproj.io/sync-dependencies: a,b,c

Reasons to do this:

• The 3rd most popular enhancement request for Argo CD
• Explanation can be found in argo-cd#3517.
• Implementation is contained within the GitOps engine.

Reasons not to do this:

• Impacts core GitOps engine code (must be mitigated by exceptional testing).
• Another gun to shoot yourself in the foot with.

Open questions:

• Sync waves, hooks, and dependencies have a certain level of opacity to them. By which I mean, that it is not always clear what they're doing unless you're familiar with the app. I wonder if we should introduce a new "waiting" state for resources?
• Should dependencies take precedence over waves? I think this is yes.
• How should dependencies be described? I've used annotation, is that right? Is that the right structure?
• Are the rules that match tasks to dependencies?

[alexec]

In syncTasks we need to check for cyclic-dependencies.

[alexec]

Looks like a 2+ year old bug.

[codecov]

Codecov Report

Attention: 7 lines in your changes are missing coverage. Please review.

Comparison is base (ed70eac) 55.75% compared to head (745fb5a) 55.61%.
Report is 18 commits behind head on master.

❗ Current head 745fb5a differs from pull request most recent head de3f8c9. Consider uploading reports for the commit de3f8c9 to get more accurate results

Files	Patch %	Lines
pkg/sync/sync_context.go	64.28%	4 Missing and 1 partial ⚠️
pkg/sync/sync_tasks.go	94.73%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #514      +/-   ##
==========================================
- Coverage   55.75%   55.61%   -0.15%     
==========================================
  Files          41       42       +1     
  Lines        4525     4531       +6     
==========================================
- Hits         2523     2520       -3     
- Misses       1808     1817       +9     
  Partials      194      194              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[alexec]

Suggested change

sc.log.WithValues("tasks", tasks).V(1).Info("tasks after sorting")

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
3 Code Smells

No Coverage information
8.0% Duplication

[sonarqubecloud]

Quality Gate passed

Issues
3 New issues

Measures
0 Security Hotspots
No data about Coverage
8.0% Duplication on New Code

See analysis details on SonarCloud

[reggie-k]

I wonder whether we can hit a scenario of a dependent resource that has an auto-generated and thus unknown ahead of time name?

[andrii-korotkov-verkada]

Is it possible to fix CI?

[or-shachar]

Hey @alexec - I have a feeling that this would be a major gamechanger in syncing large changes with fine-grain dependencies. Is there anything we can do to help this get pushed? What are the outstanding issues? We can help in coding/review - whatever needed.

[crenshaw-dev]

@or-shachar see discussion here: argoproj/argo-cd#7437

I don't think there's been sufficient work done on the proposal side to show that the feature is necessary/justified or describing the details of how it will function.

Gonna close this for now, since I don't think Alex is actively pursuing the feature.