fix(appset): Don't use revision cache when reconciling after webhook (#16062)

[dhruvang1]

Repo Server is using ref caching in GetGitFiles which prevents AppSet from seeing the new commit. There are few approaches to solve it:

• Disable Ref caching for GetGitFiles.
  • For each AppSet reconciliation, both from webhook & periodic sync, poll the Git Server to return the latest refs.
• Disable Ref caching only for webhook requests (Implemented here)
  • Add noRevisionCache parameter in GitFilesRequest. On reconciliation triggered from webhook, pass noRevisionCache: true to repo server to force the Git Server polling to get the latest commit.
• Explicitly pass the commit sha in request (complex as an AppSet can have multiple git generator)
  • We store the commit sha obtained from the webhook in AppSet annotation. On next reconciliation, pass the commit sha as revision. Repo server has the logic to handle commit sha and no changes are required there.

Checklist:

• Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this does not need to be in the release notes.
• The title of the PR states what changed and the related issues number (used for the release note).
• The title of the PR conforms to the Toolchain Guide
• I've included "Closes [ISSUE #]" or "Fixes [ISSUE #]" in the description to automatically close the associated issue.
• I've updated both the CLI and UI to expose my feature, or I plan to submit a second PR with them.
• Does this PR require documentation updates?
• I've updated documentation as required by this PR.
• Optional. My organization is added to USERS.md.
• I have signed off all my commits as required by DCO
• I have written unit and/or e2e tests for my change. PRs without these are unlikely to be merged.
• My build is green (troubleshooting builds).
• My new feature complies with the feature status guidelines.
• I have added a brief description of why this PR is necessary and/or what this PR solves.

Fixes #16062

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (58da6a3) 49.51% compared to head (95ec130) 49.52%.

Additional details and impacted files

@@           Coverage Diff           @@
##           master   #16241   +/-   ##
=======================================
  Coverage   49.51%   49.52%           
=======================================
  Files         269      269           
  Lines       46990    46995    +5     
=======================================
+ Hits        23269    23274    +5     
  Misses      21437    21437           
  Partials     2284     2284           

Files	Coverage Δ
applicationset/generators/git.go	84.50% <100.00%> (+0.14%)	⬆️
applicationset/services/repo_service.go	88.88% <100.00%> (+0.42%)	⬆️
reposerver/repository/repository.go	60.12% <100.00%> (+0.02%)	⬆️

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ericblackburn]

LGTM, but I am not a Code Owner

[dhruvang1]

@crenshaw-dev could you take a look when you are available.

[Laakso]

Awesome! Thanks for the fix! We've been waiting for this :) ❤️

[alexmt]

LGTM!

[alexmt]

/cherry-pick release-2.9

[alexmt]

/cherry-pick release-2.8

[gcp-cherry-pick-bot]

Cherry-pick failed with Merge error 7408292bb058fcb1feb62b27cc7b348b2a4b2c68 into temp-cherry-pick-680fdf-release-2.8

[jmalloc]

Any chance of getting a tag of 2.9.x that includes this fix? 🙏

[jolet]

hi we are experiencing same issue with AppSets not reconciliating new Apps, currently we found temp fix is to do manual hard refresh on any existing App from AppSet which triggers reconcilliation immediately, but thats cumbersome and prone to pointing teams to overuse hard refresh for everything