chore(cmp): refactor and test plugin discovery by crenshaw-dev · Pull Request #20217 · argoproj/argo-cd

crenshaw-dev · 2024-10-03T17:23:40Z

Plugin discovery logic is complicated and has resulted in difficult to diagnose bugs.

This PR makes a few changes:

Deprecate RepositoryResponse.IsDiscoveryEnabled field. No code actually uses that field anymore since we do a pre-flight CMP request to determine whether it has discovery enabled.
Break cmpSupports into namedCMPSupports and unnamedCMPSupports. Reading cmpSupports was difficult, because the namedPlugin parameter implied more than the name suggested. Specifically, when namedPlugin == false, we could assume that the app spec did not contain a plugin.name. By using two different functions with two different names, we make it very clear that one is for use when the plugin name is specified, and the other is used when the plugin name is not specified.
Make the CMP client constructor mock-able. It requires some code overhead, but discovery is so deeply entangled with CMP server API calls that I think it's worth it.
Test the behavior of both unnamedCMPSupports and namedCMPSupports. I'd love to test all the way up to DetectConfigManagementPlugin, but I think that would require more mock logic than I'm comfortable with.
Use securejoin instead of our internal path traversal detection logic - it's more robust.

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

bunnyshell · 2024-10-03T17:23:44Z

❗ Preview Environment undeploy from Bunnyshell failed

See: Environment Details | Pipeline Logs

Available commands (reply to this comment):

🚀 /bns:deploy to redeploy the environment
❌ /bns:delete to try again to remove the environment

bunnyshell · 2024-10-03T17:23:45Z

✅ Preview Environment created on Bunnyshell but will not be auto-deployed

See: Environment Details

Available commands (reply to this comment):

🚀 /bns:deploy to deploy the environment

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

codecov · 2024-10-03T18:50:30Z

Codecov Report

Attention: Patch coverage is 79.48718% with 16 lines in your changes missing coverage. Please review.

Please upload report for BASE (master@b2091e3). Learn more about missing BASE report.
Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
util/app/discovery/discovery.go	76.47%	11 Missing and 5 partials ⚠️

Additional details and impacted files

@@            Coverage Diff            @@
##             master   #20217   +/-   ##
=========================================
  Coverage          ?   55.98%           
=========================================
  Files             ?      322           
  Lines             ?    44569           
  Branches          ?        0           
=========================================
  Hits              ?    24951           
  Misses            ?    17047           
  Partials          ?     2571

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

crenshaw-dev · 2024-10-03T19:11:41Z

@CodiumAI-Agent /review

QodoAI-Agent · 2024-10-03T19:12:26Z

PR Reviewer Guide 🔍

(Review updated until commit `cf553e6`)

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review Refactoring The refactoring of the `DetectConfigManagementPlugin` function to improve modularity and testability is significant. Ensure that the new structure does not introduce any regressions or misunderstandings about how plugins are detected and managed. Logic Change The changes in how discovery is handled (removing `isDiscoveryEnabled` and using `isSupported` only) could affect existing functionality. Verify that these changes align with the intended use cases and do not negatively impact existing workflows.

util/app/discovery/discovery.go

leoluz · 2024-10-04T01:55:10Z

util/app/discovery/discovery.go

-		// check if the given plugin supports the repo
-		conn, cmpClient, connFound = cmpSupports(ctx, pluginSockFilePath, appPath, repoPath, fmt.Sprintf("%v.sock", pluginName), env, tarExcludedGlobs, true)
-		if !connFound {
+		c := newSocketCMPClientConstructorForPluginName(pluginSockFilePath, pluginName)


Not sure if I am understanding it correctly but this PR introduces the new CMPClientConstructor interface however it seems that it is impossible to use a different implementation than the socketCMPClientConstructor. Not sure if this was intentional, but it would be great if we could enable the flexibility to use different CMPClientConstructor implementations. This could help simplifying the CMP as a Service PR.

it seems that it is impossible to use a different implementation

The only other implementation is the mock implementation used for testing.

The other option would have been to have (un)namedCMPSupports accept a getClient function, but the code just looked ickier. I could go either way.

util/app/discovery/discovery.go

pradithya

I wonder if the e2e test include plugin by default?

cmpserver/plugin/plugin.go

util/app/discovery/discovery.go

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

pradithya · 2024-10-04T23:36:48Z

LGTM

QodoAI-Agent · 2024-10-04T23:37:07Z

Persistent review updated to latest commit cf553e6

andrii-korotkov-verkada · 2024-11-09T04:30:27Z

cmpserver/plugin/plugin.proto

+    //             longer uses an expensive MatchRepository service to determine if discovery is enabled. Instead, we
+    //             make a relatively cheap pre-flight request to the CheckPluginConfiguration service. We'll leave this
+    //             field here to avoid errors on mismatched repo-server and cmp-server versions.
    bool isDiscoveryEnabled = 2;


Do we use a strict parsing mode for protobuf where unrecognized fields would trigger error?

crenshaw-dev added 4 commits October 3, 2024 10:46

chore(cmp): clean up and test discovery logic

5fd79e1

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

refactor

2f197e2

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

better comment

ecfe24f

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

sort imports

932beb7

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

crenshaw-dev added 2 commits October 3, 2024 13:24

remove needless whitespace

7911b2f

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

Merge remote-tracking branch 'origin/master' into test-cmp-discovery

60a5b40

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

crenshaw-dev marked this pull request as ready for review October 3, 2024 19:11

crenshaw-dev requested a review from a team as a code owner October 3, 2024 19:11

leoluz requested changes Oct 4, 2024

View reviewed changes

pradithya reviewed Oct 4, 2024

View reviewed changes

cmpserver/plugin/plugin.go Show resolved Hide resolved

util/app/discovery/discovery.go Show resolved Hide resolved

util/app/discovery/discovery.go Show resolved Hide resolved

util/app/discovery/discovery.go Outdated Show resolved Hide resolved

fixes from comments

cf553e6

Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com>

andrii-korotkov-verkada reviewed Nov 9, 2024

View reviewed changes

AJChandler mentioned this pull request Nov 20, 2024

fix: Memory leak in repo-server #20876

Merged

14 tasks

Conversation

crenshaw-dev commented Oct 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bunnyshell bot commented Oct 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❗ Preview Environment undeploy from Bunnyshell failed

Uh oh!

bunnyshell bot commented Oct 3, 2024

✅ Preview Environment created on Bunnyshell but will not be auto-deployed

Uh oh!

codecov bot commented Oct 3, 2024

Codecov Report

Uh oh!

crenshaw-dev commented Oct 3, 2024

Uh oh!

QodoAI-Agent commented Oct 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Reviewer Guide 🔍

(Review updated until commit cf553e6)

Uh oh!

Uh oh!

leoluz Oct 4, 2024

Choose a reason for hiding this comment

Uh oh!

crenshaw-dev Oct 4, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pradithya left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pradithya commented Oct 4, 2024

Uh oh!

QodoAI-Agent commented Oct 4, 2024

Uh oh!

andrii-korotkov-verkada Nov 9, 2024

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

crenshaw-dev commented Oct 3, 2024 •

edited

Loading

bunnyshell bot commented Oct 3, 2024 •

edited

Loading

QodoAI-Agent commented Oct 3, 2024 •

edited

Loading

(Review updated until commit `cf553e6`)