This repository has been archived by the owner on Jan 8, 2024. It is now read-only.
Feature: plugin debugging #1716
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
izaaklauer
commented
Jun 25, 2021
go.mod
Outdated
| @@ -49,7 +49,7 @@ require ( | |||
| github.com/hashicorp/go-hclog v0.14.1 | |||
| github.com/hashicorp/go-memdb v1.3.2 | |||
| github.com/hashicorp/go-multierror v1.1.0 | |||
| github.com/hashicorp/go-plugin v1.3.0 | |||
| github.com/hashicorp/go-plugin v1.4.2 | |||
There was a problem hiding this comment.
Upgrading to get the ProtocolVersion field on ReattachConfig, which we need to know which VersionedPlugins to use.
mitchellh
reviewed
Jun 25, 2021
izaaklauer
commented
Jun 25, 2021
| @@ -93,7 +94,7 @@ func (c *Project) initLocalServer(ctx context.Context) (*grpc.ClientConn, error) | |||
| Timeout: 1 * time.Second, | |||
| }) | |||
| if err != nil { | |||
| return nil, err | |||
| return nil, fmt.Errorf("failed opening boltdb path %s. Is another server already running against this db?: %w", path, err) | |||
There was a problem hiding this comment.
This is mostly unrelated, but I spent a good long while trying to get to the bottom of why this change was breaking waypoint's local server mode. It turns out I wasn't initializing the plugin command with the WithNoAutoServer() option, so the plugin was trying to spool up another server, and timing out on opening bolt's data.db file. The only thing I had to go on from the logs was this:
! Unrecognized remote plugin message: ! failed to create client: timeout
So i'm enhancing that log message here.
Also, hurray for the integration test! I would not have caught this otherwise.
Hard-coded for pack
I'm not sure if this is the _right_ place to be parsing out unmanaged plugins though.
This manifested as a bug that broke local server mode. The plugin command would try to open the server's boltdb file, timeout, and crash. I'm also making the error message a bit friendlier here, for the next person to have this category of problem.
mitchellh
approved these changes
Jul 12, 2021
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
It is currently difficult to debug the builtin plugins, as waypoint itself launches plugin processes as it requires them. Debugging is easiest when you can use a debugger to launch the process of interest.
This experimental change allows Waypoint to launch builtin plugins in debug mode, and connect to and use an existing plugin processes. It is dependent on these waypoint-plugin-sdk changes: hashicorp/waypoint-plugin-sdk#39
These changes are inspired by (and in many cases copied from) Terraform's plugin debug mode. I've tried to keep the user facing side (flag/env var names, workflow, etc) as similar to Terraform as possible for consistency.
Debugging workflow
Step 1: Launch a plugin process of interest (via delve or goland debug) with the new debug flag (i.e.
waypoint plugin -debug pack). The plugin will launch, print reconnection info to the console (example below), then wait for core to attach.Step 2: Export that environment variable, then run a core command that will require that plugin (i.e.
waypoint build). When initializing plugins, core will unmarshall and inspect the contents of theWP_REATTACH_PLUGINSenv var. If it finds that a requested plugin is in that map, it will attempt to reconnect to it, rather than launching it itself.The flow is a little clunky for me - if you're iterating fast and relaunching the plugin process frequently, you have to do a lot of env var copying and pasting. It would be nice if the plugin could use a consistent socket path on every launch. I think this is good enough for now though, and consistency with Terraform is nice.
Documentation
My intent is to use this feature myself for this cycle, and document it next cycle once it's been hardened.