Skip to content

[vtadmin] GetWorkflow(s) endpoints#7662

Merged
rohit-nayak-ps merged 14 commits intovitessio:masterfrom
tinyspeck:am_vtadmin_getworkflows
Mar 16, 2021
Merged

[vtadmin] GetWorkflow(s) endpoints#7662
rohit-nayak-ps merged 14 commits intovitessio:masterfrom
tinyspeck:am_vtadmin_getworkflows

Conversation

@ajm188
Copy link
Copy Markdown
Contributor

@ajm188 ajm188 commented Mar 11, 2021

Description

This PR adds two endpoints to the vtadmin api - GetWorkflow, which returns a single Workflow for a given (cluster, keyspace, name); and GetWorkflows, which returns multiple workflows across multiple clusters according to a bunch of filtering criteria.

For GetWorkflows, you can pass the following options:

  • Keyspaces: list of keyspaces to restrict the workflow search to. this applies across all clusters, so for finer-grained controls (e.g. "keyspace 'foo' in cluster1, but ignore keyspace 'foo' in cluster2", you'll need to make two separate requests)
  • IgnoreKeyspaces: inverse of keyspaces, but is completely ignored if the Keyspaces option is non-empty. Same caveat about applying across all clusters as the Keyspaces mentioned above.

Related Issue(s)

Checklist

  • Should this PR be backported? no
  • Tests were added or are not required
  • Documentation was added or is not required

Deployment Notes

Impacted Areas in Vitess

Components that this PR will affect:

  • Query Serving
  • VReplication
  • Cluster Management
  • Build/CI
  • VTAdmin

Andrew Mason added 9 commits March 9, 2021 22:07
Signed-off-by: Andrew Mason <amason@slack-corp.com>
Signed-off-by: Andrew Mason <amason@slack-corp.com>
Signed-off-by: Andrew Mason <amason@slack-corp.com>
Signed-off-by: Andrew Mason <amason@slack-corp.com>
Signed-off-by: Andrew Mason <amason@slack-corp.com>
Signed-off-by: Andrew Mason <amason@slack-corp.com>
Signed-off-by: Andrew Mason <amason@slack-corp.com>
Signed-off-by: Andrew Mason <amason@slack-corp.com>
Signed-off-by: Andrew Mason <amason@slack-corp.com>
(I swear I know how the alphabet works .... maybe ......)

Signed-off-by: Andrew Mason <amason@slack-corp.com>
return c.findWorkflows(ctx, keyspaces, opts)
}

func (c *Cluster) findWorkflows(ctx context.Context, keyspaces []string, opts FindWorkflowsOptions) ([]*vtadminpb.Workflow, error) {
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to reviewers: this function is the bulk of the logic; all (non-error) codepaths end up calling this function.

}, nil
}

func (api *API) getClustersForRequest(ids []string) ([]*cluster.Cluster, []string) {
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just moving this to the bottom of the file; it felt a bit out of place being stuck between two public functions without really being a helper function for either of them in particular

@ajm188 ajm188 force-pushed the am_vtadmin_getworkflows branch 2 times, most recently from b776838 to f38ace3 Compare March 12, 2021 01:29
Andrew Mason added 4 commits March 12, 2021 08:02
Signed-off-by: Andrew Mason <amason@slack-corp.com>
… the param name

Signed-off-by: Andrew Mason <amason@slack-corp.com>
Signed-off-by: Andrew Mason <amason@slack-corp.com>
Signed-off-by: Andrew Mason <amason@slack-corp.com>
@ajm188 ajm188 force-pushed the am_vtadmin_getworkflows branch from f38ace3 to 7b9e498 Compare March 12, 2021 13:02
Copy link
Copy Markdown
Contributor

@doeg doeg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much!!! This is perfect.

Name: "workflow1",
},
expected: nil,
shouldErr: true,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious -- in tests I have been defining an expectedError property on tests, and then asserting on errors.Is. This seems like it offers more granularity than shouldErr, but maybe it's not possible to do in all cases?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thinking is generally that if I care about the actual class of error (which is probably only true if I add a doc comment along the lines of "this function returns an <error type> when <condition>") then I will do the expectedErr + errors.Is. If I just care that it returns some error, then I'll do this.

// Callers should use this function when they want more fine-grained filtering,
// and GetWorkflows when they just want to filter on keyspace name.
//
// Note that if only a subset of keyspaces error on their vtctld GetWorkflows
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can see what you mean about this being a little Complicated but FWIW it makes sense to me + works perfectly for the front-end. 😎

// Keyspaces is a list of keyspaces to restrict the workflow search to. Note
// that the keyspaces list applies across all cluster IDs in the request.
//
// If, for example, you have two clusters, each with a keyspace called "foo"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense + thank you for documenting it so clearly!

Comment on lines +50 to +54
// Its route is /workflows, with query params:
// - cluster: repeated, cluster IDs
// - active_only
// - keyspace: repeated
// - ignore_keyspace: repeated
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sometimes... I have gross thoughts about somehow... adding protos for our HTTP request shapes, since I end up redefining the types on the front-end anyway. Probably our time would be better spent on gRPC-on-the-front-end... 😭 ...unless...............

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(my vote is also for grpc-on-the-front-end, some day ................)


// AssertClusterWorkflowsEqual is a test helper for asserting that two
// ClusterWorkflows objects are equal.
func AssertClusterWorkflowsEqual(t *testing.T, expected *vtadminpb.ClusterWorkflows, actual *vtadminpb.ClusterWorkflows, msgAndArgs ...interface{}) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lolsob

Copy link
Copy Markdown
Member

@rohit-nayak-ps rohit-nayak-ps left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@rohit-nayak-ps rohit-nayak-ps merged commit deaf818 into vitessio:master Mar 16, 2021
@askdba askdba added the Component: VTAdmin VTadmin interface label Mar 18, 2021
@askdba askdba added this to the v10.0 milestone Mar 18, 2021
@ajm188 ajm188 deleted the am_vtadmin_getworkflows branch May 29, 2021 10:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants