-
Notifications
You must be signed in to change notification settings - Fork 39
Adds RPCs for workflow list and get history #93
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| # Workflow List and History RPCs for Durable Task | ||
|
|
||
| * Author(s): @joshvanl | ||
|
|
||
| ## Overview | ||
|
|
||
| This proposal adds two new RPCs to the durabletask framework which are used to support observability and discoverability of running and completed workflows in Dapr. | ||
| Specifically, adding a `ListInstances` and `GetInstanceHistory` RPC to the durabletask gRPC service. | ||
|
|
||
| ## Background | ||
|
|
||
| Today, there is no ways of discovering the list of workflow instances that are currently running or have completed in the past without using external storage queries. | ||
| The Dapr CLI [introduced list and workflow history commands](https://github.com/dapr/cli/pull/1560) to get information about running and completed workflows, however these commands rely on direct queries to the underlying storage provider. | ||
| By introducing this functionality into the durabletask framework itself, these commands need only talk to Daprd, removing the requirement for direct access to the storage provider as well as authentication. | ||
| Daprd can make these queries itself, and use the Actor State Store component to access the underlying storage. | ||
|
|
||
| ## Related Items | ||
|
|
||
| ## Implementation Details | ||
|
|
||
| ### Design | ||
|
|
||
| Two new RPCs will be added to the durabletask gRPC service, which will be available to both the application client, as well as the dapr CLI. | ||
| The list RPC will be used to discover the workflow instance IDs, which their metadatda can then be fetched. | ||
| The workflow history RPC will be used to fetch the full history of a given workflow instance. | ||
|
|
||
| ```proto | ||
| service TaskHubSidecarService { | ||
| rpc ListInstances (ListInstancesRequest) returns (ListInstancesResponse); | ||
| rpc GetInstanceHistory (GetInstanceHistoryRequest) returns (GetInstanceHistoryResponse); | ||
| } | ||
|
|
||
| // ListInstancesRequest is used to list all orchestration instances. | ||
| message ListInstancesRequest { | ||
| // continuationToken is the continuation token to use for pagination. This | ||
| // is the index which the next page should start from. If not given, the first | ||
| // page will be returned. | ||
| optional uint32 continuationToken = 1; | ||
|
|
||
| // pageSize is the maximum number of instances to return for this page. If | ||
| // not given, all instances will be attempted to be returned. | ||
| optional uint32 pageSize = 2; | ||
| } | ||
|
|
||
| // ListInstancesResponse is the response to executing ListInstances. | ||
| message ListInstancesResponse { | ||
| // instanceIds is the list of instance IDs returned. | ||
| repeated string instanceIds = 1; | ||
| } | ||
|
||
|
|
||
| // GetInstanceHistoryRequest is used to get the full history of an | ||
| // orchestration instance. | ||
| message GetInstanceHistoryRequest { | ||
| string instanceId = 1; | ||
| } | ||
|
|
||
| // GetInstanceHistoryResponse is the response to executing GetInstanceHistory. | ||
| message GetInstanceHistoryResponse { | ||
| repeated HistoryEvent events = 1; | ||
| } | ||
| ``` | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it'd be safer to set a maximum allowed value.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What would you consider safe in this context?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
100? completely arbitrary, but seems large enough to be useful and small enough to not stress the system.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about 1024? 😄
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you think ~1000 items are manageable? It feels like a bit too much for me. Github API normally limits to 100 per page, with a default of 30. But their payloads are big. What do you suggest? No limit and just return them all?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I should think so- we are only returning the key strings, no the value payloads.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In that case I think we can keep it high, fine with the 1000 if you feel it's fine, but I'd enforce a limit just to protect us from situations where there are just too many keys to return.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should it not be the case that the user should enforce that limit? They have the controls with the page size and continuation token.