-
Notifications
You must be signed in to change notification settings - Fork 187
Feature workflow history propagation #823
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
7e0a345
example for history propagation
cicoyle b4440bb
update user experience for a better look/feel
cicoyle b9061a9
add k8s for signing ex
cicoyle 2b539ed
Merge branch 'main' into feature/wf-ctx-propagation
cicoyle 77fdbed
fix go.sum
cicoyle ec4a0fb
PR feedback
cicoyle d051ecc
bump versions
cicoyle 1022d1b
merge in master and fix conflicts
cicoyle 4bb6082
add mechanical markdown
cicoyle File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| # The binary is expected to be built on the host and copied in. | ||
| # Build this from examples/workflow-history-propagation, where examples/go.mod | ||
| # provides the local durabletask-go / go-sdk paths via replace directives. | ||
| # Build with: | ||
| # CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -o payment-app . | ||
| # (use GOARCH=amd64 on Intel machines) | ||
| FROM gcr.io/distroless/base-debian12 | ||
| COPY payment-app /payment-app | ||
| ENTRYPOINT ["/payment-app"] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,237 @@ | ||
| # Dapr Workflow History Propagation Example | ||
|
|
||
| This example demonstrates how workflows can propagate their execution history | ||
| to child workflows and activities, enabling downstream consumers to inspect | ||
| the full (or partial) execution context of their caller. | ||
|
|
||
| ## Workflow Architecture | ||
|
|
||
| ``` | ||
| MerchantCheckout (workflow) | ||
| ├── ValidateMerchant (activity, no propagation) | ||
| └── ProcessPayment (child workflow, PropagateLineage) | ||
| ├── ValidateCard (activity, no propagation) | ||
| ├── CheckSpendingLimits (activity, no propagation) | ||
| ├── FraudDetection (child workflow, PropagateLineage) | ||
| │ → sees 15 events: MerchantCheckout + ProcessPayment | ||
| └── SettlePayment (activity, PropagateOwnHistory) | ||
| → sees 12 events: ProcessPayment only | ||
| ``` | ||
|
|
||
| ### Propagation Scope | ||
|
|
||
| | Mode | What it sends | Use case | | ||
| |------|--------------|----------| | ||
| | `PropagateLineage()` | Caller's own events + any ancestor events it received | Full chain-of-custody verification | | ||
| | `PropagateOwnHistory()` | Caller's own events only (no ancestor chain) | Trust boundary — downstream only sees the immediate caller | | ||
|
|
||
| ### Key Demonstration | ||
|
|
||
| - **FraudDetection** receives 15 events via `PropagateLineage()` — it can verify | ||
| that `ValidateMerchant` ran in the top-level/grandparent workflow (MerchantCheckout), | ||
| plus `ValidateCard` and `CheckSpendingLimits` ran in ProcessPayment. | ||
|
|
||
| - **SettlePayment** receives 12 events via `PropagateOwnHistory()` — it only sees | ||
| ProcessPayment's events. The MerchantCheckout ancestral history is excluded. | ||
|
|
||
| ## Running the Example | ||
|
|
||
| Propagation works in either of two modes: | ||
|
|
||
| - **Standalone / `dapr run`** — quickest path. Propagation runs end-to-end with | ||
| no mTLS or signing setup. Each propagating dispatch logs a warning so | ||
| operators are aware the chunks are unsigned and can't be cryptographically | ||
| verified by receivers. | ||
| - **Kubernetes with mTLS + WorkflowHistorySigning** — production-grade path. | ||
| Chunks travel between sidecars over mTLS and the workflow's own history is | ||
| signed. No warning logs are emitted. | ||
|
|
||
| ### Option A: Standalone (`dapr run`) | ||
|
|
||
| First-time setup: run `dapr init` if you haven't already. The commands below assume | ||
| you're in `examples/workflow-history-propagation`. | ||
|
|
||
| Build the app binary: | ||
|
|
||
| <!-- STEP | ||
| name: Build payment-app | ||
| timeout_seconds: 300 | ||
| --> | ||
|
|
||
| ```bash | ||
| go build -o payment-app . | ||
| ``` | ||
|
|
||
| <!-- END_STEP --> | ||
|
|
||
| Run it under Dapr — propagation runs end-to-end: | ||
|
|
||
| <!-- STEP | ||
| name: Run history propagation demo | ||
| output_match_mode: substring | ||
| expected_stdout_lines: | ||
| - 'WORKFLOW HISTORY PROPAGATION DEMO' | ||
| - '[MerchantCheckout] Starting checkout' | ||
| - '[ValidateMerchant] Validating merchant' | ||
| - '[ProcessPayment] Starting payment' | ||
| - 'events (scope: LINEAGE)' | ||
| - '[FraudDetection] APPROVED' | ||
| - 'scope=OWN_HISTORY' | ||
| - '[SettlePayment] SETTLED' | ||
| - '= COMPLETE =' | ||
| background: true | ||
| sleep: 30 | ||
| timeout_seconds: 90 | ||
| --> | ||
|
|
||
| ```bash | ||
| dapr run --app-id payment-app --resources-path config -- ./payment-app | ||
| ``` | ||
|
|
||
| <!-- END_STEP --> | ||
|
|
||
| Note: build the binary and run it directly rather than `go run .` to ensure Ctrl+C | ||
| properly allows `dapr run` to exit. | ||
|
|
||
| You'll see lines like: | ||
|
|
||
| ``` | ||
| [FraudDetection] Received propagated history: 15 events (scope: LINEAGE) | ||
| ... level=warning msg="propagating unsigned workflow history to ..." | ||
| ``` | ||
|
|
||
| The warnings are expected — they're telling you that without | ||
| `WorkflowHistorySigning` enabled, the chunks aren't signed. | ||
|
|
||
| ### Option B: Kubernetes with mTLS + signing | ||
|
|
||
| This path adds Sentry-issued mTLS for sidecar-to-sidecar traffic and turns on | ||
| `WorkflowHistorySigning` so propagated chunks travel within a signed | ||
| trust boundary. | ||
|
|
||
| #### Prerequisites | ||
|
|
||
| - A running Kubernetes cluster with `kubectl` context set (eg `kind`). | ||
| - `docker` to build the example app image. | ||
| - `dapr` CLI on your PATH. | ||
|
|
||
| #### 1. Install Dapr into the cluster with mTLS on | ||
|
|
||
| Use the latest RC that includes workflow history propagation. mTLS is on by | ||
| default for Helm/`dapr init -k` installs. | ||
|
|
||
| ```bash | ||
| dapr init -k --runtime-version=1.18.0 # 1.18.0+ | ||
| ``` | ||
|
|
||
| Wait for the dapr-system pods to be Running: | ||
|
|
||
| ```bash | ||
| kubectl get pods -n dapr-system | ||
| ``` | ||
|
|
||
| Expected: `dapr-sentry`, `dapr-placement-server-0`, `dapr-scheduler-server-0`, | ||
| `dapr-operator`, `dapr-sidecar-injector`. | ||
|
|
||
| #### 2. Build and load the example app image | ||
|
|
||
| From this example's directory: | ||
|
|
||
| ```bash | ||
| cd examples/workflow-history-propagation | ||
|
|
||
| CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -o payment-app . | ||
|
|
||
| docker build -t payment-app:dev . | ||
| kind load docker-image payment-app:dev # kind clusters only; otherwise push to your registry | ||
| ``` | ||
|
|
||
| #### 3. Deploy the example | ||
|
|
||
| ```bash | ||
| kubectl apply -f k8s/ | ||
| ``` | ||
|
|
||
| This applies four manifests: | ||
|
|
||
| - `redis-deploy.yaml` — Redis Deployment + Service (`payment-app-redis`) | ||
| - `wf-store.yaml` — Dapr `state.redis` Component pointing at `payment-app-redis:6379` | ||
| - `signing-config.yaml` — Configuration CRD enabling `WorkflowHistorySigning` | ||
| - `app-deploy.yaml` — `payment-app` Deployment with Dapr sidecar annotations | ||
|
|
||
| #### 4. Watch the output | ||
|
|
||
| ```bash | ||
| kubectl logs -l app=payment-app -c payment-app -f | ||
| ``` | ||
|
|
||
| Expected substrings (all should appear): | ||
|
|
||
| ``` | ||
| WORKFLOW HISTORY PROPAGATION DEMO | ||
| [MerchantCheckout] Starting checkout for merchant | ||
| [ValidateMerchant] Validating merchant | ||
| [ProcessPayment] Starting payment | ||
| [FraudDetection] Received propagated history: 15 events (scope: LINEAGE) | ||
| [FraudDetection] Verification: ValidateMerchant=true, ValidateCard=true, CheckSpendingLimits=true | ||
| [FraudDetection] APPROVED | ||
| [SettlePayment] Settling | ||
| propagated history: 12 events, scope=OWN_HISTORY | ||
| [SettlePayment] SETTLED | ||
| COMPLETE | ||
| ``` | ||
|
|
||
| In this mode the daprd sidecar should NOT log | ||
| `propagating unsigned workflow history` — the chunks are signed because | ||
| `WorkflowHistorySigning` is on. | ||
|
|
||
| ### Troubleshooting | ||
|
|
||
| - **FraudDetection reports 0 events** — the workflow code didn't request | ||
| propagation. Confirm the parent calls `CallChildWorkflow` / | ||
| `CallActivity` with `workflow.WithHistoryPropagation(...)`. | ||
| - **`propagating unsigned workflow history` warnings in standalone** — | ||
| expected; switch to the Kubernetes path above (or enable Sentry + | ||
| `WorkflowHistorySigning`) if you want signed chunks. | ||
| - **K8s: Sentry connection errors** — check `kubectl get pods -n dapr-system` | ||
| and re-deploy the control plane. | ||
|
|
||
| ### Cleanup | ||
|
|
||
| Tear down the example resources (`payment-app` Deployment, `payment-app-redis` | ||
| Deployment + Service, `wf-store` Component, `signing` Configuration): | ||
|
|
||
| ```bash | ||
| kubectl delete -f k8s/ | ||
| ``` | ||
|
|
||
| Confirm nothing is left over: | ||
|
|
||
| ```bash | ||
| kubectl get deploy,svc,po,component,configuration -l app=payment-app | ||
| kubectl get deploy,svc,po -l app=payment-app-redis | ||
| ``` | ||
|
|
||
| Both should return `No resources found`. | ||
|
|
||
| To uninstall the Dapr control plane (matching `dapr init -k`): | ||
|
|
||
| ```bash | ||
| dapr uninstall -k --all # removes dapr-system namespace + CRDs | ||
| ``` | ||
|
|
||
| ## Files | ||
|
|
||
| ``` | ||
| workflow-history-propagation/ | ||
| ├── README.md # this file | ||
| ├── main.go # workflow + activity definitions | ||
| ├── config/ | ||
| │ └── redis.yaml # local self-hosted Dapr Component (dapr run) | ||
| ├── Dockerfile # packages the pre-built payment-app binary | ||
| └── k8s/ | ||
| ├── signing-config.yaml # Configuration CRD enabling WorkflowHistorySigning | ||
| ├── wf-store.yaml # Dapr Component (state.redis, actorStateStore) | ||
| ├── redis-deploy.yaml # Redis Deployment + Service (payment-app-redis) | ||
| └── app-deploy.yaml # payment-app Deployment with dapr sidecar annotations | ||
| ``` | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| apiVersion: dapr.io/v1alpha1 | ||
| kind: Component | ||
| metadata: | ||
| name: wf-store | ||
| spec: | ||
| type: state.redis | ||
| version: v1 | ||
| metadata: | ||
| - name: redisHost | ||
| value: localhost:6379 | ||
| - name: redisPassword | ||
| value: "" | ||
| - name: actorStateStore | ||
| value: "true" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| apiVersion: apps/v1 | ||
| kind: Deployment | ||
| metadata: | ||
| name: payment-app | ||
| namespace: default | ||
|
cicoyle marked this conversation as resolved.
|
||
| labels: | ||
| app: payment-app | ||
| spec: | ||
| replicas: 1 | ||
| selector: | ||
| matchLabels: | ||
| app: payment-app | ||
| template: | ||
| metadata: | ||
| labels: | ||
| app: payment-app | ||
| annotations: | ||
| dapr.io/enabled: "true" | ||
| dapr.io/app-id: "payment-app" | ||
| dapr.io/config: "signing" | ||
| dapr.io/log-level: "info" | ||
| spec: | ||
| containers: | ||
| - name: payment-app | ||
| image: payment-app:dev | ||
| imagePullPolicy: IfNotPresent | ||
36 changes: 36 additions & 0 deletions
36
examples/workflow-history-propagation/k8s/redis-deploy.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| apiVersion: apps/v1 | ||
| kind: Deployment | ||
| metadata: | ||
| name: payment-app-redis | ||
| namespace: default | ||
|
cicoyle marked this conversation as resolved.
|
||
| labels: | ||
| app: payment-app-redis | ||
| spec: | ||
| replicas: 1 | ||
| selector: | ||
| matchLabels: | ||
| app: payment-app-redis | ||
| template: | ||
| metadata: | ||
| labels: | ||
| app: payment-app-redis | ||
| spec: | ||
| containers: | ||
| - name: redis | ||
| image: redis:7-alpine | ||
| ports: | ||
| - containerPort: 6379 | ||
| --- | ||
| apiVersion: v1 | ||
| kind: Service | ||
| metadata: | ||
| name: payment-app-redis | ||
| namespace: default | ||
|
cicoyle marked this conversation as resolved.
|
||
| labels: | ||
| app: payment-app-redis | ||
| spec: | ||
| selector: | ||
| app: payment-app-redis | ||
| ports: | ||
| - port: 6379 | ||
| targetPort: 6379 | ||
11 changes: 11 additions & 0 deletions
11
examples/workflow-history-propagation/k8s/signing-config.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| apiVersion: dapr.io/v1alpha1 | ||
| kind: Configuration | ||
| metadata: | ||
| name: signing | ||
| namespace: default | ||
|
cicoyle marked this conversation as resolved.
|
||
| labels: | ||
| app: payment-app | ||
| spec: | ||
| features: | ||
| - name: WorkflowHistorySigning | ||
| enabled: true | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.