diff --git a/content/en/serverless/step_functions/distributed-maps.md b/content/en/serverless/step_functions/distributed-maps.md index 26bea22062a..904fcb5fc01 100644 --- a/content/en/serverless/step_functions/distributed-maps.md +++ b/content/en/serverless/step_functions/distributed-maps.md @@ -18,7 +18,7 @@ In AWS Step Functions, you can set up a large-scale parallel workload by includi "ItemBatcher": { "MaxItemsPerBatch": N, "BatchInput": { - "_datadog": "{% ($execInput := $states.context.Execution.Input; $hasDatadogTraceId := $exists($execInput._datadog.`x-datadog-trace-id`); $hasDatadogRootExecutionId := $exists($execInput._datadog.RootExecutionId); $ddTraceContext := $hasDatadogTraceId ? {'x-datadog-trace-id': $execInput._datadog.`x-datadog-trace-id`, 'x-datadog-tags': $execInput._datadog.`x-datadog-tags`} : {'RootExecutionId': $hasDatadogRootExecutionId ? $execInput._datadog.RootExecutionId : $states.context.Execution.Id}; $merge([$ddTraceContext, {'serverless-version': 'v1', 'timestamp': $millis()}])) %}" + "_datadog": "{% ($ddctx := ($states.context.**._datadog)[0];$maybeSnsCtx := ($parse($parse(($states.context.**.body)[0]).**._datadog.Value))[0];$ddctx := $exists($maybeSnsCtx) ? $maybeSnsCtx : $ddctx;$ddTraceContext := $exists($ddctx.`x-datadog-trace-id`) ? {'x-datadog-trace-id': $ddctx.`x-datadog-trace-id`, 'x-datadog-tags': $ddctx.`x-datadog-tags`} : {'RootExecutionId': $exists($ddctx.RootExecutionId) ? $ddctx.RootExecutionId : $states.context.Execution.Id};$merge([$ddTraceContext, {'serverless-version': 'v1', 'timestamp': $millis()}])) %}" } } {{< /highlight >}} diff --git a/content/en/serverless/step_functions/merge-step-functions-lambda.md b/content/en/serverless/step_functions/merge-step-functions-lambda.md index 1a597e1816d..73b53fc8f89 100644 --- a/content/en/serverless/step_functions/merge-step-functions-lambda.md +++ b/content/en/serverless/step_functions/merge-step-functions-lambda.md @@ -6,7 +6,7 @@ further_reading: text: 'Install Serverless Monitoring for AWS Step Functions' --- -This page describes how to merge your AWS Step Functions traces with related AWS Lambda traces or nested Step Functions traces. These instructions assume that you have already instrumented these [AWS Step Functions][1] and [Lambda functions][2] to send traces to Datadog. +This page describes how to merge your AWS Step Functions traces with related AWS Lambda traces and Step Functions traces. These instructions assume that you have already instrumented these [AWS Step Functions][1] and [Lambda functions][2] to send traces to Datadog.
Datadog recommends using JSONata to define your Step Function definitions for the most complete end-to-end tracing experience. This approach ensures that any context upstream to the Step Function is preserved and passed down.
@@ -26,7 +26,7 @@ Your State Machine definition must use `JSONata` as the query language. To enabl #### Setup -On the Lambda Task, set the `Payload` in the `Arguments` field as follows: +On the Lambda Task, set the `Payload` in the `Arguments` field as follows: {{< highlight json "hl_lines=7-7" >}} "Lambda Invoke": { @@ -35,7 +35,7 @@ On the Lambda Task, set the `Payload` in the `Arguments` field as follows: "Output": "{% $states.result.Payload %}", "Arguments": { "FunctionName": "MyFunctionName", - "Payload": "{% ($execInput := $exists($states.context.Execution.Input.BatchInput) ? $states.context.Execution.Input.BatchInput : $states.context.Execution.Input; $hasDatadogTraceId := $exists($execInput._datadog.`x-datadog-trace-id`); $hasDatadogRootExecutionId := $exists($execInput._datadog.RootExecutionId); $ddTraceContext := $hasDatadogTraceId ? {'x-datadog-trace-id': $execInput._datadog.`x-datadog-trace-id`, 'x-datadog-tags': $execInput._datadog.`x-datadog-tags`} : {'RootExecutionId': $hasDatadogRootExecutionId ? $execInput._datadog.RootExecutionId : $states.context.Execution.Id}; $sfnContext := $merge([$states.context, {'Execution': $sift($states.context.Execution, function($v, $k) { $k != 'Input' })}]); $merge([$states.input, {'_datadog': $merge([$sfnContext, $ddTraceContext, {'serverless-version': 'v1'}])}])) %}" + "Payload": "{% ($ddctx := ($states.context.**._datadog)[0]; $maybeSnsCtx := ($parse($parse(($states.context.**.body)[0]).**._datadog.Value))[0]; $ddctx := $exists($maybeSnsCtx) ? $maybeSnsCtx : $ddctx; $ddTraceContext := $exists($ddctx.`x-datadog-trace-id`) ? {'x-datadog-trace-id': $ddctx.`x-datadog-trace-id`, 'x-datadog-tags': $ddctx.`x-datadog-tags`} : {'RootExecutionId': $exists($ddctx.RootExecutionId) ? $ddctx.RootExecutionId : $states.context.Execution.Id}; $sfnContext := $merge([$states.context, {'Execution': $sift($states.context.Execution, function($v, $k) { $k != 'Input' })}]); $merge([$sfnContext, $ddTraceContext, {'serverless-version': 'v1'}])) %}" } } {{< /highlight >}} @@ -188,7 +188,7 @@ For additional code examples in TypeScript and Go, see [CDK Examples for Instrum {{% /tab %}} {{% tab "Custom" %}} -On the Lambda Task, set the `Parameters` key as follows: +On the Lambda Task, set the `Parameters` key as follows: ```json "Parameters": { @@ -240,7 +240,7 @@ Alternatively, if you have business logic defined in the payload, you can also u ### Through managed services -Follow these instructions if your Step Function indirectly invokes a Lambda through EventBridge, SQS, or SNS. To trace through another managed AWS service, [contact Datadog Support][3] to open a feature request. +Requests between Step Functions and Lambdas can be traced through many AWS managed services, including SNS, SQS, and EventBridge. To trace through another managed AWS service, [contact Datadog Support][3] to open a feature request. #### Requirements @@ -254,7 +254,7 @@ Merging Step Functions with Lambda traces through managed services is only suppo #### EventBridge -If an EventBridge rule has a Lambda function as a target, edit your EventBridge PutEvents Task to set `_datadog` in the `Detail` field as follows: +If an EventBridge rule has a Lambda function as a target, edit your EventBridge PutEvents Task to set `_datadog` in the `Detail` field as follows: {{< highlight json "hl_lines=10-10" >}} "EventBridge PutEvents": { @@ -266,7 +266,7 @@ If an EventBridge rule has a Lambda function as a target, edit your EventBridge "Detail": { "Message": "Hello from Step Functions!", "TaskToken": "{% $states.context.Task.Token %}", - "_datadog": "{% ($execInput := $exists($states.context.Execution.Input.BatchInput) ? $states.context.Execution.Input.BatchInput : $states.context.Execution.Input; $hasDatadogTraceId := $exists($execInput._datadog.`x-datadog-trace-id`); $hasDatadogRootExecutionId := $exists($execInput._datadog.RootExecutionId); $ddTraceContext := $hasDatadogTraceId ? {'x-datadog-trace-id': $execInput._datadog.`x-datadog-trace-id`, 'x-datadog-tags': $execInput._datadog.`x-datadog-tags`} : {'RootExecutionId': $hasDatadogRootExecutionId ? $execInput._datadog.RootExecutionId : $states.context.Execution.Id}; $sfnContext := $merge([$states.context, {'Execution': $sift($states.context.Execution, function($v, $k) { $k != 'Input' })}]); $merge([$sfnContext, $ddTraceContext, {'serverless-version': 'v1'}])) %}" + "_datadog": "{% ($ddctx := ($states.context.**._datadog)[0]; $maybeSnsCtx := ($parse($parse(($states.context.**.body)[0]).**._datadog.Value))[0]; $ddctx := $exists($maybeSnsCtx) ? $maybeSnsCtx : $ddctx; $ddTraceContext := $exists($ddctx.`x-datadog-trace-id`) ? {'x-datadog-trace-id': $ddctx.`x-datadog-trace-id`, 'x-datadog-tags': $ddctx.`x-datadog-tags`} : {'RootExecutionId': $exists($ddctx.RootExecutionId) ? $ddctx.RootExecutionId : $states.context.Execution.Id}; $sfnContext := $merge([$states.context, {'Execution': $sift($states.context.Execution, function($v, $k) { $k != 'Input' })}]); $merge([$sfnContext, $ddTraceContext, {'serverless-version': 'v1'}])) %}" }, "DetailType": "MyDetailType", "EventBusName": "MyEventBusName", @@ -279,7 +279,7 @@ If an EventBridge rule has a Lambda function as a target, edit your EventBridge #### SQS -If an SQS queue has a Lambda trigger, edit your SQS SendMessage Task to set `_datadog` in the `MessageAttributes` field as follows: +If an SQS queue has a Lambda trigger, edit your SQS SendMessage Task to set `_datadog` in the `MessageAttributes` field as follows: {{< highlight json "hl_lines=8-11" >}} "SQS SendMessage": { @@ -291,7 +291,7 @@ If an SQS queue has a Lambda trigger, edit your SQS SendMessage Task to set `_da "MessageAttributes": { "_datadog": { "DataType": "String", - "StringValue": "{% ($execInput := $exists($states.context.Execution.Input.BatchInput) ? $states.context.Execution.Input.BatchInput : $states.context.Execution.Input; $hasDatadogTraceId := $exists($execInput._datadog.`x-datadog-trace-id`); $hasDatadogRootExecutionId := $exists($execInput._datadog.RootExecutionId); $ddTraceContext := $hasDatadogTraceId ? {'x-datadog-trace-id': $execInput._datadog.`x-datadog-trace-id`, 'x-datadog-tags': $execInput._datadog.`x-datadog-tags`} : {'RootExecutionId': $hasDatadogRootExecutionId ? $execInput._datadog.RootExecutionId : $states.context.Execution.Id}; $sfnContext := $merge([$states.context, {'Execution': $sift($states.context.Execution, function($v, $k) { $k != 'Input' })}]); $merge([$sfnContext, $ddTraceContext, {'serverless-version': 'v1'}])) %}" + "StringValue": "{% ($ddctx := ($states.context.**._datadog)[0]; $maybeSnsCtx := ($parse($parse(($states.context.**.body)[0]).**._datadog.Value))[0]; $ddctx := $exists($maybeSnsCtx) ? $maybeSnsCtx : $ddctx; $ddTraceContext := $exists($ddctx.`x-datadog-trace-id`) ? {'x-datadog-trace-id': $ddctx.`x-datadog-trace-id`, 'x-datadog-tags': $ddctx.`x-datadog-tags`} : {'RootExecutionId': $exists($ddctx.RootExecutionId) ? $ddctx.RootExecutionId : $states.context.Execution.Id}; $sfnContext := $merge([$states.context, {'Execution': $sift($states.context.Execution, function($v, $k) { $k != 'Input' })}]); $merge([$sfnContext, $ddTraceContext, {'serverless-version': 'v1'}])) %}" } } } @@ -300,7 +300,7 @@ If an SQS queue has a Lambda trigger, edit your SQS SendMessage Task to set `_da #### SNS -If there is a Lambda subscription on the topic, edit the SNS Publish Task to set `_datadog` in the `MessageAttributes` field as follows: +If there is a Lambda subscription on the topic, edit the SNS Publish Task to set `_datadog` in the `MessageAttributes` field as follows: {{< highlight json "hl_lines=8-11" >}} "SNS Publish": { @@ -312,7 +312,7 @@ If there is a Lambda subscription on the topic, edit the SNS Publish Task to set "MessageAttributes": { "_datadog": { "DataType": "String", - "StringValue": "{% ($execInput := $exists($states.context.Execution.Input.BatchInput) ? $states.context.Execution.Input.BatchInput : $states.context.Execution.Input; $hasDatadogTraceId := $exists($execInput._datadog.`x-datadog-trace-id`); $hasDatadogRootExecutionId := $exists($execInput._datadog.RootExecutionId); $ddTraceContext := $hasDatadogTraceId ? {'x-datadog-trace-id': $execInput._datadog.`x-datadog-trace-id`, 'x-datadog-tags': $execInput._datadog.`x-datadog-tags`} : {'RootExecutionId': $hasDatadogRootExecutionId ? $execInput._datadog.RootExecutionId : $states.context.Execution.Id}; $sfnContext := $merge([$states.context, {'Execution': $sift($states.context.Execution, function($v, $k) { $k != 'Input' })}]); $merge([$sfnContext, $ddTraceContext, {'serverless-version': 'v1'}])) %}" + "StringValue": "{% ($ddctx := ($states.context.**._datadog)[0]; $maybeSnsCtx := ($parse($parse(($states.context.**.body)[0]).**._datadog.Value))[0]; $ddctx := $exists($maybeSnsCtx) ? $maybeSnsCtx : $ddctx; $ddTraceContext := $exists($ddctx.`x-datadog-trace-id`) ? {'x-datadog-trace-id': $ddctx.`x-datadog-trace-id`, 'x-datadog-tags': $ddctx.`x-datadog-tags`} : {'RootExecutionId': $exists($ddctx.RootExecutionId) ? $ddctx.RootExecutionId : $states.context.Execution.Id}; $sfnContext := $merge([$states.context, {'Execution': $sift($states.context.Execution, function($v, $k) { $k != 'Input' })}]); $merge([$sfnContext, $ddTraceContext, {'serverless-version': 'v1'}])) %}" } } } @@ -323,7 +323,7 @@ If there is a Lambda subscription on the topic, edit the SNS Publish Task to set ### With JSONata -Edit the Step Functions Task to set `_datadog` in the `Input` field as follows: +Edit the Step Functions Task to set `_datadog` in the `Input` field as follows: {{< highlight json "hl_lines=7-7" >}} "Step Functions StartExecution": { @@ -332,7 +332,7 @@ Edit the Step Functions Task to set `_datadog` in the `Input` field as follows: "Arguments": { "StateMachineArn": "arn:aws:states:::stateMachine:", "Input": { - "_datadog": "{% ($execInput := $exists($states.context.Execution.Input.BatchInput) ? $states.context.Execution.Input.BatchInput : $states.context.Execution.Input; $hasDatadogTraceId := $exists($execInput._datadog.`x-datadog-trace-id`); $hasDatadogRootExecutionId := $exists($execInput._datadog.RootExecutionId); $ddTraceContext := $hasDatadogTraceId ? {'x-datadog-trace-id': $execInput._datadog.`x-datadog-trace-id`, 'x-datadog-tags': $execInput._datadog.`x-datadog-tags`} : {'RootExecutionId': $hasDatadogRootExecutionId ? $execInput._datadog.RootExecutionId : $states.context.Execution.Id}; $sfnContext := $merge([$states.context, {'Execution': $sift($states.context.Execution, function($v, $k) { $k != 'Input' })}]); $merge([$sfnContext, $ddTraceContext, {'serverless-version': 'v1'}])) %}" + "_datadog": "{% ($ddctx := ($states.context.**._datadog)[0]; $maybeSnsCtx := ($parse($parse(($states.context.**.body)[0]).**._datadog.Value))[0]; $ddctx := $exists($maybeSnsCtx) ? $maybeSnsCtx : $ddctx; $ddTraceContext := $exists($ddctx.`x-datadog-trace-id`) ? {'x-datadog-trace-id': $ddctx.`x-datadog-trace-id`, 'x-datadog-tags': $ddctx.`x-datadog-tags`} : {'RootExecutionId': $exists($ddctx.RootExecutionId) ? $ddctx.RootExecutionId : $states.context.Execution.Id}; $sfnContext := $merge([$states.context, {'Execution': $sift($states.context.Execution, function($v, $k) { $k != 'Input' })}]); $merge([$sfnContext, $ddTraceContext, {'serverless-version': 'v1'}])) %}" } } } @@ -342,7 +342,7 @@ Edit the Step Functions Task to set `_datadog` in the `Input` field as follows: Configure your task according to the following example: -{{< highlight json "hl_lines=9-13" >}} +{{< highlight json "hl_lines=8-12" >}} "Step Functions StartExecution": { "Type": "Task", "Resource": "arn:aws:states:::states:startExecution", @@ -363,7 +363,7 @@ Configure your task according to the following example: ## Merge Lambda traces with downstream Step Functions traces -Follow these instructions if a Lambda function directly invokes a Step Function using `StartExecution` or `StartSyncExecution`. +Traces can be connected downstream from Lambda functions when called directly using `StartExecution` or `StartSyncExecution`, or when called indirectly through managed services like SNS, SQS, or EventBridge. ### Requirements | Runtime | Requirement | @@ -379,6 +379,36 @@ If the layer or tracer version requirements are fulfilled, no further setup is r
To ensure proper trace merging, provide input to the Step Functions Start Execution command, even if the input is an empty JSON object.
+## Merge Step Functions traces with other services + +For services not covered in this guide, you can merge traces by manually propagating [Datadog trace context][3]. + +### Upstream services to Step Functions + +Include trace context in the Step Function input payload as JSON under the `_datadog` key, which can be located anywhere in the payload. +- The context object must contain the `x-datadog-trace-id` and `x-datadog-parent-id` keys. +- The `x-datadog-tags` key is optional and is used to pass additional tags. + +{{< highlight json >}} +"_datadog": { + "x-datadog-trace-id": "280166049706551372", + "x-datadog-parent-id": "611647714644695775", + "x-datadog-tags": "_dd.p.tid=66bcb5eb00000000,_dd.p.dm=-0" +} +{{< /highlight >}} + +### Step Functions to downstream services + +Add Step Function execution context to your task definitions using the patterns described above in the [propagating traces to Lambda via Managed Services][4] section. Additional configuration of the downstream service may be required. + +For assistance with custom integrations, [contact Datadog Support][5]. + +## Further Reading + +{{< partial name="whats-next/whats-next.html" >}} + [1]: /serverless/step_functions/installation [2]: /serverless/aws_lambda/installation -[3]: /help \ No newline at end of file +[3]: /tracing/trace_collection/trace_context_propagation/?tab=java#datadog-format +[4]: #through-managed-services +[5]: /help