diff --git a/README.md b/README.md index 58d3901dc..e771db05c 100644 --- a/README.md +++ b/README.md @@ -26,7 +26,7 @@ Powertools for AWS Lambda (Java) is available in Maven Central. You can use your software.amazon.lambda - powertools-logging + powertools-logging-log4j 2.6.0 @@ -116,6 +116,7 @@ Next, configure the aspectj-maven-plugin to compile-time weave (CTW) the aws-lam aspect 'software.amazon.lambda:powertools-logging:{{ powertools.version }}' aspect 'software.amazon.lambda:powertools-tracing:{{ powertools.version }}' aspect 'software.amazon.lambda:powertools-metrics:{{ powertools.version }}' + implementation 'software.amazon.lambda:powertools-logging-log4j:{{ powertools.version }}' implementation "org.aspectj:aspectjrt:1.9.22" } @@ -126,10 +127,10 @@ Next, configure the aspectj-maven-plugin to compile-time weave (CTW) the aws-lam ### Java Compatibility -Powertools for AWS Lambda (Java) supports all Java version from 11 up to 21 as well as the -[corresponding Lambda runtimes](https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html). +Powertools for AWS Lambda (Java) supports all Java versions from 11 to 25 in line with the [corresponding Lambda runtimes](https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html). + For the modules that provide annotations, Powertools for AWS Lambda (Java) leverages the **aspectj** library. -You may need to add the good version of `aspectjrt` to your dependencies based on the JDK used for building your function: +You may need to add the appropriate version of `aspectjrt` to your dependencies based on the JDK used for building your function: ```xml @@ -142,12 +143,13 @@ You may need to add the good version of `aspectjrt` to your dependencies based o
JDK - aspectj dependency matrix +Use the following [dependency matrix](https://github.com/eclipse-aspectj/aspectj/blob/master/docs/release/JavaVersionCompatibility.adoc) to understand which AspectJ version to use based on your JDK version: + | JDK version | aspectj version | |-------------|------------------------| | `11-17` | `1.9.20.1` (or higher) | | `21` | `1.9.21` (or higher) | - -More info [here](https://github.com/aws-powertools/powertools-lambda-java/pull/1519/files#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R191). +| `25` | `1.9.25` (or higher) |
diff --git a/docs/FAQs.md b/docs/FAQs.md index 75f699c91..cea4b774f 100644 --- a/docs/FAQs.md +++ b/docs/FAQs.md @@ -7,6 +7,8 @@ description: Frequently Asked Questions Many utilities in this library use `aspectj-maven-plugin` to compile-time weave (CTW) aspects into the project. In case you want to use `Lombok` or other compile-time preprocessor for your project, it is required to change `aspectj-maven-plugin` configuration to enable in-place weaving feature. Otherwise the plugin will ignore changes introduced by `Lombok` and will use `.java` files as a source. +Alternatively, you can use the [functional approach](./usage-patterns.md#functional-approach) which does not require AspectJ configuration. + To enable in-place weaving feature you need to use following `aspectj-maven-plugin` configuration: ```xml hl_lines="2-6" @@ -31,6 +33,8 @@ To enable in-place weaving feature you need to use following `aspectj-maven-plug Many utilities use `aspectj-maven-plugin` to compile-time weave (CTW) aspects into the project. When using it with Kotlin projects, it is required to `forceAjcCompile`. No explicit configuration should be required for gradle projects. +Alternatively, you can use the [functional approach](./usage-patterns.md#functional-approach) which does not require AspectJ configuration. + To enable `forceAjcCompile` you need to use following `aspectj-maven-plugin` configuration: ```xml hl_lines="2" diff --git a/docs/core/logging.md b/docs/core/logging.md index db01a3ec0..8358087d2 100644 --- a/docs/core/logging.md +++ b/docs/core/logging.md @@ -23,13 +23,12 @@ Logging provides an opinionated logger with output structured as JSON. You can find complete examples in the [project repository](https://github.com/aws-powertools/powertools-lambda-java/tree/v2/examples/powertools-examples-core-utilities){target="_blank"}. ### Installation -Depending on preference, you must choose to use either _log4j2_ or _logback_ as your log provider. In both cases you need to configure _aspectj_ -to weave the code and make sure the annotation is processed. +Depending on preference, you must choose to use either _log4j2_ or _logback_ as your log provider. If you use the AspectJ annotation approach, you must configure _aspectj_ to weave the code and make sure the annotation is processed. If you prefer the [functional approach](../usage-patterns.md#functional-approach), AspectJ configuration is not required. #### Maven === "log4j2" - ```xml hl_lines="3-7 24-27" + ```xml hl_lines="3-12 30-33" ... @@ -37,10 +36,16 @@ to weave the code and make sure the annotation is processed. powertools-logging-log4j {{ powertools.version }} + + software.amazon.lambda + powertools-logging + {{ powertools.version }} + ... ... + ... @@ -82,7 +87,7 @@ to weave the code and make sure the annotation is processed. === "logback" - ```xml hl_lines="3-7 24-27" + ```xml hl_lines="3-12 30-33" ... @@ -90,10 +95,16 @@ to weave the code and make sure the annotation is processed. powertools-logging-logback {{ powertools.version }} + + software.amazon.lambda + powertools-logging + {{ powertools.version }} + ... ... + ... @@ -137,10 +148,10 @@ to weave the code and make sure the annotation is processed. === "log4j2" - ```groovy hl_lines="3 11" + ```groovy hl_lines="3 11-12" plugins { id 'java' - id 'io.freefair.aspectj.post-compile-weaving' version '8.1.0' + id 'io.freefair.aspectj.post-compile-weaving' version '8.1.0' // Not needed when using the functional approach } repositories { @@ -148,7 +159,8 @@ to weave the code and make sure the annotation is processed. } dependencies { - aspect 'software.amazon.lambda:powertools-logging-log4j:{{ powertools.version }}' + aspect 'software.amazon.lambda:powertools-logging:{{ powertools.version }}' // Not needed when using the functional approach + implementation 'software.amazon.lambda:powertools-logging-log4j:{{ powertools.version }}' } sourceCompatibility = 11 @@ -157,10 +169,10 @@ to weave the code and make sure the annotation is processed. === "logback" - ```groovy hl_lines="3 11" + ```groovy hl_lines="3 11-12" plugins { id 'java' - id 'io.freefair.aspectj.post-compile-weaving' version '8.1.0' + id 'io.freefair.aspectj.post-compile-weaving' version '8.1.0' // Not needed when using the functional approach } repositories { @@ -168,7 +180,8 @@ to weave the code and make sure the annotation is processed. } dependencies { - aspect 'software.amazon.lambda:powertools-logging-logback:{{ powertools.version }}' + aspect 'software.amazon.lambda:powertools-logging:{{ powertools.version }}' // Not needed when using the functional approach + implementation 'software.amazon.lambda:powertools-logging-logback:{{ powertools.version }}' } sourceCompatibility = 11 @@ -317,9 +330,9 @@ If you set `POWERTOOLS_LOG_LEVEL` lower than ALC, we will emit a warning informi ## Basic Usage -To use Lambda Powertools for AWS Lambda Logging, use the `@Logging` annotation in your code and the standard _SLF4J_ logger: +You can use Powertools for AWS Lambda Logging with either the `@Logging` annotation or the functional API: -=== "PaymentFunction.java" +=== "@Logging annotation" ```java hl_lines="8 10 12 14" import org.slf4j.Logger; @@ -341,6 +354,30 @@ To use Lambda Powertools for AWS Lambda Logging, use the `@Logging` annotation i } ``` +=== "Functional API" + + ```java hl_lines="8 11 12 14 17" + import org.slf4j.Logger; + import org.slf4j.LoggerFactory; + import software.amazon.lambda.powertools.logging.PowertoolsLogging; + // ... other imports + + public class PaymentFunction implements RequestHandler { + + private static final Logger LOGGER = LoggerFactory.getLogger(PaymentFunction.class); + + public APIGatewayProxyResponseEvent handleRequest(final APIGatewayProxyRequestEvent input, final Context context) { + return PowertoolsLogging.withLogging(context, () -> { + LOGGER.info("Collecting payment"); + // ... + LOGGER.debug("order={}, amount={}", order.getId(), order.getAmount()); + // ... + return new APIGatewayProxyResponseEvent().withStatusCode(200); + }); + } + } + ``` + ## Standard structured keys Your logs will always include the following keys in your structured logging: @@ -376,11 +413,10 @@ The following keys will also be added to all your structured logs (unless [confi #### Logging a correlation ID -You can set a correlation ID using the `correlationIdPath` attribute of the `@Logging`annotation, -by passing a [JMESPath expression](https://jmespath.org/tutorial.html){target="_blank"}, +You can set a correlation ID using the `correlationIdPath` parameter by passing a [JMESPath expression](https://jmespath.org/tutorial.html){target="_blank"}, including our custom [JMESPath Functions](../utilities/serialization.md#built-in-functions). -=== "AppCorrelationIdPath.java" +=== "@Logging annotation" ```java hl_lines="5" public class AppCorrelationIdPath implements RequestHandler { @@ -395,6 +431,24 @@ including our custom [JMESPath Functions](../utilities/serialization.md#built-in } } ``` + +=== "Functional API" + + ```java hl_lines="6" + public class AppCorrelationIdPath implements RequestHandler { + + private static final Logger LOGGER = LoggerFactory.getLogger(AppCorrelationIdPath.class); + + public APIGatewayProxyResponseEvent handleRequest(final APIGatewayProxyRequestEvent input, final Context context) { + return PowertoolsLogging.withLogging(context, "headers.my_request_id_header", input, () -> { + // ... + LOGGER.info("Collecting payment"); + // ... + return new APIGatewayProxyResponseEvent().withStatusCode(200); + }); + } + } + ``` === "Example HTTP Event" ```json hl_lines="3" @@ -422,7 +476,7 @@ including our custom [JMESPath Functions](../utilities/serialization.md#built-in To ease routine tasks like extracting correlation ID from popular event sources, we provide [built-in JMESPath expressions](#built-in-correlation-id-expressions). -=== "AppCorrelationId.java" +=== "@Logging annotation" ```java hl_lines="1 7" import software.amazon.lambda.powertools.logging.CorrelationIdPaths; @@ -440,6 +494,26 @@ we provide [built-in JMESPath expressions](#built-in-correlation-id-expressions) } ``` +=== "Functional API" + + ```java hl_lines="1 8" + import software.amazon.lambda.powertools.logging.CorrelationIdPaths; + + public class AppCorrelationId implements RequestHandler { + + private static final Logger LOGGER = LoggerFactory.getLogger(AppCorrelationId.class); + + public APIGatewayProxyResponseEvent handleRequest(final APIGatewayProxyRequestEvent input, final Context context) { + return PowertoolsLogging.withLogging(context, CorrelationIdPaths.API_GATEWAY_REST, input, () -> { + // ... + LOGGER.info("Collecting payment"); + // ... + return new APIGatewayProxyResponseEvent().withStatusCode(200); + }); + } + } + ``` + === "Example Event" ```json hl_lines="3" @@ -668,10 +742,9 @@ You can remove additional keys added with the MDC using `MDC.remove("key")`. #### Clearing state Logger is commonly initialized in the global scope. Due to [Lambda Execution Context reuse](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-context.html){target="_blank"}, -this means that custom keys, added with the MDC can be persisted across invocations. If you want all custom keys to be deleted, you can use -`clearState=true` attribute on the `@Logging` annotation. +this means that custom keys, added with the MDC can be persisted across invocations. You can clear state using `clearState=true` on the `@Logging` annotation, or use the functional API which handles cleanup automatically. -=== "CreditCardFunction.java" +=== "@Logging annotation" ```java hl_lines="5 8" public class CreditCardFunction implements RequestHandler { @@ -716,15 +789,18 @@ this means that custom keys, added with the MDC can be persisted across invocati `clearState` is based on `MDC.clear()`. State clearing is automatically done at the end of the execution of the handler if set to `true`. +???+ tip + When using the functional API with `PowertoolsLogging.withLogging()`, state is automatically cleared at the end of execution, so you don't need to manage it manually. + ## Logging incoming event -When debugging in non-production environments, you can instruct the `@Logging` annotation to log the incoming event with `logEvent` param or via `POWERTOOLS_LOGGER_LOG_EVENT` env var. +When debugging in non-production environments, you can log the incoming event using the `@Logging` annotation with the `logEvent` parameter, via the `POWERTOOLS_LOGGER_LOG_EVENT` environment variable, or manually with the functional API. ???+ warning - This is disabled by default to prevent sensitive info being logged + This is disabled by default to prevent sensitive info being logged. -=== "AppLogEvent.java" +=== "@Logging annotation" ```java hl_lines="5" public class AppLogEvent implements RequestHandler { @@ -738,17 +814,36 @@ When debugging in non-production environments, you can instruct the `@Logging` a } ``` +=== "Functional API" + + ```java hl_lines="1 9" + import static software.amazon.lambda.powertools.logging.argument.StructuredArguments.entry; + + public class AppLogEvent implements RequestHandler { + + private static final Logger LOGGER = LoggerFactory.getLogger(AppLogEvent.class); + + public APIGatewayProxyResponseEvent handleRequest(final APIGatewayProxyRequestEvent input, final Context context) { + return PowertoolsLogging.withLogging(context, () -> { + LOGGER.info("Handler Event", entry("event", input)); + // ... + return new APIGatewayProxyResponseEvent().withStatusCode(200); + }); + } + } + ``` + ???+ note - If you use this on a RequestStreamHandler, the SDK must duplicate input streams in order to log them. + If you use this on a RequestStreamHandler, the SDK must duplicate input streams in order to log them when used together with the `@Logging` annotation. ## Logging handler response -When debugging in non-production environments, you can instruct the `@Logging` annotation to log the response with `logResponse` param or via `POWERTOOLS_LOGGER_LOG_RESPONSE` env var. +When debugging in non-production environments, you can log the response using the `@Logging` annotation with the `logResponse` parameter, via the `POWERTOOLS_LOGGER_LOG_RESPONSE` environment variable, or manually with the functional API. ???+ warning - This is disabled by default to prevent sensitive info being logged + This is disabled by default to prevent sensitive info being logged. -=== "AppLogResponse.java" +=== "@Logging annotation" ```java hl_lines="5" public class AppLogResponse implements RequestHandler { @@ -762,18 +857,41 @@ When debugging in non-production environments, you can instruct the `@Logging` a } ``` +=== "Functional API" + + ```java hl_lines="1 11" + import static software.amazon.lambda.powertools.logging.argument.StructuredArguments.entry; + + public class AppLogResponse implements RequestHandler { + + private static final Logger LOGGER = LoggerFactory.getLogger(AppLogResponse.class); + + public APIGatewayProxyResponseEvent handleRequest(final APIGatewayProxyRequestEvent input, final Context context) { + return PowertoolsLogging.withLogging(context, () -> { + // ... + APIGatewayProxyResponseEvent response = new APIGatewayProxyResponseEvent().withStatusCode(200); + LOGGER.info("Handler Response", entry("response", response)); + return response; + }); + } + } + ``` + ???+ note - If you use this on a RequestStreamHandler, Powertools must duplicate output streams in order to log them. + If you use this on a RequestStreamHandler, Powertools must duplicate output streams in order to log them when used together with the `@Logging` annotation. ## Logging handler uncaught exception By default, AWS Lambda logs any uncaught exception that might happen in the handler. However, this log is not structured -and does not contain any additional context. You can instruct the `@Logging` annotation to log this kind of exception +and does not contain any additional context. When using the `@Logging` annotation, you can enable structured exception logging with `logError` param or via `POWERTOOLS_LOGGER_LOG_ERROR` env var. ???+ warning - This is disabled by default to prevent double logging + This is disabled by default to prevent double logging. -=== "AppLogResponse.java" +???+ note + This feature is only available when using the `@Logging` annotation. When using the functional API, you must catch and log exceptions manually using try-catch blocks. + +=== "@Logging annotation" ```java hl_lines="5" public class AppLogError implements RequestHandler { @@ -787,6 +905,29 @@ with `logError` param or via `POWERTOOLS_LOGGER_LOG_ERROR` env var. } ``` +=== "Functional API" + + ```java hl_lines="1 9 12-13" + import org.slf4j.MarkerFactory; + + public class AppLogError implements RequestHandler { + + private static final Logger LOGGER = LoggerFactory.getLogger(AppLogError.class); + + public APIGatewayProxyResponseEvent handleRequest(final APIGatewayProxyRequestEvent input, final Context context) { + return PowertoolsLogging.withLogging(context, () -> { + try { + // ... + return new APIGatewayProxyResponseEvent().withStatusCode(200); + } catch (Exception e) { + LOGGER.error(MarkerFactory.getMarker("FATAL"), "Exception in Lambda Handler", e); + throw e; + } + }); + } + } + ``` + ## Advanced ### Buffering logs @@ -1050,7 +1191,10 @@ You can manually control the log buffer using the `PowertoolsLogging` utility cl Use the `@Logging` annotation to automatically flush buffered logs when an uncaught exception is raised in your Lambda function. This is enabled by default (`flushBufferOnUncaughtError = true`), but you can explicitly configure it if needed. -=== "PaymentFunction.java" +???+ warning + This feature is only available when using the `@Logging` annotation. When using the functional API, you must manually flush the buffer in exception handlers. + +=== "@Logging annotation" ```java hl_lines="5 11" public class PaymentFunction implements RequestHandler { @@ -1068,6 +1212,30 @@ Use the `@Logging` annotation to automatically flush buffered logs when an uncau } ``` +=== "Functional API" + + ```java hl_lines="14" + import software.amazon.lambda.powertools.logging.PowertoolsLogging; + + public class PaymentFunction implements RequestHandler { + + private static final Logger LOGGER = LoggerFactory.getLogger(PaymentFunction.class); + + public APIGatewayProxyResponseEvent handleRequest(final APIGatewayProxyRequestEvent input, final Context context) { + return PowertoolsLogging.withLogging(context, () -> { + try { + LOGGER.debug("a debug log"); // this is buffered + // do stuff + throw new RuntimeException("Something went wrong"); + } catch (Exception e) { + PowertoolsLogging.flushBuffer(); // Manually flush buffered logs + throw e; + } + }); + } + } + ``` + #### Buffering workflows ##### Manual flush @@ -1161,13 +1329,13 @@ sequenceDiagram ## Sampling debug logs -You can dynamically set a percentage of your logs to`DEBUG` level to be included in the logger output, regardless of configured log leve, using the`POWERTOOLS_LOGGER_SAMPLE_RATE` environment variable or -via `samplingRate` attribute on the `@Logging` annotation. +You can dynamically set a percentage of your logs to`DEBUG` level to be included in the logger output, regardless of configured log level, using the`POWERTOOLS_LOGGER_SAMPLE_RATE` environment variable, +via the `samplingRate` attribute on the `@Logging` annotation, or as a parameter in the functional API. !!! info - Configuration on environment variable is given precedence over sampling rate configuration on annotation, provided it's in valid value range. + Configuration via environment variable is given precedence over sampling rate configuration, provided it's in valid value range. -=== "Sampling via annotation attribute" +=== "@Logging annotation" ```java hl_lines="5" public class App implements RequestHandler { @@ -1182,6 +1350,23 @@ via `samplingRate` attribute on the `@Logging` annotation. } ``` +=== "Functional API" + + ```java hl_lines="6" + public class App implements RequestHandler { + + private static final Logger LOGGER = LoggerFactory.getLogger(App.class); + + public APIGatewayProxyResponseEvent handleRequest(final APIGatewayProxyRequestEvent input, final Context context) { + return PowertoolsLogging.withLogging(context, 0.5, () -> { + // will eventually be logged based on the sampling rate + LOGGER.debug("Handle payment"); + return new APIGatewayProxyResponseEvent().withStatusCode(200); + }); + } + } + ``` + === "Sampling via environment variable" ```yaml hl_lines="8" @@ -1198,7 +1383,7 @@ via `samplingRate` attribute on the `@Logging` annotation. ## Built-in Correlation ID expressions -You can use any of the following built-in JMESPath expressions as part of `@Logging(correlationIdPath = ...)`: +You can use any of the following built-in JMESPath expressions with the `@Logging` annotation or the functional API: ???+ note "Note: Any object key named with `-` must be escaped" For example, **`request.headers."x-amzn-trace-id"`**. @@ -1237,8 +1422,7 @@ The `JsonTemplateLayout` is automatically configured with the provided template: "field": "name" }, "message": { - "$resolver": "powertools", - "field": "message" + "$resolver": "message" }, "error": { "message": { @@ -1299,6 +1483,10 @@ The `JsonTemplateLayout` is automatically configured with the provided template: "$resolver": "powertools", "field": "xray_trace_id" }, + "correlation_id": { + "$resolver": "powertools", + "field": "correlation_id" + }, "": { "$resolver": "powertools" } diff --git a/docs/core/metrics.md b/docs/core/metrics.md index 71c56bb8b..e7f7bd87f 100644 --- a/docs/core/metrics.md +++ b/docs/core/metrics.md @@ -48,6 +48,7 @@ Visit the AWS documentation for a complete explanation for [Amazon CloudWatch co ... + ... @@ -89,10 +90,10 @@ Visit the AWS documentation for a complete explanation for [Amazon CloudWatch co === "Gradle" - ```groovy hl_lines="3 11" + ```groovy hl_lines="3 11 12" plugins { id 'java' - id 'io.freefair.aspectj.post-compile-weaving' version '8.1.0' + id 'io.freefair.aspectj.post-compile-weaving' version '8.1.0' // Not needed when using the functional approach } repositories { @@ -100,7 +101,8 @@ Visit the AWS documentation for a complete explanation for [Amazon CloudWatch co } dependencies { - aspect 'software.amazon.lambda:powertools-metrics:{{ powertools.version }}' + aspect 'software.amazon.lambda:powertools-metrics:{{ powertools.version }}' // Not needed when using the functional approach + implementation 'software.amazon.lambda:powertools-metrics:{{ powertools.version }}' // Use this instead of 'aspect' when using the functional approach } sourceCompatibility = 11 @@ -127,27 +129,12 @@ Metrics has three global settings that will be used across all metrics emitted. The `Metrics` Singleton can be configured by three different interfaces. The following order of precedence applies: 1. `@FlushMetrics` annotation -2. `MetricsBuilder` using Builder pattern (see [Advanced section](#usage-without-metrics-annotation)) +2. `MetricsBuilder` using Builder pattern (see [Advanced section](#usage-without-flushmetrics-annotation)) 3. Environment variables (recommended) For most use-cases, we recommend using Environment variables and only overwrite settings in code where needed using either the `@FlushMetrics` annotation or `MetricsBuilder` if the annotation cannot be used. -=== "template.yaml" - - ```yaml hl_lines="9 10" - Resources: - HelloWorldFunction: - Type: AWS::Serverless::Function - Properties: - ... - Runtime: java11 - Environment: - Variables: - POWERTOOLS_SERVICE_NAME: payment - POWERTOOLS_METRICS_NAMESPACE: ServerlessAirline - ``` - -=== "MetricsEnabledHandler.java" +=== "@FlushMetrics annotation" ```java hl_lines="9" import software.amazon.lambda.powertools.metrics.FlushMetrics; @@ -165,9 +152,45 @@ For most use-cases, we recommend using Environment variables and only overwrite } ``` -`Metrics` is implemented as a Singleton to keep track of your aggregate metrics in memory and make them accessible anywhere in your code. To guarantee that metrics are flushed properly the `@FlushMetrics` annotation must be added on the lambda handler. +=== "MetricsBuilder" + + ```java hl_lines="7-8" + import software.amazon.lambda.powertools.metrics.Metrics; + import software.amazon.lambda.powertools.metrics.MetricsBuilder; + + public class MetricsEnabledHandler implements RequestHandler { + + private static final Metrics metrics = MetricsBuilder.builder() + .withNamespace("ServerlessAirline") + .withService("payment") + .build(); + + @Override + public Object handleRequest(Object input, Context context) { + // ... + metrics.flush(); + } + } + ``` + +=== "Environment variables" + + ```yaml hl_lines="9 10" + Resources: + HelloWorldFunction: + Type: AWS::Serverless::Function + Properties: + ... + Runtime: java11 + Environment: + Variables: + POWERTOOLS_SERVICE_NAME: payment + POWERTOOLS_METRICS_NAMESPACE: ServerlessAirline + ``` + +`Metrics` is implemented as a Singleton to keep track of your aggregate metrics in memory and make them accessible anywhere in your code. The `@FlushMetrics` annotation automatically flushes metrics at the end of the Lambda handler execution. Alternatively, you can use the functional approach and manually flush metrics using `metrics.flush()`. -!!!info "You can use the Metrics utility without the `@FlushMetrics` annotation and flush manually. Read more in the [advanced section below](#usage-without-metrics-annotation)." +!!!info "Read more about the functional approach in the [advanced section below](#usage-without-flushmetrics-annotation)." ## Creating metrics @@ -381,7 +404,7 @@ You can use `addMetadata` for advanced use cases, where you want to add metadata This will not be available during metrics visualization, use Dimensions for this purpose. !!! info - Adding metadata with a key that is the same as an existing metric will be ignored + Adding metadata with a key that is the same as an existing metric will be ignored. === "App.java" @@ -468,7 +491,7 @@ You can create metrics with different configurations e.g. different namespace an === "App.java" - ```java hl_lines="12-18" + ```java hl_lines="12-22" import software.amazon.lambda.powertools.metrics.Metrics; import software.amazon.lambda.powertools.metrics.MetricsFactory; import software.amazon.lambda.powertools.metrics.model.DimensionSet; @@ -504,22 +527,22 @@ You can create metrics with different configurations e.g. different namespace an ### Usage without `@FlushMetrics` annotation -The `Metrics` Singleton provides all configuration options via `MetricsBuilder` in addition to the `@FlushMetrics` annotation. This can be useful if work in an environment or framework that does not leverage the vanilla Lambda `handleRequest` method. +You can use the **functional API** approach (see [usage patterns](../usage-patterns.md#functional-approach)) to work with Metrics without the `@FlushMetrics` annotation. The `Metrics` Singleton provides all configuration options via `MetricsBuilder`. This approach eliminates the AspectJ runtime dependency and is useful if you work in an environment or with a framework that does not leverage the vanilla Lambda `handleRequest` method. !!!info "The environment variables for Service and Namespace configuration still apply but can be overwritten with `MetricsBuilder` if needed." -The following example shows how to configure a custom `Metrics` Singleton using the Builder pattern. Note that it is necessary to manually flush metrics now. +The following example shows how to configure a custom `Metrics` Singleton using the Builder pattern. With the functional approach, you must manually flush metrics using `metrics.flush()`. === "App.java" - ```java hl_lines="7-12 19 23" + ```java hl_lines="7-12 19 24" import software.amazon.lambda.powertools.metrics.Metrics; import software.amazon.lambda.powertools.metrics.MetricsBuilder; import software.amazon.lambda.powertools.metrics.model.DimensionSet; import software.amazon.lambda.powertools.metrics.model.MetricUnit; public class App implements RequestHandler { - // Create and configure a Metrics singleton without annotation + // Create and configure a Metrics singleton using the functional approach private static final Metrics metrics = MetricsBuilder.builder() .withNamespace("ServerlessAirline") .withRaiseOnEmptyMetrics(true) @@ -533,8 +556,9 @@ The following example shows how to configure a custom `Metrics` Singleton using // Dimensions are also optional. metrics.captureColdStartMetric(context, DimensionSet.of("FunctionName", "MyFunction", "Service", "payment")); - // Add metrics to the custom metrics singleton + // Add metrics metrics.addMetric("CustomMetric", 1, MetricUnit.COUNT); + // Manually flush metrics metrics.flush(); } } diff --git a/docs/core/tracing.md b/docs/core/tracing.md index 8129d45ba..95fbe6d06 100644 --- a/docs/core/tracing.md +++ b/docs/core/tracing.md @@ -20,7 +20,7 @@ a provides functionality to reduce the overhead of performing common tracing tas === "Maven" - ```xml hl_lines="3-7 16 18 24-27" + ```xml hl_lines="3-7 25-28" ... @@ -32,6 +32,7 @@ a provides functionality to reduce the overhead of performing common tracing tas ... + ... @@ -73,10 +74,10 @@ a provides functionality to reduce the overhead of performing common tracing tas === "Gradle" - ```groovy hl_lines="3 11" + ```groovy hl_lines="3 11 12" plugins { id 'java' - id 'io.freefair.aspectj.post-compile-weaving' version '8.1.0' + id 'io.freefair.aspectj.post-compile-weaving' version '8.1.0' // Not needed when using the functional approach } repositories { @@ -84,11 +85,12 @@ a provides functionality to reduce the overhead of performing common tracing tas } dependencies { - aspect 'software.amazon.lambda:powertools-tracing:{{ powertools.version }}' + aspect 'software.amazon.lambda:powertools-tracing:{{ powertools.version }}' // Not needed when using the functional approach + implementation 'software.amazon.lambda:powertools-tracing:{{ powertools.version }}' // Use this instead of 'aspect' when using the functional approach } - sourceCompatibility = 11 - targetCompatibility = 11 + sourceCompatibility = 11 // or higher + targetCompatibility = 11 // or higher ``` ## Initialization @@ -118,11 +120,13 @@ The Powertools for AWS Lambda (Java) service name is used as the X-Ray namespace ### Lambda handler -To enable Powertools for AWS Lambda (Java) tracing to your function add the `@Tracing` annotation to your `handleRequest` method or on -any method will capture the method as a separate subsegment automatically. You can optionally choose to customize -segment name that appears in traces. +You can enable tracing using either the `@Tracing` annotation or the functional API. -=== "Tracing annotation" +**With the `@Tracing` annotation**, add it to your `handleRequest` method or any method to capture it as a separate subsegment automatically. You can optionally customize the segment name that appears in traces. + +**With the functional API**, use `TracingUtils.withSubsegment()` to manually create subsegments without AspectJ configuration. + +=== "@Tracing annotation" ```java hl_lines="3 10 15" public class App implements RequestHandler { @@ -146,6 +150,25 @@ segment name that appears in traces. } ``` +=== "Functional API" + + ```java hl_lines="1 6 7 8 10 11 12" + import software.amazon.lambda.powertools.tracing.TracingUtils; + + public class App implements RequestHandler { + + public APIGatewayProxyResponseEvent handleRequest(APIGatewayProxyRequestEvent input, Context context) { + TracingUtils.withSubsegment("businessLogic1", subsegment -> { + // Business logic 1 + }); + + TracingUtils.withSubsegment("businessLogic2", subsegment -> { + // Business logic 2 + }); + } + } + ``` + === "Custom Segment names" ```java hl_lines="3" @@ -157,22 +180,25 @@ segment name that appears in traces. } ``` -When using this `@Tracing` annotation, Utility performs these additional tasks to ease operations: +When using the `@Tracing` annotation, the utility performs these additional tasks to ease operations: * Creates a `ColdStart` annotation to easily filter traces that have had an initialization overhead. * Creates a `Service` annotation if service parameter or `POWERTOOLS_SERVICE_NAME` is set. * Captures any response, or full exceptions generated by the handler, and include as tracing metadata. +By default, the `@Tracing` annotation uses `captureMode=ENVIRONMENT_VAR`, which means it will only record method responses and exceptions if you set +the environment variables `POWERTOOLS_TRACER_CAPTURE_RESPONSE` and `POWERTOOLS_TRACER_CAPTURE_ERROR` to `true`. You can override this behavior by +specifying a different `captureMode` to always record response, exception, both, or neither. -By default, this annotation will automatically record method responses and exceptions. You can change the default behavior by setting -the environment variables `POWERTOOLS_TRACER_CAPTURE_RESPONSE` and `POWERTOOLS_TRACER_CAPTURE_ERROR` as needed. Optionally, you can override behavior by -different supported `captureMode` to record response, exception or both. +!!! note + When using the functional API with `TracingUtils.withSubsegment()`, response and exception capture is not automatic. You can manually add metadata using `TracingUtils.putMetadata()` as needed. -!!! warning "Returning sensitive information from your Lambda handler or functions, where `Tracing` is used?" - You can disable annotation from capturing their responses and exception as tracing metadata with **`captureMode=DISABLED`** - or globally by setting environment variables **`POWERTOOLS_TRACER_CAPTURE_RESPONSE`** and **`POWERTOOLS_TRACER_CAPTURE_ERROR`** to **`false`** +!!! warning "Returning sensitive information from your Lambda handler or functions?" + When using the `@Tracing` annotation, you can disable it from capturing responses and exceptions as tracing metadata with **`captureMode=DISABLED`** + or globally by setting the environment variables **`POWERTOOLS_TRACER_CAPTURE_RESPONSE`** and **`POWERTOOLS_TRACER_CAPTURE_ERROR`** to **`false`**. + When using the functional API, you have full control over what metadata is captured. -=== "Disable on annotation" +=== "@Tracing annotation - Disable on method" ```java hl_lines="3" public class App implements RequestHandler { @@ -183,7 +209,7 @@ different supported `captureMode` to record response, exception or both. } ``` -=== "Disable Globally" +=== "@Tracing annotation - Disable Globally" ```yaml hl_lines="11 12" Resources: @@ -200,6 +226,20 @@ different supported `captureMode` to record response, exception or both. POWERTOOLS_TRACER_CAPTURE_ERROR: false ``` +=== "Functional API" + + ```java hl_lines="6 7 8" + import software.amazon.lambda.powertools.tracing.TracingUtils; + + public class App implements RequestHandler { + + public APIGatewayProxyResponseEvent handleRequest(APIGatewayProxyRequestEvent input, Context context) { + TracingUtils.withSubsegment("businessLogic", subsegment -> { + // With functional API, you control what metadata is captured + }); + } + ``` + ### Annotations & Metadata **Annotations** are key-values associated with traces and indexed by AWS X-Ray. You can use them to filter traces and to @@ -272,32 +312,13 @@ specific fields from received event due to security. } ``` -## Utilities - -Tracing modules comes with certain utility method when you don't want to use annotation for capturing a code block -under a subsegment, or you are doing multithreaded programming. Refer examples below. +## Advanced usage -=== "Functional Api" +### Multi-threaded programming - ```java hl_lines="7 8 9 11 12 13" - import software.amazon.lambda.powertools.tracing.Tracing; - import software.amazon.lambda.powertools.tracing.TracingUtils; - - public class App implements RequestHandler { - - public APIGatewayProxyResponseEvent handleRequest(APIGatewayProxyRequestEvent input, Context context) { - TracingUtils.withSubsegment("loggingResponse", subsegment -> { - // Some business logic - }); - - TracingUtils.withSubsegment("localNamespace", "loggingResponse", subsegment -> { - // Some business logic - }); - } - } - ``` +When working with multiple threads, you need to pass the trace entity to ensure proper trace context propagation. -=== "Multi Threaded Programming" +=== "Multi-threaded example" ```java hl_lines="7 9 10 11" import static software.amazon.lambda.powertools.tracing.TracingUtils.withEntitySubsegment; @@ -317,25 +338,33 @@ under a subsegment, or you are doing multithreaded programming. Refer examples b ## Instrumenting SDK clients and HTTP calls -Powertools for Lambda (Java) cannot intercept SDK clients instantiation to add X-Ray instrumentation. You should make sure to instrument the SDK clients explicitly. Refer details on -[how to instrument SDK client with Xray](https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-java.html#xray-sdk-java-awssdkclients) -and [outgoing http calls](https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-java.html#xray-sdk-java-httpclients). For example: +### AWS SDK for Java 2.x -=== "LambdaHandler.java" +Powertools for AWS Lambda (Java) includes the `aws-xray-recorder-sdk-aws-sdk-v2-instrumentor` library, which **automatically instruments all AWS SDK v2 clients** when you add the `powertools-tracing` dependency to your project. This means downstream calls to AWS services are traced without any additional configuration. - ```java hl_lines="1 2 7" - import com.amazonaws.xray.AWSXRay; - import com.amazonaws.xray.handlers.TracingHandler; +If you need more control over which clients are instrumented, you can manually add the `TracingInterceptor` to specific clients: + +=== "Manual instrumentation (optional)" + + ```java hl_lines="1 2 3 8 9 10 11" + import com.amazonaws.xray.interceptors.TracingInterceptor; + import software.amazon.awssdk.core.client.config.ClientOverrideConfiguration; + import software.amazon.awssdk.services.dynamodb.DynamoDbClient; public class LambdaHandler { - private AmazonDynamoDB client = AmazonDynamoDBClientBuilder.standard() - .withRegion(Regions.fromName(System.getenv("AWS_REGION"))) - .withRequestHandlers(new TracingHandler(AWSXRay.getGlobalRecorder())) + private DynamoDbClient client = DynamoDbClient.builder() + .region(Region.US_WEST_2) + .overrideConfiguration(ClientOverrideConfiguration.builder() + .addExecutionInterceptor(new TracingInterceptor()) + .build() + ) .build(); // ... } ``` +For more details, refer to the [AWS X-Ray documentation on tracing AWS SDK calls](https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-java-awssdkclients.html) and [outgoing HTTP calls](https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-java-httpclients.html). + ## Testing your code When using `@Tracing` annotation, your Junit test cases needs to be configured to create parent Segment required by [AWS X-Ray SDK for Java](https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-java.html). @@ -351,7 +380,7 @@ used internally via AWS X-Ray SDK to configure itself properly for lambda runtim === "Maven (pom.xml)" - ```xml hl_lines="4-13" + ```xml ... @@ -370,9 +399,9 @@ used internally via AWS X-Ray SDK to configure itself properly for lambda runtim ``` -=== "Gradle (build.gradle) " +=== "Gradle (build.gradle)" - ```json hl_lines="2-4" + ```json // Configures environment variable to avoid initialization of AWS X-Ray segments for each tests test { environment "LAMBDA_TASK_ROOT", "handler" @@ -418,6 +447,3 @@ Below is an example configuration needed for each test case. // test logic } ``` - - - diff --git a/docs/index.md b/docs/index.md index 9c5c803cb..655c16e03 100644 --- a/docs/index.md +++ b/docs/index.md @@ -26,6 +26,7 @@ This project separates core utilities that will be available in other runtimes v ## Install + -**Manual installation** Powertools for AWS Lambda (Java) dependencies are available in Maven Central. You can use your favourite dependency management tool to install it * [Maven](https://maven.apache.org/) @@ -90,7 +91,7 @@ Powertools for AWS Lambda (Java) dependencies are available in Maven Central. Yo software.amazon.lambda - powertools-logging + powertools-logging-log4j {{ powertools.version }} @@ -107,7 +108,8 @@ Powertools for AWS Lambda (Java) dependencies are available in Maven Central. Yo ... ... - + + ... @@ -175,7 +177,8 @@ Powertools for AWS Lambda (Java) dependencies are available in Maven Central. Yo } dependencies { - aspect 'software.amazon.lambda:powertools-logging:{{ powertools.version }}' + // Note: This AspectJ configuration is not needed when using the functional approach + aspect 'software.amazon.lambda:powertools-logging-log4j:{{ powertools.version }}' aspect 'software.amazon.lambda:powertools-tracing:{{ powertools.version }}' aspect 'software.amazon.lambda:powertools-metrics:{{ powertools.version }}' } @@ -184,28 +187,15 @@ Powertools for AWS Lambda (Java) dependencies are available in Maven Central. Yo targetCompatibility = 11 ``` -???+ tip "Why a different configuration?" - Powertools for AWS Lambda (Java) is using [AspectJ](https://eclipse.dev/aspectj/doc/released/progguide/starting.html) internally - to handle annotations. Recently, in order to support Java 17 we had to move to `dev.aspectj:aspectj-maven-plugin` because - `org.codehaus.mojo:aspectj-maven-plugin` does not support Java 17. - Under the hood, `org.codehaus.mojo:aspectj-maven-plugin` is based on AspectJ 1.9.7, - while `dev.aspectj:aspectj-maven-plugin` is based on AspectJ 1.9.8, compiled for Java 11+. +???+ tip "Don't want to use AspectJ?" + Powertools for AWS Lambda (Java) now provides a functional API that doesn't require AspectJ configuration. Learn more about the [functional approach](./usage-patterns.md#functional-approach). ### Java Compatibility -Powertools for AWS Lambda (Java) supports all Java version from 11 up to 21 as well as the -[corresponding Lambda runtimes](https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html). +Powertools for AWS Lambda (Java) supports all Java versions from 11 to 25 in line with the [corresponding Lambda runtimes](https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html). -For the following modules, Powertools for AWS Lambda (Java) leverages the **aspectj** library to provide annotations: -- Logging -- Metrics -- Tracing -- Parameters -- Idempotency -- Validation -- Large messages +In addition to the functional approach, [Logging](./core/logging.md), [Metrics](./core/metrics.md), [Tracing](./core/tracing.md), [Parameters](./utilities/parameters.md), [Idempotency](./utilities/idempotency.md), [Validation](./utilities/validation.md), and [Large Messages](./utilities/large_messages.md) utilities support annotations using AspectJ, which require configuration of the `aspectjrt` runtime library. - -You may need to add the good version of `aspectjrt` to your dependencies based on the jdk used for building your function: +You may need to add the appropriate version of `aspectjrt` to your dependencies based on the JDK used for building your function: ```xml @@ -215,17 +205,18 @@ You may need to add the good version of `aspectjrt` to your dependencies based o ``` -Use the following [dependency matrix](https://github.com/eclipse-aspectj/aspectj/blob/master/docs/dist/doc/JavaVersionCompatibility.md) between this library and the JDK: +Use the following [dependency matrix](https://github.com/eclipse-aspectj/aspectj/blob/master/docs/release/JavaVersionCompatibility.adoc) to understand which AspectJ version to use based on your JDK version: | JDK version | aspectj version | |-------------|------------------------| | `11-17` | `1.9.20.1` (or higher) | | `21` | `1.9.21` (or higher) | +| `25` | `1.9.25` (or higher) | ## Environment variables !!! info - **Explicit parameters take precedence over environment variables.** + Explicit parameters take precedence over environment variables. | Environment variable | Description | Utility | | -------------------------------------- | -------------------------------------------------------------------------------------- | ------------------------- | diff --git a/docs/processes/maintainers.md b/docs/processes/maintainers.md index 8f7f6a8fd..f2839c532 100644 --- a/docs/processes/maintainers.md +++ b/docs/processes/maintainers.md @@ -17,7 +17,6 @@ This is document explains who the maintainers are, their responsibilities, and h | Maintainer | GitHub ID | Affiliation | | --------------- | -------------------------------------------------------------------- | ----------- | | Philipp Page | [phipag](https://github.com/phipag){target="\_blank" rel="nofollow"} | Amazon | -| Simon Thulbourn | [sthulb](https://github.com/sthulb){target="\_blank" rel="nofollow"} | Amazon | ## Emeritus @@ -25,6 +24,7 @@ Previous active maintainers who contributed to this project. | Maintainer | GitHub ID | Affiliation | | --------------------- | -------------------------------------------------------------------------------------- | ------------- | +| Simon Thulbourn | [sthulb](https://github.com/sthulb){target="\_blank" rel="nofollow"} | Former Amazon | | Jerome Van Der Linden | [jeromevdl](https://github.com/jeromevdl){target="\_blank" rel="nofollow"} | Amazon | | Michele Ricciardi | [mriccia](https://github.com/mriccia){target="\_blank" rel="nofollow"} | Amazon | | Scott Gerring | [scottgerring](https://github.com/scottgerring){target="\_blank" rel="nofollow"} | DataDog | diff --git a/docs/usage-patterns.md b/docs/usage-patterns.md new file mode 100644 index 000000000..e66538937 --- /dev/null +++ b/docs/usage-patterns.md @@ -0,0 +1,183 @@ +--- +title: Usage patterns +description: Getting to know the Powertools for AWS Lambda toolkit +--- + + + +Powertools for AWS Lambda (Java) is a collection of utilities designed to help you build serverless applications on AWS. + +The toolkit is modular, so you can pick and choose the utilities you need for your application, but also combine them for a complete solution for your serverless applications. + +## Patterns + +Many of the utilities provided can be used with different patterns, depending on your preferences and the structure of your code. + +### AspectJ Annotation + +If you prefer using annotations to apply cross-cutting concerns to your Lambda handlers, the AspectJ annotation pattern is a good fit. This approach lets you decorate methods with Powertools utilities using annotations, applying their functionality with minimal code changes. + +This pattern works well when you want to keep your business logic clean and separate concerns using aspect-oriented programming. + + +!!! note + This approach requires configuring AspectJ compile-time weaving in your build tool (Maven or Gradle). See the [installation guide](./index.md#install) for setup instructions. + +=== "Logging" + + ```java + import com.amazonaws.services.lambda.runtime.Context; + import com.amazonaws.services.lambda.runtime.RequestHandler; + import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyRequestEvent; + import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyResponseEvent; + import org.slf4j.Logger; + import org.slf4j.LoggerFactory; + import software.amazon.lambda.powertools.logging.CorrelationIdPaths; + import software.amazon.lambda.powertools.logging.Logging; + + public class App implements RequestHandler { + private static final Logger log = LoggerFactory.getLogger(App.class); + + @Logging(logEvent = true, correlationIdPath = CorrelationIdPaths.API_GATEWAY_REST) + public APIGatewayProxyResponseEvent handleRequest(APIGatewayProxyRequestEvent input, Context context) { + log.info("Processing request"); + return new APIGatewayProxyResponseEvent().withStatusCode(200).withBody("Success"); + } + } + ``` + +=== "Metrics" + + ```java + import com.amazonaws.services.lambda.runtime.Context; + import com.amazonaws.services.lambda.runtime.RequestHandler; + import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyRequestEvent; + import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyResponseEvent; + import software.amazon.lambda.powertools.metrics.FlushMetrics; + import software.amazon.lambda.powertools.metrics.Metrics; + import software.amazon.lambda.powertools.metrics.MetricsFactory; + import software.amazon.lambda.powertools.metrics.model.MetricUnit; + + public class App implements RequestHandler { + private static final Metrics metrics = MetricsFactory.getMetricsInstance(); + + @FlushMetrics(namespace = "ServerlessApp", service = "payment") + public APIGatewayProxyResponseEvent handleRequest(APIGatewayProxyRequestEvent input, Context context) { + metrics.addMetric("SuccessfulBooking", 1, MetricUnit.COUNT); + return new APIGatewayProxyResponseEvent().withStatusCode(200).withBody("Success"); + } + } + ``` + +=== "Tracing" + + ```java + import com.amazonaws.services.lambda.runtime.Context; + import com.amazonaws.services.lambda.runtime.RequestHandler; + import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyRequestEvent; + import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyResponseEvent; + import software.amazon.lambda.powertools.tracing.Tracing; + import software.amazon.lambda.powertools.tracing.TracingUtils; + + public class App implements RequestHandler { + + @Tracing + public APIGatewayProxyResponseEvent handleRequest(APIGatewayProxyRequestEvent input, Context context) { + TracingUtils.putAnnotation("operation", "payment"); + return processPayment(); + } + + @Tracing + private APIGatewayProxyResponseEvent processPayment() { + return new APIGatewayProxyResponseEvent().withStatusCode(200).withBody("Success"); + } + } + ``` + +### Functional Approach + +If you prefer a more functional programming style or want to avoid AspectJ configuration, you can use the Powertools for AWS Lambda (Java) utilities directly in your code. This approach is more explicit and provides full control over how the utilities are applied. + +This pattern is ideal when you want to avoid AspectJ setup or prefer a more imperative style. It also eliminates the AspectJ runtime dependency, making your deployment package more lightweight. + +=== "Logging" + + ```java + import com.amazonaws.services.lambda.runtime.Context; + import com.amazonaws.services.lambda.runtime.RequestHandler; + import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyRequestEvent; + import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyResponseEvent; + import org.slf4j.Logger; + import org.slf4j.LoggerFactory; + import software.amazon.lambda.powertools.logging.CorrelationIdPaths; + import software.amazon.lambda.powertools.logging.PowertoolsLogging; + + public class App implements RequestHandler { + private static final Logger log = LoggerFactory.getLogger(App.class); + + public APIGatewayProxyResponseEvent handleRequest(APIGatewayProxyRequestEvent input, Context context) { + return PowertoolsLogging.withLogging( + context, + 0.7, + CorrelationIdPaths.API_GATEWAY_REST, + input, + () -> processRequest(input)); + } + + private APIGatewayProxyResponseEvent processRequest(APIGatewayProxyRequestEvent input) { + // do something with input + log.info("Processing request"); + return new APIGatewayProxyResponseEvent().withStatusCode(200).withBody("Success"); + } + } + ``` + +=== "Metrics" + + ```java + import com.amazonaws.services.lambda.runtime.Context; + import com.amazonaws.services.lambda.runtime.RequestHandler; + import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyRequestEvent; + import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyResponseEvent; + import software.amazon.lambda.powertools.metrics.Metrics; + import software.amazon.lambda.powertools.metrics.MetricsFactory; + import software.amazon.lambda.powertools.metrics.model.MetricUnit; + + public class App implements RequestHandler { + private static final Metrics metrics = MetricsFactory.getMetricsInstance(); + + public APIGatewayProxyResponseEvent handleRequest(APIGatewayProxyRequestEvent input, Context context) { + try { + metrics.addMetric("SuccessfulBooking", 1, MetricUnit.COUNT); + return new APIGatewayProxyResponseEvent().withStatusCode(200).withBody("Success"); + } finally { + metrics.flush(); + } + } + } + ``` + +=== "Tracing" + + ```java + import com.amazonaws.services.lambda.runtime.Context; + import com.amazonaws.services.lambda.runtime.RequestHandler; + import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyRequestEvent; + import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyResponseEvent; + import software.amazon.lambda.powertools.tracing.TracingUtils; + + public class App implements RequestHandler { + + public APIGatewayProxyResponseEvent handleRequest(APIGatewayProxyRequestEvent input, Context context) { + TracingUtils.withSubsegment("processPayment", subsegment -> { + subsegment.putAnnotation("operation", "payment"); + // Business logic here + }); + return new APIGatewayProxyResponseEvent().withStatusCode(200).withBody("Success"); + } + } + ``` + + +!!! note + The functional approach is available for all utilities. Further examples and detailed usage can be found in the individual documentation pages for each utility. diff --git a/docs/utilities/idempotency.md b/docs/utilities/idempotency.md index 83f256e6b..cecc65d7b 100644 --- a/docs/utilities/idempotency.md +++ b/docs/utilities/idempotency.md @@ -29,7 +29,7 @@ times with the same parameters**. This makes idempotent operations safe to retry === "Maven" - ```xml hl_lines="3-7 16 18 24-27" + ```xml hl_lines="3-7 16 18 25-28" ... @@ -41,6 +41,7 @@ times with the same parameters**. This makes idempotent operations safe to retry ... + ... @@ -82,10 +83,10 @@ times with the same parameters**. This makes idempotent operations safe to retry === "Gradle" - ```groovy hl_lines="3 11" + ```groovy hl_lines="3 11 12" plugins { id 'java' - id 'io.freefair.aspectj.post-compile-weaving' version '8.1.0' + id 'io.freefair.aspectj.post-compile-weaving' version '8.1.0' // Not needed when using the functional approach } repositories { @@ -93,7 +94,8 @@ times with the same parameters**. This makes idempotent operations safe to retry } dependencies { - aspect 'software.amazon.lambda:powertools-idempotency-dynamodb:{{ powertools.version }}' + aspect 'software.amazon.lambda:powertools-idempotency-core:{{ powertools.version }}' // Not needed when using the functional approach + implementation 'software.amazon.lambda:powertools-idempotency-dynamodb:{{ powertools.version }}' } sourceCompatibility = 11 // or higher @@ -104,7 +106,7 @@ times with the same parameters**. This makes idempotent operations safe to retry Before getting started, you need to create a persistent storage layer where the idempotency utility can store its state - your Lambda functions will need read and write access to it. -As of now, Amazon DynamoDB is the only supported persistent storage layer, so you'll need to create a table first. +As of now, Amazon DynamoDB is the only supported persistent storage layer, so you'll need to create a table first or [bring your own persistence store](#bring-your-own-persistent-store). **Default table configuration** @@ -148,29 +150,29 @@ Resources: ``` !!! warning "Warning: Large responses with DynamoDB persistence layer" - When using this utility with DynamoDB, your function's responses must be [smaller than 400KB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html#limits-items). + When using this utility with DynamoDB, your function's responses must be [smaller than 400KB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Constraints.html#limits-items). Larger items cannot be written to DynamoDB and will cause exceptions. !!! info "Info: DynamoDB" - Each function invocation will generally make 2 requests to DynamoDB. If the - result returned by your Lambda is less than 1kb, you can expect 2 WCUs per invocation. For retried invocations, you will - see 1WCU and 1RCU. Review the [DynamoDB pricing documentation](https://aws.amazon.com/dynamodb/pricing/) to + Each function invocation will generally make 1 request to DynamoDB. If the + result returned by your Lambda is less than 1kb, you can expect 1 WCUs per invocation. For retried invocations, you will + see 1 WCU. In some cases, the utility might make 2 requests to DynamoDB in which case you will see 1 RCU and 1 WCU. Review the [DynamoDB pricing documentation](https://aws.amazon.com/dynamodb/pricing/) to estimate the cost. -### Idempotent annotation +### Basic usage -You can quickly start by initializing the `DynamoDBPersistenceStore` and using it with the `@Idempotent` annotation on your Lambda handler. +You can use Powertools for AWS Lambda Idempotency with either the `@Idempotent` annotation or the functional API. !!! warning "Important" Initialization and configuration of the `DynamoDBPersistenceStore` must be performed outside the handler, preferably in the constructor. -=== "App.java" +=== "@Idempotent annotation" ```java hl_lines="5-9 12 19" public class App implements RequestHandler { public App() { - // we need to initialize idempotency store before the handleRequest method is called + // We need to initialize idempotency store before the handleRequest method is called Idempotency.config().withPersistenceStore( DynamoDBPersistenceStore.builder() .withTableName(System.getenv("TABLE_NAME")) @@ -191,6 +193,33 @@ You can quickly start by initializing the `DynamoDBPersistenceStore` and using i ``` +=== "Functional API" + + ```java hl_lines="5-9 13-14" + public class App implements RequestHandler { + + public App() { + // We need to initialize idempotency store before the handleRequest method is called + Idempotency.config().withPersistenceStore( + DynamoDBPersistenceStore.builder() + .withTableName(System.getenv("TABLE_NAME")) + .build() + ).configure(); + } + + public SubscriptionResult handleRequest(final Subscription event, final Context context) { + Idempotency.registerLambdaContext(context); + return Idempotency.makeIdempotent(this::processSubscription, event, SubscriptionResult.class); + } + + private SubscriptionResult processSubscription(Subscription event) { + SubscriptionPayment payment = createSubscriptionPayment(event.getUsername(), event.getProductId()); + return new SubscriptionResult(payment.getId(), "success", 200); + } + } + + ``` + === "Example event" ```json @@ -200,25 +229,32 @@ You can quickly start by initializing the `DynamoDBPersistenceStore` and using i } ``` -#### Idempotent annotation on another method +#### Making non-handler methods idempotent -You can use the `@Idempotent` annotation for any synchronous Java function, not only the `handleRequest` one. +You can make any synchronous Java function idempotent, not only the `handleRequest` handler. -When using `@Idempotent` annotation on another method, you must tell which parameter in the method signature has the data we should use: +**With the `@Idempotent` annotation**, you must specify which parameter contains the idempotency key: - If the method only has one parameter, it will be used by default. - If there are 2 or more parameters, you must set the `@IdempotencyKey` on the parameter to use. +**With the functional API**, you explicitly pass the idempotency key: + + - For single-parameter methods, use `Idempotency.makeIdempotent(this::method, param, ReturnType.class)` + - For multi-parameter methods, use `Idempotency.makeIdempotent(idempotencyKey, () -> method(param1, param2), ReturnType.class)` + !!! info "The parameter must be serializable in JSON. We use Jackson internally to (de)serialize objects" -=== "AppSqsEvent.java" +=== "@Idempotent annotation" This example also demonstrates how you can integrate with [Batch utility](batch.md), so you can process each record in an idempotent manner. - ```java hl_lines="19 23-25 30-31" - public class AppSqsEvent implements RequestHandler { + ```java hl_lines="6-15 17-19 27-28" + public class SqsBatchHandler implements RequestHandler { + + private final BatchMessageHandler handler; - public AppSqsEvent() { + public SqsBatchHandler() { Idempotency.config() .withPersistenceStore( DynamoDBPersistenceStore.builder() @@ -226,31 +262,66 @@ When using `@Idempotent` annotation on another method, you must tell which param .build() ).withConfig( IdempotencyConfig.builder() - .withEventKeyJMESPath("messageId") // see Choosing a payload subset section + .withEventKeyJMESPath("messageId") .build() ).configure(); - } + + handler = new BatchMessageHandlerBuilder() + .withSqsBatchHandler() + .buildWithRawMessageHandler(this::processMessage); + } @Override - @SqsBatch(SampleMessageHandler.class) - public String handleRequest(SQSEvent input, Context context) { - dummy("hello", "world"); - return "{\"statusCode\": 200}"; + public SQSBatchResponse handleRequest(SQSEvent sqsEvent, Context context) { + return handler.processBatch(sqsEvent, context); } @Idempotent - private String dummy(String argOne, @IdempotencyKey String argTwo) { - return "something"; + private void processMessage(@IdempotencyKey SQSEvent.SQSMessage message) { + // Process message } + } + ``` + +=== "Functional API" + + This example also demonstrates how you can integrate with the [Batch utility](batch.md), so you can process each record in an idempotent manner. **Note: The JMESPath function still applies even when passing the idempotency key manually.** + + ```java hl_lines="6-15 17-19 24 29" + public class SqsBatchHandler implements RequestHandler { - public static class SampleMessageHandler implements SqsMessageHandler { - @Override - @Idempotent - // no need to use @IdempotencyKey as there is only one parameter - public String process(SQSMessage message) { - String returnVal = doSomething(message.getBody()); - return returnVal; - } + private final BatchMessageHandler handler; + + public SqsBatchHandler() { + Idempotency.config() + .withPersistenceStore( + DynamoDBPersistenceStore.builder() + .withTableName(System.getenv("TABLE_NAME")) + .build() + ).withConfig( + IdempotencyConfig.builder() + .withEventKeyJMESPath("messageId") + .build() + ).configure(); + + handler = new BatchMessageHandlerBuilder() + .withSqsBatchHandler() + .buildWithRawMessageHandler(this::processMessage); + } + + @Override + public SQSBatchResponse handleRequest(SQSEvent sqsEvent, Context context) { + Idempotency.registerLambdaContext(context); + return handler.processBatch(sqsEvent, context); + } + + private void processMessage(SQSEvent.SQSMessage message) { + Idempotency.makeIdempotent(this::handleMessage, message, Void.class); + } + + private Void handleMessage(SQSEvent.SQSMessage message) { + // Process message + return null; } } ``` @@ -304,9 +375,9 @@ Imagine the function executes successfully, but the client never receives the re To alter this behaviour, you can use the [JMESPath built-in function](serialization.md#jmespath-functions) `powertools_json()` to treat the payload as a JSON object rather than a string. -=== "PaymentFunction.java" +=== "@Idempotent annotation" - ```java hl_lines="5-7 16 29-31" + ```java hl_lines="7 16" public class PaymentFunction implements RequestHandler { public PaymentFunction() { @@ -344,6 +415,50 @@ Imagine the function executes successfully, but the client never receives the re } ``` +=== "Functional API" + + ```java hl_lines="7 17-18" + public class PaymentFunction implements RequestHandler { + + public PaymentFunction() { + Idempotency.config() + .withConfig( + IdempotencyConfig.builder() + .withEventKeyJMESPath("powertools_json(body)") + .build()) + .withPersistenceStore( + DynamoDBPersistenceStore.builder() + .withTableName(System.getenv("TABLE_NAME")) + .build()) + .configure(); + } + + public APIGatewayProxyResponseEvent handleRequest(final APIGatewayProxyRequestEvent event, final Context context) { + Idempotency.registerLambdaContext(context); + return Idempotency.makeIdempotent(this::processPayment, event, APIGatewayProxyResponseEvent.class); + } + + private APIGatewayProxyResponseEvent processPayment(APIGatewayProxyRequestEvent event) { + APIGatewayProxyResponseEvent response = new APIGatewayProxyResponseEvent(); + + try { + Subscription subscription = JsonConfig.get().getObjectMapper().readValue(event.getBody(), Subscription.class); + + SubscriptionPayment payment = createSubscriptionPayment( + subscription.getUsername(), + subscription.getProductId() + ); + + return response + .withStatusCode(200) + .withBody(String.format("{\"paymentId\":\"%s\"}", payment.getId())); + + } catch (JsonProcessingException e) { + return response.withStatusCode(500); + } + } + ``` + === "Example event" ```json hl_lines="3" @@ -417,46 +532,82 @@ The client was successful in receiving the result after the retry. Since the Lam #### Lambda timeouts -This is automatically done when you annotate your Lambda handler with [@Idempotent annotation](#idempotent-annotation). - To prevent against extended failed retries when a [Lambda function times out](https://aws.amazon.com/premiumsupport/knowledge-center/lambda-verify-invocation-timeouts/), Powertools for AWS Lambda (Java) calculates and includes the remaining invocation available time as part of the idempotency record. !!! example If a second invocation happens **after** this timestamp, and the record is marked as `INPROGRESS`, we will execute the invocation again as if it was in the `EXPIRED` state. This means that if an invocation expired during execution, it will be quickly executed again on the next retry. -!!! important - If you are using the [@Idempotent annotation on another method](#idempotent-annotation-on-another-method) to guard isolated parts of your code, you must use `registerLambdaContext` method available in the `Idempotency` object to benefit from this protection. +**With the `@Idempotent` annotation**, this is automatically done when you annotate your Lambda handler. + +**With the functional API** or when using the `@Idempotent` annotation on methods other than the handler, you must call `Idempotency.registerLambdaContext(context)` to benefit from this protection. +!!! important Here is an example on how you register the Lambda context in your handler: - ```java hl_lines="13-19" title="Registering the Lambda context" - public class PaymentHandler implements RequestHandler> { - - public PaymentHandler() { - Idempotency.config() - .withPersistenceStore( - DynamoDBPersistenceStore.builder() - .withTableName(System.getenv("TABLE_NAME")) - .build()) - .configure(); - } + === "@Idempotent annotation" + + ```java hl_lines="14" title="Registering the Lambda context" + public class PaymentHandler implements RequestHandler> { + + public PaymentHandler() { + Idempotency.config() + .withPersistenceStore( + DynamoDBPersistenceStore.builder() + .withTableName(System.getenv("TABLE_NAME")) + .build()) + .configure(); + } + + @Override + public List handleRequest(SQSEvent sqsEvent, Context context) { + Idempotency.registerLambdaContext(context); + return sqsEvent.getRecords().stream().map(record -> process(record.getMessageId(), record.getBody())).collect(Collectors.toList()); + } + + @Idempotent + private String process(String messageId, @IdempotencyKey String messageBody) { + logger.info("Processing messageId: {}", messageId); + PaymentRequest request = extractDataFrom(messageBody).as(PaymentRequest.class); + return paymentService.process(request); + } - @Override - public List handleRequest(SQSEvent sqsEvent, Context context) { - Idempotency.registerLambdaContext(context); - return sqsEvent.getRecords().stream().map(record -> process(record.getMessageId(), record.getBody())).collect(Collectors.toList()); } - - @Idempotent - private String process(String messageId, @IdempotencyKey String messageBody) { - logger.info("Processing messageId: {}", messageId); - PaymentRequest request = extractDataFrom(messageBody).as(PaymentRequest.class); - return paymentService.process(request); + ``` + + === "Functional API" + + ```java hl_lines="14" title="Registering the Lambda context" + public class PaymentHandler implements RequestHandler> { + + public PaymentHandler() { + Idempotency.config() + .withPersistenceStore( + DynamoDBPersistenceStore.builder() + .withTableName(System.getenv("TABLE_NAME")) + .build()) + .configure(); + } + + @Override + public List handleRequest(SQSEvent sqsEvent, Context context) { + Idempotency.registerLambdaContext(context); + return sqsEvent.getRecords().stream() + .map(record -> Idempotency.makeIdempotent( + record.getBody(), + () -> process(record.getMessageId(), record.getBody()), + String.class)) + .collect(Collectors.toList()); + } + + private String process(String messageId, String messageBody) { + logger.info("Processing messageId: {}", messageId); + PaymentRequest request = extractDataFrom(messageBody).as(PaymentRequest.class); + return paymentService.process(request); + } + } - - } - ``` + ``` #### Lambda timeout sequence diagram @@ -499,9 +650,11 @@ sequenceDiagram ### Handling exceptions -If you are using the `@Idempotent` annotation on your Lambda handler or any other method, any unhandled exceptions that are thrown during the code execution will cause **the record in the persistence layer to be deleted**. +**With the `@Idempotent` annotation**, any unhandled exceptions that are thrown during the code execution will cause **the record in the persistence layer to be deleted**. This means that new invocations will execute your code again despite having the same payload. If you don't want the record to be deleted, you need to catch exceptions within the idempotent function and return a successful response. +**With the functional API**, exceptions are handled the same way - unhandled exceptions will cause the record to be deleted. You should catch and handle exceptions within your idempotent function if you want to preserve the record. +
```mermaid sequenceDiagram @@ -553,7 +706,7 @@ If an Exception is raised _outside_ the scope of a decorated method and after yo This persistence store is built-in, and you can either use an existing DynamoDB table or create a new one dedicated for idempotency state (recommended). Use the builder to customize the table structure: -```java hl_lines="3-7" title="Customizing DynamoDBPersistenceStore to suit your table structure" +```java hl_lines="2-7" title="Customizing DynamoDBPersistenceStore to suit your table structure" DynamoDBPersistenceStore.builder() .withTableName(System.getenv("TABLE_NAME")) .withKeyAttr("idempotency_key") @@ -579,11 +732,68 @@ When using DynamoDB as a persistence layer, you can alter the attribute names by ## Advanced +### Using explicit function names + +When using the functional API, if you need to call different methods with the same payload as the idempotency key, you must provide explicit function names to differentiate between them. This ensures each function has its own idempotency scope. + +=== "Functional API with explicit names" + + ```java hl_lines="5-9 11-15" + public Response handleRequest(Order order, Context context) { + Idempotency.registerLambdaContext(context); + + // Same orderId, different operations - need explicit function names + Idempotency.makeIdempotent( + "processPayment", + order.getId(), + () -> processPayment(order), + PaymentResult.class); + + Idempotency.makeIdempotent( + "sendConfirmation", + order.getId(), + () -> sendEmail(order), + EmailResult.class); + + return new Response("success"); + } + ``` + +!!! note + When using the `@Idempotent` annotation, the function name is automatically inferred from the method name, so this is not needed. + +### Generic return types support + +The functional API supports making methods with generic return types idempotent using Jackson's `TypeReference`. This is not possible with the `@Idempotent` annotation due to type erasure. + +=== "Functional API with TypeReference" + + ```java hl_lines="1 6-10" + import com.fasterxml.jackson.core.type.TypeReference; + + public Map handleRequest(Product input, Context context) { + Idempotency.registerLambdaContext(context); + + return Idempotency.makeIdempotent( + this::processProduct, + input, + new TypeReference>() {} + ); + } + + private Map processProduct(Product product) { + // business logic returning generic type + Map result = new HashMap<>(); + // ... + return result; + } + ``` + ### Customizing the default behavior Idempotency behavior can be further configured with **`IdempotencyConfig`** using a builder: -```java hl_lines="2-8" title="Customizing IdempotencyConfig" +```java hl_lines="2-9" title="Customizing IdempotencyConfig" IdempotencyConfig.builder() .withEventKeyJMESPath("id") .withPayloadValidationJMESPath("paymentId") @@ -667,7 +877,7 @@ By default, we will return the same result as it returned before, however in thi With **`PayloadValidationJMESPath`**, you can provide an additional JMESPath expression to specify which part of the event body should be validated against previous idempotent invocations -=== "App.java" +=== "@Idempotent annotation" ```java hl_lines="8 13 20 26" public App() { @@ -700,6 +910,43 @@ With **`PayloadValidationJMESPath`**, you can provide an additional JMESPath exp } ``` +=== "Functional API" + + ```java hl_lines="8 14-15 24 30" + public App() { + Idempotency.config() + .withPersistenceStore(DynamoDBPersistenceStore.builder() + .withTableName(System.getenv("TABLE_NAME")) + .build()) + .withConfig(IdempotencyConfig.builder() + .withEventKeyJMESPath("[userDetail, productId]") + .withPayloadValidationJMESPath("amount") + .build()) + .configure(); + } + + public SubscriptionResult handleRequest(final Subscription input, final Context context) { + Idempotency.registerLambdaContext(context); + return Idempotency.makeIdempotent(this::processSubscription, input, SubscriptionResult.class); + } + + private SubscriptionResult processSubscription(Subscription input) { + // Creating a subscription payment is a side + // effect of calling this function! + SubscriptionPayment payment = createSubscriptionPayment( + input.getUserDetail().getUsername(), + input.getProductId(), + input.getAmount() + ) + // ... + return new SubscriptionResult( + "success", 200, + payment.getId(), + payment.getAmount() + ); + } + ``` + === "Example Event 1" ```json hl_lines="8" @@ -745,9 +992,9 @@ This means that we will throw **`IdempotencyKeyException`** if the evaluation of When set to `false` (the default), if the idempotency key is null, then the data is not persisted in the store. -=== "App.java" +=== "@Idempotent annotation" - ```java hl_lines="9-10 13" + ```java hl_lines="9" public App() { Idempotency.config() .withPersistenceStore(DynamoDBPersistenceStore.builder() @@ -767,6 +1014,32 @@ When set to `false` (the default), if the idempotency key is null, then the data } ``` +=== "Functional API" + + ```java hl_lines="9" + public App() { + Idempotency.config() + .withPersistenceStore(DynamoDBPersistenceStore.builder() + .withTableName(System.getenv("TABLE_NAME")) + .build()) + .withConfig(IdempotencyConfig.builder() + // Requires "user"."uid" and "orderId" to be present + .withEventKeyJMESPath("[user.uid, orderId]") + .withThrowOnNoIdempotencyKey(true) + .build()) + .configure(); + } + + public OrderResult handleRequest(final Order input, final Context context) { + Idempotency.registerLambdaContext(context); + return Idempotency.makeIdempotent(this::processOrder, input, OrderResult.class); + } + + private OrderResult processOrder(Order input) { + // ... + } + ``` + === "Success Event" ```json hl_lines="3 6" @@ -977,7 +1250,7 @@ To unit test your function with DynamoDB Local, you can refer to this guide to [ === "pom.xml" - ```xml hl_lines="4-6 24-26 28-31 42 45-47" + ```xml @@ -1046,7 +1319,7 @@ To unit test your function with DynamoDB Local, you can refer to this guide to [ === "AppTest.java" - ```java hl_lines="13-18 24-30 34" + ```java public class AppTest { @Mock private Context context; @@ -1143,7 +1416,7 @@ To unit test your function with DynamoDB Local, you can refer to this guide to [ === "App.java" - ```java hl_lines="8 9 16" + ```java public class App implements RequestHandler { public App() { @@ -1174,7 +1447,7 @@ To unit test your function with DynamoDB Local, you can refer to this guide to [ === "shell" - ```shell hl_lines="2 6 7 12 16 21 22" + ```shell # use or create a docker network docker network inspect sam-local || docker network create sam-local @@ -1201,7 +1474,7 @@ To unit test your function with DynamoDB Local, you can refer to this guide to [ === "env.json" - ```json hl_lines="3" + ```json { "IdempotentFunction": { "TABLE_NAME": "idempotency" diff --git a/docs/utilities/large_messages.md b/docs/utilities/large_messages.md index 38228afe9..9d14c8228 100644 --- a/docs/utilities/large_messages.md +++ b/docs/utilities/large_messages.md @@ -4,7 +4,7 @@ description: Utility --- The large message utility handles SQS and SNS messages which have had their payloads -offloaded to S3 if they are larger than the maximum allowed size (256 KB). +offloaded to S3 if they are larger than the maximum allowed size (1 MB). ## Features @@ -27,12 +27,12 @@ stateDiagram-v2 sendMsg --> extendLib state extendLib { state if_big <> - bigMsg: MessageBody > 256KB ? + bigMsg: MessageBody > 1MB ? putObject: putObject(S3Bucket, S3Key, Body) updateMsg: Update MessageBody
with a pointer to S3
and add a message attribute bigMsg --> if_big - if_big --> [*]: size(body) <= 256kb - if_big --> putObject: size(body) > 256kb + if_big --> [*]: size(body) <= 1MB + if_big --> putObject: size(body) > 1MB putObject --> updateMsg updateMsg --> [*] } @@ -72,7 +72,7 @@ stateDiagram-v2 ``` -SQS and SNS message payload is limited to 256KB. If you wish to send messages with a larger payload, you can leverage the +SQS and SNS message payload is limited to 1MB. If you wish to send messages with a larger payload, you can leverage the [amazon-sqs-java-extended-client-lib](https://github.com/awslabs/amazon-sqs-java-extended-client-lib) or [amazon-sns-java-extended-client-lib](https://github.com/awslabs/amazon-sns-java-extended-client-lib) which offload the message to Amazon S3. See documentation @@ -87,16 +87,14 @@ extended client libraries. Once a message's payload has been processed successfu utility deletes the payload from S3. This utility is compatible with -versions *[1.1.0+](https://github.com/awslabs/amazon-sqs-java-extended-client-lib/releases/tag/1.1.0)* -of amazon-sqs-java-extended-client-lib -and *[1.0.0+](https://github.com/awslabs/amazon-sns-java-extended-client-lib/releases/tag/1.0.0)* -of amazon-sns-java-extended-client-lib. +versions *[1.1.0+](https://github.com/awslabs/amazon-sqs-java-extended-client-lib/releases/tag/1.1.0)* and *[2.0.0+](https://github.com/awslabs/amazon-sqs-java-extended-client-lib/releases/tag/2.0.0)* +of [amazon-sqs-java-extended-client-lib](https://github.com/awslabs/amazon-sqs-java-extended-client-lib) / [amazon-sns-java-extended-client-lib](https://github.com/awslabs/amazon-sns-java-extended-client-lib). ## Install === "Maven" - ```xml hl_lines="3-7 16 18 24-27" + ```xml hl_lines="3-7 25-28" ... @@ -108,6 +106,7 @@ of amazon-sns-java-extended-client-lib. ... + ... @@ -152,7 +151,7 @@ of amazon-sns-java-extended-client-lib. ```groovy hl_lines="3 11" plugins { id 'java' - id 'io.freefair.aspectj.post-compile-weaving' version '8.1.0' + id 'io.freefair.aspectj.post-compile-weaving' version '8.1.0' // Not needed when using the functional approach } repositories { @@ -160,7 +159,7 @@ of amazon-sns-java-extended-client-lib. } dependencies { - aspect 'software.amazon.lambda:powertools-large-messages:{{ powertools.version }}' + aspect 'software.amazon.lambda:powertools-large-messages:{{ powertools.version }}' // Use 'implementation' instead of 'aspect' when using the functional approach } sourceCompatibility = 11 // or higher @@ -175,14 +174,20 @@ on the S3 bucket used for the large messages offloading: - `s3:GetObject` - `s3:DeleteObject` -## Annotation +## Usage -The annotation `@LargeMessage` can be used on any method where the *first* parameter is one of: +You can use the Large Messages utility with either the `@LargeMessage` annotation or the functional API. + +The `@LargeMessage` annotation can be used on any method where the *first* parameter is one of: - `SQSEvent.SQSMessage` - `SNSEvent.SNSRecord` -=== "SQS Example" +The functional API `LargeMessages.processLargeMessage()` accepts the same message types. + +### Basic usage + +=== "@LargeMessage annotation - SQS" ```java hl_lines="8 13 15" import software.amazon.lambda.powertools.largemessages.LargeMessage; @@ -204,7 +209,28 @@ The annotation `@LargeMessage` can be used on any method where the *first* param } ``` -=== "SNS Example" +=== "Functional API - SQS" + + ```java hl_lines="1 8" + import software.amazon.lambda.powertools.largemessages.LargeMessages; + + public class SqsMessageHandler implements RequestHandler { + + @Override + public SQSBatchResponse handleRequest(SQSEvent event, Context context) { + for (SQSMessage message: event.getRecords()) { + LargeMessages.processLargeMessage(message, this::processRawMessage); + } + return SQSBatchResponse.builder().build(); + } + + private void processRawMessage(SQSEvent.SQSMessage sqsMessage) { + // sqsMessage.getBody() will contain the content of the S3 object + } + } + ``` + +=== "@LargeMessage annotation - SNS" ```java hl_lines="7 11 13" import software.amazon.lambda.powertools.largemessages.LargeMessage; @@ -224,6 +250,25 @@ The annotation `@LargeMessage` can be used on any method where the *first* param } ``` +=== "Functional API - SNS" + + ```java hl_lines="1 7" + import software.amazon.lambda.powertools.largemessages.LargeMessages; + + public class SnsRecordHandler implements RequestHandler { + + @Override + public String handleRequest(SNSEvent event, Context context) { + return LargeMessages.processLargeMessage(event.records.get(0), this::processSNSRecord); + } + + private String processSNSRecord(SNSEvent.SNSRecord snsRecord) { + // snsRecord.getSNS().getMessage() will contain the content of the S3 object + return "Hello World"; + } + } + ``` + When the Lambda function is invoked with a SQS or SNS event, the utility first checks if the content was offloaded to S3. In the case of a large message, there is a message attribute specifying the size of the offloaded message and the message contains a pointer to the S3 object. @@ -233,9 +278,9 @@ and place the content of the object in the message payload. You can then directl If there was an error during the S3 download, the function will fail with a `LargeMessageProcessingException`. After your code is invoked and returns without error, the object is deleted from S3 -using the `deleteObject(bucket, key)` API. You can disable the deletion of S3 objects with the following configuration: +using the `deleteObject(bucket, key)` API. You can disable the deletion of S3 objects: -=== "Don't delete S3 Objects" +=== "@LargeMessage annotation" ```java @LargeMessage(deleteS3Object = false) private void processRawMessage(SQSEvent.SQSMessage sqsMessage) { @@ -243,71 +288,143 @@ using the `deleteObject(bucket, key)` API. You can disable the deletion of S3 ob } ``` +=== "Functional API" + ```java + LargeMessages.processLargeMessage(message, this::processRawMessage, false); + ``` + !!! tip "Use together with batch module" This utility works perfectly together with the batch module (`powertools-batch`), especially for SQS: - ```java hl_lines="2 5-7 12 15 16" title="Combining batch and large message modules" - public class SqsBatchHandler implements RequestHandler { - private final BatchMessageHandler handler; - - public SqsBatchHandler() { - handler = new BatchMessageHandlerBuilder() - .withSqsBatchHandler() - .buildWithRawMessageHandler(this::processMessage); - } + === "@LargeMessage annotation" + ```java hl_lines="2 5-7 12 15 16" title="Combining batch and large message modules" + public class SqsBatchHandler implements RequestHandler { + private final BatchMessageHandler handler; + + public SqsBatchHandler() { + handler = new BatchMessageHandlerBuilder() + .withSqsBatchHandler() + .buildWithRawMessageHandler(this::processMessage); + } - @Override - public SQSBatchResponse handleRequest(SQSEvent sqsEvent, Context context) { - return handler.processBatch(sqsEvent, context); + @Override + public SQSBatchResponse handleRequest(SQSEvent sqsEvent, Context context) { + return handler.processBatch(sqsEvent, context); + } + + @LargeMessage + private void processMessage(SQSEvent.SQSMessage sqsMessage) { + // do something with the message + } } + ``` - @LargeMessage - private void processMessage(SQSEvent.SQSMessage sqsMessage) { - // do something with the message + === "Functional API" + ```java hl_lines="7-9 14 18" title="Combining batch and large message modules" + import software.amazon.lambda.powertools.largemessages.LargeMessages; + + public class SqsBatchHandler implements RequestHandler { + private final BatchMessageHandler handler; + + public SqsBatchHandler() { + handler = new BatchMessageHandlerBuilder() + .withSqsBatchHandler() + .buildWithRawMessageHandler(this::processMessage); + } + + @Override + public SQSBatchResponse handleRequest(SQSEvent sqsEvent, Context context) { + return handler.processBatch(sqsEvent, context); + } + + private void processMessage(SQSEvent.SQSMessage sqsMessage) { + LargeMessages.processLargeMessage(sqsMessage, this::handleProcessedMessage); + } + + private void handleProcessedMessage(SQSEvent.SQSMessage processedMessage) { + // do something with the message + } } - } - ``` + ``` !!! tip "Use together with idempotency module" - This utility also works together with the idempotency module (`powertools-idempotency`). - You can add both the `@LargeMessage` and `@Idempotent` annotations, in any order, to the same method. - The `@Idempotent` takes precedence over the `@LargeMessage` annotation. - It means Idempotency module will use the initial raw message (containing the S3 pointer) and not the large message. + When using the `@LargeMessage` annotation, you can combine it with the `@Idempotent` annotation on the same method. + The `@Idempotent` takes precedence over the `@LargeMessage` annotation, meaning the Idempotency module will use the initial raw message (containing the S3 pointer) and not the large message. + + When using the functional API, call `LargeMessages.processLargeMessage()` from within the `@Idempotent` method to ensure idempotency is based on the S3 pointer, not the unwrapped large blob. - ```java hl_lines="6 23-25" title="Combining idempotency and large message modules" - public class SqsBatchHandler implements RequestHandler { + === "@LargeMessage annotation" + ```java hl_lines="6 23-25" title="Combining idempotency and large message modules" + public class SqsBatchHandler implements RequestHandler { - public SqsBatchHandler() { - Idempotency.config().withConfig( - IdempotencyConfig.builder() - .withEventKeyJMESPath("body") // get the body of the message for the idempotency key - .build()) - .withPersistenceStore( - DynamoDBPersistenceStore.builder() - .withTableName(System.getenv("IDEMPOTENCY_TABLE")) - .build() - ).configure(); - } + public SqsBatchHandler() { + Idempotency.config().withConfig( + IdempotencyConfig.builder() + .withEventKeyJMESPath("body") // get the body of the message for the idempotency key + .build()) + .withPersistenceStore( + DynamoDBPersistenceStore.builder() + .withTableName(System.getenv("IDEMPOTENCY_TABLE")) + .build() + ).configure(); + } - @Override - public SQSBatchResponse handleRequest(SQSEvent sqsEvent, Context context) { - for (SQSMessage message: event.getRecords()) { - processRawMessage(message, context); + @Override + public SQSBatchResponse handleRequest(SQSEvent sqsEvent, Context context) { + for (SQSMessage message: event.getRecords()) { + processRawMessage(message, context); + } + return SQSBatchResponse.builder().build(); + } + + @Idempotent + @LargeMessage + private String processRawMessage(@IdempotencyKey SQSEvent.SQSMessage sqsMessage, Context context) { + // do something with the message } - return SQSBatchResponse.builder().build(); } + ``` - @Idempotent - @LargeMessage - private String processRawMessage(@IdempotencyKey SQSEvent.SQSMessage sqsMessage, Context context) { - // do something with the message + === "Functional API" + ```java hl_lines="8 25 27" title="Combining idempotency and large message modules" + import software.amazon.lambda.powertools.largemessages.LargeMessages; + + public class SqsBatchHandler implements RequestHandler { + + public SqsBatchHandler() { + Idempotency.config().withConfig( + IdempotencyConfig.builder() + .withEventKeyJMESPath("body") // get the body of the message for the idempotency key + .build()) + .withPersistenceStore( + DynamoDBPersistenceStore.builder() + .withTableName(System.getenv("IDEMPOTENCY_TABLE")) + .build() + ).configure(); + } + + @Override + public SQSBatchResponse handleRequest(SQSEvent sqsEvent, Context context) { + for (SQSMessage message: event.getRecords()) { + processRawMessage(message, context); + } + return SQSBatchResponse.builder().build(); + } + + @Idempotent + private String processRawMessage(@IdempotencyKey SQSEvent.SQSMessage sqsMessage, Context context) { + return LargeMessages.processLargeMessage(sqsMessage, this::handleProcessedMessage); + } + + private String handleProcessedMessage(SQSEvent.SQSMessage processedMessage) { + // do something with the message + } } - } - ``` + ``` ## Customizing S3 client configuration -To interact with S3, the utility creates a default S3 Client : +To interact with S3, the utility creates a default S3 Client: === "Default S3 Client" ```java @@ -317,9 +434,9 @@ To interact with S3, the utility creates a default S3 Client : .build(); ``` -If you need to customize this `S3Client`, you can leverage the `LargeMessageConfig` singleton: +If you need to customize this `S3Client`, you can leverage the `LargeMessageConfig` singleton. This works with both the annotation and functional API: -=== "Custom S3 Client" +=== "@LargeMessage annotation" ```java hl_lines="6" import software.amazon.lambda.powertools.largemessages.LargeMessage; @@ -342,6 +459,28 @@ If you need to customize this `S3Client`, you can leverage the `LargeMessageConf } ``` +=== "Functional API" + ```java hl_lines="1 6" + import software.amazon.lambda.powertools.largemessages.LargeMessages; + + public class SnsRecordHandler implements RequestHandler { + + public SnsRecordHandler() { + LargeMessageConfig.init().withS3Client(/* put your custom S3Client here */); + } + + @Override + public String handleRequest(SNSEvent event, Context context) { + return LargeMessages.processLargeMessage(event.records.get(0), this::processSNSRecord); + } + + private String processSNSRecord(SNSEvent.SNSRecord snsRecord) { + // snsRecord.getSNS().getMessage() will contain the content of the S3 object + return "Hello World"; + } + } + ``` + ## Migration from the SQS Large Message utility - Replace the dependency in maven / gradle: `powertools-sqs` ==> `powertools-large-messages` diff --git a/docs/utilities/parameters.md b/docs/utilities/parameters.md index beb460aa6..6de47df68 100644 --- a/docs/utilities/parameters.md +++ b/docs/utilities/parameters.md @@ -49,6 +49,7 @@ Note that you must provide the concrete parameters module you want to use below ... + ... @@ -91,10 +92,10 @@ Note that you must provide the concrete parameters module you want to use below === "Gradle" - ```groovy hl_lines="3 11 12" + ```groovy hl_lines="3 11-13" plugins { id 'java' - id 'io.freefair.aspectj.post-compile-weaving' version '8.1.0' + id 'io.freefair.aspectj.post-compile-weaving' version '8.1.0' // Not needed when using provider classes directly (without annotations) } repositories { @@ -103,7 +104,8 @@ Note that you must provide the concrete parameters module you want to use below dependencies { // TODO! Provide the parameters module you want to use here - aspect 'software.amazon.lambda:powertools-parameters-secrets:{{ powertools.version }}' + aspect 'software.amazon.lambda:powertools-parameters-secrets:{{ powertools.version }}' // Not needed when using provider classes directly (without annotations) + implementation 'software.amazon.lambda:powertools-parameters-secrets:{{ powertools.version }}' // Use this instead of 'aspect' when using provider classes directly } sourceCompatibility = 11 // or higher @@ -124,19 +126,18 @@ This utility requires additional permissions to work as expected. See the table | AppConfig | `AppConfigProvider.get(String)` `AppConfigProvider.getMultiple(string)` | `appconfig:StartConfigurationSession`, `appConfig:GetLatestConfiguration` | ## Retrieving Parameters -You can retrieve parameters either using annotations or by using the `xParamProvider` class for each parameter -provider directly. The latter is useful if you need to configure the underlying SDK client, for example to use -a different region or credentials, the former is simpler to use. +You can retrieve parameters using either annotations or provider classes directly: + +- **Annotations** (e.g., `@SecretsParam`, `@SSMParam`) - Simpler syntax with field injection, but requires AspectJ configuration +- **Provider classes** (e.g., `SecretsProvider`, `SSMProvider`) - No AspectJ required, useful when you need to configure the underlying SDK client (e.g., different region or credentials), or prefer avoiding AspectJ setup ## Built-in provider classes -This section describes the built-in provider classes for each parameter store, providing -examples showing how to inject parameters using annotations, and how to use the provider -interface. In cases where a provider supports extra features, these will also be described. +This section describes the built-in provider classes for each parameter store. For each provider, examples are shown for both the annotation-based approach and the provider class approach. In cases where a provider supports extra features, these will also be described. ### Secrets Manager -=== "Secrets Manager: @SecretsParam" +=== "@SecretsParam annotation" ```java hl_lines="8 9" import com.amazonaws.services.lambda.runtime.Context; @@ -156,7 +157,7 @@ interface. In cases where a provider supports extra features, these will also be } ``` -=== "Secrets Manager: SecretsProvider" +=== "SecretsProvider class" ```java hl_lines="12-15 19" import static software.amazon.lambda.powertools.parameters.transform.Transformer.base64; @@ -195,7 +196,7 @@ The AWS Systems Manager Parameter Store provider supports two additional argumen | **recursive()** | `False` | For `getMultiple()` only, will fetch all parameter values recursively based on a path prefix. | -=== "SSM Parameter Store: @SSMParam" +=== "@SSMParam annotation" ```java hl_lines="8 9" import com.amazonaws.services.lambda.runtime.Context; @@ -214,7 +215,7 @@ The AWS Systems Manager Parameter Store provider supports two additional argumen } ``` -=== "SSM Parameter Store: SSMProvider" +=== "SSMProvider class" ```java hl_lines="12-15 19-20 22" import static software.amazon.lambda.powertools.parameters.transform.Transformer.base64; @@ -246,15 +247,14 @@ The AWS Systems Manager Parameter Store provider supports two additional argumen } ``` -=== "SSM Parameter Store: Additional Options" +=== "Additional Options" - ```java hl_lines="9 12" - import software.amazon.lambda.powertools.parameters.SSMProvider; - import software.amazon.lambda.powertools.parameters.ParamManager; + ```java hl_lines="5 9 12" + import software.amazon.lambda.powertools.parameters.ssm.SSMProvider; public class AppWithSSM implements RequestHandler { // Get an instance of the SSM Provider - SSMProvider ssmProvider = ParamManager.getSsmProvider(); + SSMProvider ssmProvider = SSMProvider.builder().build(); // Retrieve a single parameter and decrypt it String value = ssmProvider.withDecryption().get("/my/parameter"); @@ -267,7 +267,7 @@ The AWS Systems Manager Parameter Store provider supports two additional argumen ### DynamoDB -=== "DynamoDB: @DyanmoDbParam" +=== "@DynamoDbParam annotation" ```java hl_lines="8 9" import com.amazonaws.services.lambda.runtime.Context; @@ -286,7 +286,7 @@ The AWS Systems Manager Parameter Store provider supports two additional argumen } ``` -=== "DynamoDB: DynamoDbProvider" +=== "DynamoDbProvider class" ```java hl_lines="12-15 19-20 22" import static software.amazon.lambda.powertools.parameters.transform.Transformer.base64; @@ -320,7 +320,7 @@ The AWS Systems Manager Parameter Store provider supports two additional argumen ### AppConfig -=== "AppConfig: @AppConfigParam" +=== "@AppConfigParam annotation" ```java hl_lines="8 9" import com.amazonaws.services.lambda.runtime.Context; @@ -339,7 +339,7 @@ The AWS Systems Manager Parameter Store provider supports two additional argumen } ``` -=== "AppConfig: AppConfigProvider" +=== "AppConfigProvider class" ```java hl_lines="12-15 19-20" import static software.amazon.lambda.powertools.parameters.transform.Transformer.base64; diff --git a/docs/utilities/validation.md b/docs/utilities/validation.md index ec35b7034..8e0d2c631 100644 --- a/docs/utilities/validation.md +++ b/docs/utilities/validation.md @@ -26,6 +26,7 @@ This utility provides JSON Schema validation for payloads held within events and ... + ... @@ -67,10 +68,10 @@ This utility provides JSON Schema validation for payloads held within events and === "Gradle" - ```groovy hl_lines="3 11" + ```groovy hl_lines="3 11 12" plugins { id 'java' - id 'io.freefair.aspectj.post-compile-weaving' version '8.1.0' + id 'io.freefair.aspectj.post-compile-weaving' version '8.1.0' // Not needed when using the functional approach with ValidationUtils.validate() } repositories { @@ -78,7 +79,8 @@ This utility provides JSON Schema validation for payloads held within events and } dependencies { - aspect 'software.amazon.lambda:powertools-validation:{{ powertools.version }}' + aspect 'software.amazon.lambda:powertools-validation:{{ powertools.version }}' // Not needed when using the functional approach with ValidationUtils.validate() + implementation 'software.amazon.lambda:powertools-validation:{{ powertools.version }}' // Use this instead of 'aspect' when using the functional approach } sourceCompatibility = 11 // or higher @@ -87,17 +89,18 @@ This utility provides JSON Schema validation for payloads held within events and ## Validating events -You can validate inbound and outbound events using `@Validation` annotation. +You can validate inbound and outbound events using either the `@Validation` annotation or the functional approach with `ValidationUtils.validate()` methods: -You can also use the `Validator#validate()` methods, if you want more control over the validation process such as handling a validation error. +- **@Validation annotation** - Simpler syntax with automatic validation, but requires AspectJ configuration +- **ValidationUtils.validate()** - No AspectJ required, provides more control over the validation process such as handling validation errors -We support JSON schema version 4, 6, 7, 2019-09 and 2020-12 using the [NetworkNT JSON Schema Validator](https://github.com/networknt/json-schema-validator). ([Compatibility with JSON Schema versions](https://github.com/networknt/json-schema-validator/blob/master/doc/compatibility.md)). +We support JSON schema version 4, 6, 7, 2019-09 and 2020-12 using the [NetworkNT JSON Schema Validator](https://github.com/networknt/json-schema-validator) ([Compatibility with JSON Schema versions](https://github.com/networknt/json-schema-validator/blob/master/doc/compatibility.md)). The validator is configured to enable format assertions by default even for 2019-09 and 2020-12. ### Validation annotation -`@Validation` annotation is used to validate either inbound events or functions' response. +The `@Validation` annotation is used to validate either inbound events or functions' response. It will fail fast if an event or response doesn't conform with given JSON Schema. For most type of events a `ValidationException` will be thrown. @@ -129,11 +132,11 @@ While it is easier to specify a json schema file in the classpath (using the not **NOTE**: It's not a requirement to validate both inbound and outbound schemas - You can either use one, or both. -### Validate function +### Functional approach with ValidationUtils -Validate standalone function is used within the Lambda handler, or any other methods that perform data validation. +The `ValidationUtils.validate()` method provides a functional approach that can be used within the Lambda handler or any other methods that perform data validation. This approach does not require AspectJ configuration. -You can also gracefully handle schema validation errors by catching `ValidationException`. +With this approach, you can gracefully handle schema validation errors by catching `ValidationException`. === "MyFunctionHandler.java" diff --git a/mkdocs.yml b/mkdocs.yml index da4303c38..3914cfa1a 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -1,9 +1,10 @@ site_name: Powertools for AWS Lambda (Java) site_description: Powertools for AWS Lambda (Java) site_author: Amazon Web Services -site_url: https://docs.powertools.aws.dev/lambda/java/latest/ +site_url: https://docs.aws.amazon.com/powertools/java/latest/ nav: - Homepage: index.md + - Usage patterns: usage-patterns.md - Changelog: changelog.md - Upgrade Guide: upgrade.md - FAQs: FAQs.md @@ -99,6 +100,7 @@ plugins: sections: Project Overview: - index.md + - usage-patterns.md - changelog.md - FAQs.md - roadmap.md