Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(runtime): add Python 3.13 support #5527

Merged
merged 19 commits into from
Nov 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ body:
- "3.10"
- "3.11"
- "3.12"
- "3.13"
validations:
required: true
- type: dropdown
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/static_typing.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ body:
- "3.10"
- "3.11"
- "3.12"
- "3.13"
validations:
required: true
- type: input
Expand Down
13 changes: 8 additions & 5 deletions .github/workflows/layer_govcloud.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ jobs:
- AWSLambdaPowertoolsPythonV3-python310
- AWSLambdaPowertoolsPythonV3-python311
- AWSLambdaPowertoolsPythonV3-python312
- AWSLambdaPowertoolsPythonV3-python313
arch:
- arm64
- x86_64
Expand Down Expand Up @@ -94,6 +95,7 @@ jobs:
- AWSLambdaPowertoolsPythonV3-python310
- AWSLambdaPowertoolsPythonV3-python311
- AWSLambdaPowertoolsPythonV3-python312
- AWSLambdaPowertoolsPythonV3-python313
arch:
- arm64
- x86_64
Expand All @@ -110,7 +112,7 @@ jobs:
- name: Verify Layer Signature
run: |
SHA=$(jq -r '.Content.CodeSha256' '${{ matrix.layer }}_${{ matrix.arch }}.json')
test "$(openssl dgst -sha256 -binary ${{ matrix.layer }}_${{ matrix.arch }}.zip | openssl enc -base64)" == "$SHA" && echo "SHA OK: ${SHA}" || exit 1
test "$(openssl dgst -sha256 -binary ${{ matrix.layer }}_${{ matrix.arch }}.zip | openssl enc -base64)" == "$SHA" && echo "SHA OK: ${SHA}" || exit 1
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@e3dd6a429d7300a6a4c196c26e071d42e0343502 # v4.0.2
with:
Expand Down Expand Up @@ -144,7 +146,7 @@ jobs:
run: |
REMOTE_SHA=$(aws --region us-gov-east-1 lambda get-layer-version-by-arn --arn 'arn:aws-us-gov:lambda:us-gov-east-1:${{ secrets.AWS_ACCOUNT_ID }}:layer:${{ matrix.layer }}-${{ matrix.arch }}:${{ env.LAYER_VERSION }}' --query 'Content.CodeSha256' --output text)
SHA=$(jq -r '.Content.CodeSha256' '${{ matrix.layer }}_${{ matrix.arch }}.json')
test "$REMOTE_SHA" == "$SHA" && echo "SHA OK: ${SHA}" || exit 1
test "$REMOTE_SHA" == "$SHA" && echo "SHA OK: ${SHA}" || exit 1
aws --region us-gov-east-1 lambda get-layer-version-by-arn --arn 'arn:aws-us-gov:lambda:us-gov-east-1:${{ secrets.AWS_ACCOUNT_ID }}:layer:${{ matrix.layer }}-${{ matrix.arch }}:${{ env.LAYER_VERSION }}' --output table
copy_west:
Expand All @@ -162,6 +164,7 @@ jobs:
- AWSLambdaPowertoolsPythonV3-python310
- AWSLambdaPowertoolsPythonV3-python311
- AWSLambdaPowertoolsPythonV3-python312
- AWSLambdaPowertoolsPythonV3-python313
arch:
- arm64
- x86_64
Expand All @@ -179,7 +182,7 @@ jobs:
- name: Verify Layer Signature
run: |
SHA=$(jq -r '.Content.CodeSha256' '${{ matrix.layer }}_${{ matrix.arch }}.json')
test "$(openssl dgst -sha256 -binary ${{ matrix.layer }}_${{ matrix.arch }}.zip | openssl enc -base64)" == "$SHA" && echo "SHA OK: ${SHA}" || exit 1
test "$(openssl dgst -sha256 -binary ${{ matrix.layer }}_${{ matrix.arch }}.zip | openssl enc -base64)" == "$SHA" && echo "SHA OK: ${SHA}" || exit 1
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@e3dd6a429d7300a6a4c196c26e071d42e0343502 # v4.0.2
with:
Expand Down Expand Up @@ -213,5 +216,5 @@ jobs:
run: |
REMOTE_SHA=$(aws --region us-gov-west-1 lambda get-layer-version-by-arn --arn 'arn:aws-us-gov:lambda:us-gov-west-1:${{ secrets.AWS_ACCOUNT_ID }}:layer:${{ matrix.layer }}-${{ matrix.arch }}:${{ env.LAYER_VERSION }}' --query 'Content.CodeSha256' --output text)
SHA=$(jq -r '.Content.CodeSha256' '${{ matrix.layer }}_${{ matrix.arch }}.json')
test "$REMOTE_SHA" == "$SHA" && echo "SHA OK: ${SHA}" || exit 1
aws --region us-gov-west-1 lambda get-layer-version-by-arn --arn 'arn:aws-us-gov:lambda:us-gov-west-1:${{ secrets.AWS_ACCOUNT_ID }}:layer:${{ matrix.layer }}-${{ matrix.arch }}:${{ env.LAYER_VERSION }}' --output table
test "$REMOTE_SHA" == "$SHA" && echo "SHA OK: ${SHA}" || exit 1
aws --region us-gov-west-1 lambda get-layer-version-by-arn --arn 'arn:aws-us-gov:lambda:us-gov-west-1:${{ secrets.AWS_ACCOUNT_ID }}:layer:${{ matrix.layer }}-${{ matrix.arch }}:${{ env.LAYER_VERSION }}' --output table
3 changes: 3 additions & 0 deletions .github/workflows/layer_govcloud_verify.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ jobs:
- AWSLambdaPowertoolsPythonV3-python310
- AWSLambdaPowertoolsPythonV3-python311
- AWSLambdaPowertoolsPythonV3-python312
- AWSLambdaPowertoolsPythonV3-python313
arch:
- arm64
- x86_64
Expand Down Expand Up @@ -63,6 +64,7 @@ jobs:
- AWSLambdaPowertoolsPythonV3-python310
- AWSLambdaPowertoolsPythonV3-python311
- AWSLambdaPowertoolsPythonV3-python312
- AWSLambdaPowertoolsPythonV3-python313
arch:
- arm64
- x86_64
Expand Down Expand Up @@ -94,6 +96,7 @@ jobs:
- AWSLambdaPowertoolsPythonV3-python310
- AWSLambdaPowertoolsPythonV3-python311
- AWSLambdaPowertoolsPythonV3-python312
- AWSLambdaPowertoolsPythonV3-python313
arch:
- arm64
- x86_64
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/publish_v3_layer.yml
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ jobs:
strategy:
max-parallel: 5
matrix:
python-version: ["3.8","3.9","3.10","3.11","3.12"]
python-version: ["3.8","3.9","3.10","3.11","3.12","3.13"]
defaults:
run:
working-directory: ./layer_v3
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/quality_check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ jobs:
strategy:
max-parallel: 4
matrix:
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
python-version: ["3.8","3.9","3.10","3.11","3.12","3.13"]
env:
PYTHON: "${{ matrix.python-version }}"
permissions:
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/quality_code_cdk_constructor.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,25 +15,25 @@ name: Code quality - CDK constructor
on:
pull_request:
paths:
- "layer/layer_constructors/**"
- "layer_v3/layer_constructors/**"
branches:
- develop
push:
paths:
- "layer/layer_constructors/**"
- "layer_v3/layer_constructors/**"
branches:
- develop

permissions:
contents: read

jobs:
quality_check:
quality_check_cdk:
runs-on: ubuntu-latest
strategy:
max-parallel: 4
matrix:
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
python-version: ["3.12"]
env:
PYTHON: "${{ matrix.python-version }}"
permissions:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/reusable_deploy_v3_layer_stack.yml
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ jobs:
"eu-north-1", "eu-south-1", "eu-south-2", "eu-west-1", "eu-west-2", "eu-west-3",
"il-central-1", "me-central-1", "me-south-1", "sa-east-1", "us-east-1",
"us-east-2", "us-west-1", "us-west-2"]
python-version: ["3.8","3.9","3.10","3.11","3.12"]
python-version: ["3.8","3.9","3.10","3.11","3.12","3.13"]
include:
- region: "af-south-1"
has_arm64_support: "true"
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/reusable_deploy_v3_sar.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ name: Deploy V3 SAR
#
# 1. This workflow starts after the layer artifact is produced on `publish_v3_layer`
# 2. We use the same layer artifact to ensure the SAR app is consistent with the published Lambda Layer
# 3. We publish the SAR for 3.8 to 3.12 Python runtime and both x86_64 and arm64 (see `matrix` section)
# 3. We publish the SAR for 3.8 to 3.13 Python runtime and both x86_64 and arm64 (see `matrix` section)
# 4. We use `sam package` and `sam publish` to publish the SAR app
# 5. We remove the previous Canary stack (if present) and deploy a new one to test the SAR App. We retain the Canary in the account for debugging purposes
# 6. Finally the published SAR app is made public on the PROD environment
Expand Down Expand Up @@ -72,7 +72,7 @@ jobs:
strategy:
matrix:
architecture: ["x86_64", "arm64"]
python-version: ["3.8","3.9","3.10","3.11","3.12"]
python-version: ["3.8","3.9","3.10","3.11","3.12","3.13"]
steps:
- name: checkout
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/run-e2e-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ jobs:
strategy:
fail-fast: false # needed so if a version fails, the others will still be able to complete and cleanup
matrix:
version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
version: ["3.8", "3.9", "3.10", "3.11", "3.12","3.13"]
if: ${{ github.actor != 'dependabot[bot]' && github.repository == 'aws-powertools/powertools-lambda-python' }}
steps:
- name: "Checkout"
Expand Down
1 change: 0 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ target:

dev:
pip install --upgrade pip pre-commit poetry
poetry config --local virtualenvs.in-project true
@$(MAKE) dev-version-plugin
poetry install --extras "all redis datamasking"
pre-commit install
Expand Down
2 changes: 1 addition & 1 deletion aws_lambda_powertools/event_handler/openapi/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -390,7 +390,7 @@ class HTTPBase(SecurityBase):
scheme: str


class HTTPBearer(HTTPBase):
class HTTPBearer(HTTPBase): # type: ignore[override]
scheme: Literal["bearer"] = "bearer"
bearerFormat: Optional[str] = None

Expand Down
2 changes: 1 addition & 1 deletion aws_lambda_powertools/utilities/parameters/dynamodb.py
Original file line number Diff line number Diff line change
Expand Up @@ -230,4 +230,4 @@ def _get_multiple(self, path: str, **sdk_options) -> dict[str, str]:

# maintenance: look for better ways to correctly type DynamoDB multiple return types
# without a breaking change within ABC return type
return {item[self.sort_attr]: item[self.value_attr] for item in items}
return {item[self.sort_attr]: item[self.value_attr] for item in items} # type: ignore[misc]
Original file line number Diff line number Diff line change
Expand Up @@ -14,16 +14,16 @@ class CloudFormationCustomResourceBaseModel(BaseModel):
resource_properties: Union[Dict[str, Any], BaseModel, None] = Field(None, alias="ResourceProperties")


class CloudFormationCustomResourceCreateModel(CloudFormationCustomResourceBaseModel):
class CloudFormationCustomResourceCreateModel(CloudFormationCustomResourceBaseModel): # type: ignore[override]
request_type: Literal["Create"] = Field(..., alias="RequestType")


class CloudFormationCustomResourceDeleteModel(CloudFormationCustomResourceBaseModel):
class CloudFormationCustomResourceDeleteModel(CloudFormationCustomResourceBaseModel): # type: ignore[override]
request_type: Literal["Delete"] = Field(..., alias="RequestType")
physical_resource_id: str = Field(..., alias="PhysicalResourceId")


class CloudFormationCustomResourceUpdateModel(CloudFormationCustomResourceBaseModel):
class CloudFormationCustomResourceUpdateModel(CloudFormationCustomResourceBaseModel): # type: ignore[override]
request_type: Literal["Update"] = Field(..., alias="RequestType")
physical_resource_id: str = Field(..., alias="PhysicalResourceId")
old_resource_properties: Union[Dict[str, Any], BaseModel, None] = Field(None, alias="OldResourceProperties")
2 changes: 1 addition & 1 deletion aws_lambda_powertools/utilities/parser/models/s3.py
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ class S3EventNotificationEventBridgeDetailModel(BaseModel):
destination_access_tier: Optional[str] = Field(None, alias="destination-access-tier")


class S3EventNotificationEventBridgeModel(EventBridgeModel):
class S3EventNotificationEventBridgeModel(EventBridgeModel): # type: ignore[override]
detail: S3EventNotificationEventBridgeDetailModel


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@
from aws_lambda_powertools.utilities.parser.models.sqs import SqsModel, SqsRecordModel


class S3SqsEventNotificationRecordModel(SqsRecordModel):
class S3SqsEventNotificationRecordModel(SqsRecordModel): # type: ignore[override]
body: Json[S3Model]


class S3SqsEventNotificationModel(SqsModel):
class S3SqsEventNotificationModel(SqsModel): # type: ignore[override]
Records: List[S3SqsEventNotificationRecordModel]
8 changes: 7 additions & 1 deletion docs/automation.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ This is a snapshot of our automated checks at a glance.
To build and deploy the Lambda Layers, we run a pipeline with the following steps:

* We fetch the latest PyPi release and use it as the source for our layer.
* We build Python versions ranging from **3.8 to 3.12** for x86_64 and arm64 architectures. This is necessary because we use pre-compiled libraries like **Pydantic** and **Cryptography**, which require specific Python versions for each layer.
* We build Python versions ranging from **3.8 to 3.13** for x86_64 and arm64 architectures. This is necessary because we use pre-compiled libraries like **Pydantic** and **Cryptography**, which require specific Python versions for each layer.
* We provide layer distributions for both the **x86_64** and **arm64** architectures.
* For each Python version, we create a single CDK package containing both x86_64 and arm64 assets to optimize deployment performance.

Expand All @@ -111,6 +111,7 @@ graph LR
Fetch --> P310[<strong>Python 3.10</strong>]
Fetch --> P311[<strong>Python 3.11</strong>]
Fetch --> P312[<strong>Python 3.12</strong>]
Fetch --> P313[<strong>Python 3.13</strong>]
subgraph build ["LAYER BUILD"]
P38 --> P38x86[build x86_64]
Expand All @@ -124,6 +125,8 @@ graph LR
P311 --> P311arm64[build arm64]
P312 --> P312x86[build x86_64]
P312 --> P312arm64[build arm64]
P313 --> P313x86[build x86_64]
P313 --> P313arm64[build arm64]
P38x86 --> CDKP1[CDK Package]
P38arm64 --> CDKP1[CDK Package]
P39x86 --> CDKP2[CDK Package]
Expand All @@ -134,6 +137,8 @@ graph LR
P311arm64 --> CDKP4[CDK Package]
P312x86 --> CDKP5[CDK Package]
P312arm64 --> CDKP5[CDK Package]
P313x86 --> CDKP6[CDK Package]
P313arm64 --> CDKP6[CDK Package]
end
subgraph beta ["BETA (all regions)"]
Expand All @@ -142,6 +147,7 @@ graph LR
CDKP3 --> DeployBeta
CDKP4 --> DeployBeta
CDKP5 --> DeployBeta
CDKP6 --> DeployBeta
DeployBeta --> RunBetaCanary["Beta canary tests<br> <i>(all packages)</i>"]
end
subgraph prod ["PROD (all regions)"]
Expand Down
32 changes: 32 additions & 0 deletions docs/includes/_layer_homepage_arm64.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,3 +160,35 @@
| **`us-east-2`** | **arn:aws:lambda:us-east-2:017000801446:layer:AWSLambdaPowertoolsPythonV3-python312-arm64:3**{: .copyMe}:clipboard: |
| **`us-west-1`** | **arn:aws:lambda:us-west-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python312-arm64:3**{: .copyMe}:clipboard: |
| **`us-west-2`** | **arn:aws:lambda:us-west-2:017000801446:layer:AWSLambdaPowertoolsPythonV3-python312-arm64:3**{: .copyMe}:clipboard: |

=== "Python 3.13"

| Region | Layer ARN |
| -------------------- | --------------------------------------------------------------------------------------------------------------- |
| **`af-south-1`** | **arn:aws:lambda:af-south-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`ap-east-1`** | **arn:aws:lambda:ap-east-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`ap-northeast-1`** | **arn:aws:lambda:ap-northeast-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`ap-northeast-2`** | **arn:aws:lambda:ap-northeast-2:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`ap-northeast-3`** | **arn:aws:lambda:ap-northeast-3:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`ap-south-1`** | **arn:aws:lambda:ap-south-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`ap-south-2`** | **arn:aws:lambda:ap-south-2:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`ap-southeast-1`** | **arn:aws:lambda:ap-southeast-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`ap-southeast-2`** | **arn:aws:lambda:ap-southeast-2:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`ap-southeast-3`** | **arn:aws:lambda:ap-southeast-3:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`ca-central-1`** | **arn:aws:lambda:ca-central-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`eu-central-1`** | **arn:aws:lambda:eu-central-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`eu-central-2`** | **arn:aws:lambda:eu-central-2:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`eu-north-1`** | **arn:aws:lambda:eu-north-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`eu-south-1`** | **arn:aws:lambda:eu-south-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`eu-south-2`** | **arn:aws:lambda:eu-south-2:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`eu-west-1`** | **arn:aws:lambda:eu-west-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`eu-west-2`** | **arn:aws:lambda:eu-west-2:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`eu-west-3`** | **arn:aws:lambda:eu-west-3:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`il-central-1`** | **arn:aws:lambda:il-central-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`me-central-1`** | **arn:aws:lambda:me-central-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`me-south-1`** | **arn:aws:lambda:me-south-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`sa-east-1`** | **arn:aws:lambda:sa-east-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`us-east-1`** | **arn:aws:lambda:us-east-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`us-east-2`** | **arn:aws:lambda:us-east-2:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`us-west-1`** | **arn:aws:lambda:us-west-1:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
| **`us-west-2`** | **arn:aws:lambda:us-west-2:017000801446:layer:AWSLambdaPowertoolsPythonV3-python313-arm64:3**{: .copyMe}:clipboard: |
Loading