Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Python: Allow injecting Lambda Context #1985

Merged
merged 22 commits into from
Dec 12, 2022
Merged

Python: Allow injecting Lambda Context #1985

merged 22 commits into from
Dec 12, 2022

Conversation

unexge
Copy link
Contributor

@unexge unexge commented Nov 15, 2022 •
edited
Loading

Motivation and Context

This PR allows users to inject Lambda context to their custom context class by type-hinting LambdaContext:

from typing import Optional
from libpokemon_service_server_sdk.aws_lambda import LambdaContext

@dataclass
class Context:
    # Inject Lambda context if service is running on Lambda
    # NOTE: All the values that will be injected by the framework should be wrapped with `Optional`
    lambda_ctx: Optional[LambdaContext] = None

    ...

app.context(Context())

@app.get_pokemon_species
def get_pokemon_species(
    input: GetPokemonSpeciesInput, context: Context
) -> GetPokemonSpeciesOutput:
    if context.lambda_ctx is not None:
        logging.debug(
            "Lambda Context: %s",
            dict(
                request_id=context.lambda_ctx.request_id,
                deadline=context.lambda_ctx.deadline,
                invoked_function_arn=context.lambda_ctx.invoked_function_arn,
                function_name=context.lambda_ctx.env_config.function_name,
                memory=context.lambda_ctx.env_config.memory,
                version=context.lambda_ctx.env_config.version,
            ),
        )
    ...

Closes #1815

Description

This PR introduces PyContext class to wrap raw Python object provided by the users. At the startup we inspect raw Python object to detect injectable fields, after that AddPyContextLayer (a Tower layer), populates PyContext from incoming request (i.e. creates PyLambdaContext from LambdaContext) and then adds PyContext to request extensions, which then gets injected by the handlers and then passed to Python handler (thanks to ToPyObject implementation of PyContext).

TODOs

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@unexge unexge added the python-server Python server SDK label Nov 15, 2022
@unexge unexge requested a review from a team as a code owner November 15, 2022 10:24
@github-actions
Copy link

A new generated diff is ready to view.

A new doc preview is ready to view.

@unexge unexge force-pushed the unexge/python-lambda-ctx branch from 55713da to e5d0f8e Compare November 15, 2022 11:13
@github-actions
Copy link

A new generated diff is ready to view.

A new doc preview is ready to view.

@github-actions
Copy link

A new generated diff is ready to view.

A new doc preview is ready to view.

@unexge unexge force-pushed the unexge/python-lambda-ctx branch from 5494991 to 75bd902 Compare November 15, 2022 13:07
@github-actions
Copy link

A new generated diff is ready to view.

A new doc preview is ready to view.

crisidev
crisidev approved these changes Nov 25, 2022
View reviewed changes
Copy link
Contributor

@crisidev crisidev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

rust-runtime/aws-smithy-http-server-python/src/context.rs Outdated Show resolved Hide resolved
crisidev
crisidev reviewed Nov 25, 2022
View reviewed changes
rust-runtime/aws-smithy-http-server-python/examples/pokemon_service.py
Comment on lines +82 to +86
# Inject Lambda context if service is running on Lambda
# NOTE: All the values that will be injected by the framework should be wrapped with `Optional`
lambda_ctx: Optional[LambdaContext] = None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for my understanding: if the lambda_ctx of type Optional[LambdaContext] is not defined here, the injection will be just avoided right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that's correct

hlbarber
hlbarber approved these changes Nov 25, 2022
View reviewed changes
Copy link
Contributor

@hlbarber hlbarber left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice documentation and test coverage

rust-runtime/aws-smithy-http-server-python/examples/pokemon_service.py
@@ -195,6 +200,18 @@ def do_nothing(_: DoNothingInput) -> DoNothingOutput:
def get_pokemon_species(
input: GetPokemonSpeciesInput, context: Context
) -> GetPokemonSpeciesOutput:
if context.lambda_ctx is not None:
logging.debug(
"Lambda Context: %s",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logging in python is capitalized?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK there is not a consensus. I have always used capitalized logging in Python, but I don't have a reason for it.

rust-runtime/aws-smithy-http-server-python/src/context/testing.rs Outdated Show resolved Hide resolved
rust-runtime/aws-smithy-http-server-python/src/context/layer.rs Outdated
let py_lambda_ctx = req
.extensions()
.get::<LambdaContext>()
.cloned()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to .cloned() this? What's the difference if we .remove::<LambdaContext>()?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't want to use remove because other layers might want to use LambdaContext, is it a good practice to remove extensions?

rust-runtime/aws-smithy-http-server-python/examples/pokemon_service.py
if context.lambda_ctx is not None:
logging.debug(
"Lambda Context: %s",
dict(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a shame there's no interface on the logging package to do structured logging, just serializing this big dict seems wrong.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is, but it is something the customer should provide through their own logging format (I have used a lot JSON lines in the past).

So to be as compatible as possible, we just log the entire dict.

rust-runtime/aws-smithy-http-server-python/src/server.rs Show resolved Hide resolved
@unexge unexge force-pushed the unexge/python-lambda-ctx branch from 75bd902 to 27d25e9 Compare November 28, 2022 16:05
@github-actions
Copy link

A new generated diff is ready to view.

A new doc preview is ready to view.

@unexge unexge requested a review from a team November 29, 2022 15:07
@unexge unexge force-pushed the unexge/python-lambda-ctx branch from 27d25e9 to 7e063e4 Compare December 6, 2022 15:10
@github-actions
Copy link

github-actions bot commented Dec 6, 2022

A new generated diff is ready to view.

A new doc preview is ready to view.

@unexge unexge force-pushed the unexge/python-lambda-ctx branch from 7e063e4 to 152b8e2 Compare December 9, 2022 11:49
@github-actions
Copy link

github-actions bot commented Dec 9, 2022

A new generated diff is ready to view.

A new doc preview is ready to view.

@github-actions
Copy link

github-actions bot commented Dec 9, 2022

A new generated diff is ready to view.

A new doc preview is ready to view.

@github-actions
Copy link

github-actions bot commented Dec 9, 2022

A new generated diff is ready to view.

A new doc preview is ready to view.

@unexge unexge force-pushed the unexge/python-lambda-ctx branch from 34703cb to bfce15f Compare December 9, 2022 14:39
@github-actions
Copy link

github-actions bot commented Dec 9, 2022

A new generated diff is ready to view.

A new doc preview is ready to view.

@github-actions
Copy link

github-actions bot commented Dec 9, 2022

A new generated diff is ready to view.

A new doc preview is ready to view.

unexge and others added 17 commits December 12, 2022 17:00
@unexge
Use LambdaContext in example service
b6c2390
@unexge
Start Lambda handler in a different thread
6cdd286
@unexge
Print summary of Lambda context in Pokemon service
db6c3cb
@unexge
Make sure to include Python builtins in tests
fc51609
@unexge @crisidev
Make lambda_ctx optional
17d87ee 
Co-authored-by: Matteo Bigoi <[email protected]>
@unexge
Only inject types if they are type-hinted as Optional[T]
f546ffe
@unexge
Export Lambda module as aws_lambda instead of lambda_
83316d2
@unexge
Comment why we need to run Hyper server in a background thread
ad99a99
@unexge
Move is_optional_of to util module
fcaadfb
@unexge
Use HeaderMap::from_iter to build headers
991ba0c
@unexge
Support edge case of (None, T) in util::is_optional_of
b0a7f18
@unexge
Make Lambda related types feature gated
7ba8efe
@unexge
Remove feature gate for Lambda
394f4b3
@unexge
Make xray_trace_id an Option
21c3fdd
@unexge
Remove aws-lambda feature from generated Cargo.tomls
716b664
@unexge
Fix linting issues
0e4b138
@unexge
Pin lambda_runtime to 0.7.1
ccb3046
@unexge unexge force-pushed the unexge/python-lambda-ctx branch from d35d6c2 to ccb3046 Compare December 12, 2022 17:01
@github-actions
Copy link

A new generated diff is ready to view.

A new doc preview is ready to view.

@unexge unexge merged commit 3fb9096 into main Dec 12, 2022
@unexge unexge deleted the unexge/python-lambda-ctx branch December 12, 2022 17:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@crisidev crisidev crisidev approved these changes

@hlbarber hlbarber hlbarber approved these changes

@chadac chadac chadac approved these changes

Assignees
No one assigned
Labels
python-server Python server SDK
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Provide a way to access Lambda Context in Python
4 participants
@unexge @crisidev @chadac @hlbarber