Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: base CloudEvent class as per v1 specs, including attribute validation #242

Merged
merged 20 commits into from
Nov 16, 2024

Conversation

PlugaruT
Copy link

@PlugaruT PlugaruT commented Nov 8, 2024

Changes

The first PR towards a full rewrite of the library. What's included in this PR:

  • Support for v 1.0.2 of the specs with better validation errors and full support for custom extension attributes, including their validation.
  • Full typing support
  • Better documentation for each class and methods

One line description for the changelog

  • Tests pass
  • Appropriate changes to README are included in PR

@PlugaruT PlugaruT marked this pull request as ready for review November 9, 2024 17:46
Copy link
Member

@xSAVIKx xSAVIKx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @PlugaruT, thx for PR. Please take a look at comments and suggestions.

src/cloudevents/core/v1/event.py Show resolved Hide resolved
src/cloudevents/core/v1/event.py Outdated Show resolved Hide resolved
src/cloudevents/core/v1/event.py Outdated Show resolved Hide resolved
src/cloudevents/core/v1/event.py Outdated Show resolved Hide resolved
src/cloudevents/core/v1/event.py Outdated Show resolved Hide resolved
src/cloudevents/core/v1/event.py Outdated Show resolved Hide resolved
tests/test_core/test_v1/test_event.py Outdated Show resolved Hide resolved
src/cloudevents/core/v1/event.py Outdated Show resolved Hide resolved
src/cloudevents/core/v1/event.py Outdated Show resolved Hide resolved
src/cloudevents/core/v1/event.py Show resolved Hide resolved
@PlugaruT
Copy link
Author

PlugaruT commented Nov 9, 2024

There's a bit of a problem with mypy it seems, it doesn't like the dict[str, Any] for the attributes and also explicit str return for id field for example, since it's a required field and it's str.
I tried to play with TypedDict, but that one doesn't allow for arbitrary keys, but we need to support custom extensions. So the only way to "fix" this is to suppress the error? Let me know what you think.

@PlugaruT PlugaruT requested a review from xSAVIKx November 9, 2024 20:25
@xSAVIKx
Copy link
Member

xSAVIKx commented Nov 10, 2024

There's a bit of a problem with mypy it seems, it doesn't like the dict[str, Any] for the attributes and also explicit str return for id field for example, since it's a required field and it's str. I tried to play with TypedDict, but that one doesn't allow for arbitrary keys, but we need to support custom extensions. So the only way to "fix" this is to suppress the error? Let me know what you think.

There's a total option for TypedDict. You can read more in MyPy docs: https://mypy.readthedocs.io/en/stable/typed_dict.html#mixing-required-and-non-required-items and in the PEP: https://peps.python.org/pep-0589/#totality

We can have RequiredCloudEventAttributes(TypedDict) and then have `CloudEventAttributes(RequiredCloudEventAttributes, total=False). This will also eliminate part of the validations. E.g. typings and presence of required attributes can be checked by the typing system and we can have only optional runtime validation. This may be a good approach especially where speed is more important than runtime validation.

@xSAVIKx
Copy link
Member

xSAVIKx commented Nov 10, 2024

@PlugaruT, please don't use force push. The only merge strategy allowed in this repository is Squash and merge, so it's OK to have as many commits as you want and the way you want them.

force push makes it pretty hard to follow the review process in GitHub unfortunately

@PlugaruT
Copy link
Author

There's a bit of a problem with mypy it seems, it doesn't like the dict[str, Any] for the attributes and also explicit str return for id field for example, since it's a required field and it's str. I tried to play with TypedDict, but that one doesn't allow for arbitrary keys, but we need to support custom extensions. So the only way to "fix" this is to suppress the error? Let me know what you think.

There's a total option for TypedDict. You can read more in MyPy docs: https://mypy.readthedocs.io/en/stable/typed_dict.html#mixing-required-and-non-required-items and in the PEP: https://peps.python.org/pep-0589/#totality

We can have RequiredCloudEventAttributes(TypedDict) and then have `CloudEventAttributes(RequiredCloudEventAttributes, total=False). This will also eliminate part of the validations. E.g. typings and presence of required attributes can be checked by the typing system and we can have only optional runtime validation. This may be a good approach especially where speed is more important than runtime validation.

Yes, but the problem is with arbitrary keys, like extension names, since the TypedDict doesn't accept random keys besides the ones defined. If I try to define a CloudEvent instance with an extension, mypy complains:

src/cloudevents/core/v1/event.py:252: error: Extra key "customextension" for
TypedDict "CloudEventsAttributes"  [typeddict-unknown-key]
        e = CloudEvent({
                       ^
Found 2 errors in 1 file (checked 1 source file)

There's a whole discussion going on here https ://github.com/python/mypy/issues/ 4617 (intended not to link the PR's). So yes, it will "work" for the known attributes, but users will have to set the flag when they'll use this. Up to you what we want to do here.

@PlugaruT
Copy link
Author

@xSAVIKx, I addressed all the comments, except the mypy one that is a bit tricky. Let me know your thoughts

Signed-off-by: Tudor Plugaru <[email protected]>
Copy link
Member

@xSAVIKx xSAVIKx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@PlugaruT overall LGTM. I'd suggest we modify the error gathering a bit, but otherwise that's good.

src/cloudevents/core/v1/exceptions.py Outdated Show resolved Hide resolved
src/cloudevents/core/v1/event.py Outdated Show resolved Hide resolved
src/cloudevents/core/v1/event.py Outdated Show resolved Hide resolved
…ute name and typed exceptions

Signed-off-by: Tudor Plugaru <[email protected]>
We can't use TypedDict here becuase it does not allow for arbitrary keys which we need in order to support custom extension attributes.

Signed-off-by: Tudor Plugaru <[email protected]>
Copy link
Member

@xSAVIKx xSAVIKx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@PlugaruT PTAL at the comments and suggestions. Doing good 👍

src/cloudevents/core/v1/event.py Outdated Show resolved Hide resolved
src/cloudevents/core/v1/event.py Outdated Show resolved Hide resolved
src/cloudevents/core/v1/exceptions.py Show resolved Hide resolved
src/cloudevents/core/v1/exceptions.py Outdated Show resolved Hide resolved
src/cloudevents/core/v1/exceptions.py Outdated Show resolved Hide resolved
src/cloudevents/core/v1/exceptions.py Outdated Show resolved Hide resolved
src/cloudevents/core/v1/exceptions.py Outdated Show resolved Hide resolved
src/cloudevents/core/v1/exceptions.py Outdated Show resolved Hide resolved
src/cloudevents/core/v1/exceptions.py Outdated Show resolved Hide resolved
src/cloudevents/core/v1/exceptions.py Outdated Show resolved Hide resolved
src/cloudevents/core/v1/event.py Outdated Show resolved Hide resolved
src/cloudevents/core/v1/event.py Show resolved Hide resolved
src/cloudevents/core/v1/exceptions.py Outdated Show resolved Hide resolved
src/cloudevents/core/v1/exceptions.py Outdated Show resolved Hide resolved
src/cloudevents/core/v1/exceptions.py Outdated Show resolved Hide resolved
src/cloudevents/core/v1/exceptions.py Outdated Show resolved Hide resolved
src/cloudevents/core/v1/exceptions.py Outdated Show resolved Hide resolved
src/cloudevents/core/v1/exceptions.py Outdated Show resolved Hide resolved
Comment on lines 137 to 142
if hasattr(attributes["time"], "tzinfo") and not attributes["time"].tzinfo:
errors["time"].append(
InvalidAttributeValueError(
"time", "Attribute 'time' must be timezone aware"
)
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't recall this being required. Where did you get this requirement from?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

time attribute should respect RFC3339 which requires having timestamps with timezone attached.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think here, we have 2 options, either default to UTC, but that would be a controversial behaviour, as we can't default the timezone of the clients, or, make it an explicit requirement.
From my experience, using Python SDK and Java SDK, Java SDK will break when trying to have a naive datetime string. See here for example, time is defined as OffsetDateTime which means exactly this, timestamp with a timezone offset

Signed-off-by: Tudor Plugaru <[email protected]>
@PlugaruT
Copy link
Author

@xSAVIKx can you also merge please, I don't have the perms to do it. Thanks

@xSAVIKx xSAVIKx merged commit a73c870 into cloudevents:v2 Nov 16, 2024
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants