python: pydantic validators are incompatible with betterproto Message

[jku]

We have these validators in our classes:

@model_validator(mode="after")
def check_oneof(cls, values):
    return cls._validate_field_groups(values)

This does not seem to work because of how betterproto Messages are instantiated from json:

Bundle().from_json(data)

from_json() is an instance method so I believe the validator runs before from_json() and fails if there are any groups in the model:

 pydantic_core._pydantic_core.ValidationError: 1 validation error for Bundle
   Value error, Group content has no value; all fields are None [type=value_error, input_value=ArgsKwargs(()), input_type=ArgsKwargs]
     For further information visit https://errors.pydantic.dev/2.10/v/value_error

[jku]

CC @woodruffw based on #467

[woodruffw]

Yeah, this is one of the big hiccups with moving to Pydantic models. My approach in sigstore/sigstore-python#1276 was to move from Model().from_json to Model.from_dict(...), since the latter invokes the validators correctly.

However, the entire setup there feels brittle, perhaps unacceptably so 🙃 -- an option here would be to revert on #467 and go back to non-Pydantic-based models.

[jku]

move from Model().from_json to Model.from_dict(...), since the latter invokes the validators correctly.

Oh wow, that'll teach me to never trust api docs: I did go through it, there's nothing about a from_dict classmethod.

[woodruffw]

move from Model().from_json to Model.from_dict(...), since the latter invokes the validators correctly.

Oh wow, that'll teach me to never trust api docs: I did go through it, there's nothing about a from_dict classmethod.

Yep...I don't know why it doesn't get generated, but that method comes from Pydantic itself.

[jku]

However, the entire setup there feels brittle, perhaps unacceptably so 🙃 -- an option here would be to revert on #467 and go back to non-Pydantic-based models.

maybe yes: I for sure don't know the full picture but even just the ideas around "optionality" seem fundamentally incompatible: betterproto uses groups as a workaround and does not annotate the grouped fields as MyType | None... Instead the field is just annotated MyType but can actually still be None. I'm not sure how pydantic could ever cope with that (and still do something useful)?

EDIT: oops I was not looking in the right place: they are marked Optional in Bundle

[jku]

I'll close this: the issue is real but I Model.from_dict(...) seems like a completely valid workaround