Skip to content

feat: support deserializing explicit nulls (#73)#74

Merged
kodiakhq[bot] merged 3 commits intomainfrom
support-explicit-nulls
Oct 26, 2022
Merged

feat: support deserializing explicit nulls (#73)#74
kodiakhq[bot] merged 3 commits intomainfrom
support-explicit-nulls

Conversation

@tustvold
Copy link
Contributor

@tustvold tustvold commented Sep 30, 2022

Closes #73

Adds support for explicit null values where the corresponding protobuf values are nullable.

It is worth highlighting that JSON supports greater nullability than protobuf, and this leads to a couple of places where null is still invalid.

List Elements are not Nullable

message Test {
    repeated int32 mylist = 1;
}
{
  "mylist": [null] <-- invalid
}

Oneof nulls are not typed

message Test {
    oneof foo {
        int32 a = 1;
        int32 b = 2;
    }
}
{
  "a": null
}

Will get converted to {foo: None} which loses the fact the null was encoded on a

@@ -753,151 +753,207 @@ fn write_deserialize_field<W: Write>(
json_name
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some(deserializer) => {
write!(
writer,
"map.next_value::<::std::option::Option<{}>>()?.map(|x| {}::{}(x.0))",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the major "theme" of the change for optional fields rather than decoding T we decode to Option<T>

_ => {
match one_of {
Some(one_of) => match &field.field_type {
FieldType::Scalar(s) => match override_deserializer(*s) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fields within a oneof can't have labels, and so we don't need to worry about repeated, etc...

Copy link
Contributor

@crepererum crepererum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's tested and I believe this will fly, but I have some minor nits (see comments).

decoded.string_value = None;
verify(&decoded, r#"{}"#);

// Test explicit null optional scalar
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The statefulness makes it a bit hard to read, because what is decoded at this point? Would be nice if decoded would be specified fully before every verify[_decode] block.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The line above verifies that decoded is {}?

verify_decode(&decoded, r#"{"stringValue":null}"#);
verify_decode(&decoded, r#"{"uint32Value":null}"#);
verify_decode(&decoded, r#"{"uint64Value":null}"#);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is "List Elements are not Nullable" not tested?

@@ -427,6 +427,34 @@ mod tests {

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if checking in the generated file for the test setup would be helpful, because reviewing a bunch of writeln! calls is a bit tricky (I have to admit that I have about zero mental context for each of the writes).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The difficulty is we compile the tests with various different feature flags, to test various different builder options, so I'm not really sure how to achieve this without checking in 5 different versions of the code...

writer,
"::pbjson::private::NumberDeserialize<{}>",
key.rust_type()
"Some(map.next_value::<Vec<{}>>()?.into_iter().map(|x| x as i32).collect())",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really got confused about Vec<{}> and it took me a while to realize that {} is NOT Rust code, but placeholder for write!. I've said this below, but I think checking in the generated test code would help a lot.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also: you're really careful to use fully-qualified names for all types including Option. I wonder if this should also be done for Vec (::std::vec::Vec)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Filed #80

@tustvold tustvold added the automerge Put in the merge queue label Oct 26, 2022
@kodiakhq kodiakhq bot merged commit 5fa7d4d into main Oct 26, 2022
@tustvold tustvold mentioned this pull request Oct 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automerge Put in the merge queue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support Explicit JSON null

2 participants