Support for Value Wrappers

[ulissesalmeida]

In protobuf messages when we have enum or scalar types they can't be nil. To make it nil, a way of doing it is to wrap in a message. This pattern is very common, we can see for example in google protobufs wrappers:

https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/wrappers.proto

And it makes writing code like:

message User {
  StringValue first_name = 1;
}

Later in Elixir, we'll have to access like:

# when present
user.first_name
%Google.Protobuf.StringValue{value: "Abilidebob"}

# when not present
user.first_name
%Google.Protobuf.StringValue{value: nil}

I wonder if it would not be cool this library can understand this pattern and unwrap and wrap automatically for us when decoding/encoding?

Then we could have something like:

# when present
user.first_name
"Abilidebob"

# when not present
user.first_name
nil

Of course, to make it compatible with previous versions, it should by default false, but you can turn as an option.

For example:

protoc --elixir_out=./lib --plugin=./protoc-gen-elixir --elixir_opt=value_wrapper=true myproto.proto

What do you folks think? Do we have space for such a feature in this library?

[whatyouhide]

@ulissesalmeida we have this pattern in our codebase as well and we defined wrapper functions for this, such as Protobuf.wrap("foo") == %StringValue{value: "foo"} and Protobuf.wrap(nil) == nil. I’m not sure how that would be abstracted by the library. How do you recognize this pattern from the Elixir perspective?

[ulissesalmeida]

I can imagine some solution by using some instrospection/reflection based on the protobuf metadata.

For example, during the decodification of the binary message we could have something like:

# in decode function (pseudo-code)
# after decode the field value
if unwrap?(field_metadata, context) do
  module = field_metadata[:message_module]
  %module{value: value} = decoded_value
else
  decoded_value
end

defp unwrap?(field_metadata, context), 
  do: context[:using_values_wrapper?) and wrapper?(field_metadata)

defp wrapper?(field_metadata), 
  do: String.ends_with?(field_metada[:module), "Value") 
    and (scalar?(metadata[field]) or enum?(metadata[field])) 

In the encode part, we can have a wrap function, something like:

# in encodefunction (pseudo-code)
# before encode the field value
if wrap?(field_metadata, context) do
  module = field_metadata[:message_module]

  case value_to_encode do
    # it's already wrapped clause, just return it
     %module{} -> value_to_encode
    # not wrapped cause, let's wrap
     value -> %module{value: value}
  end
else
  value_to_encode
end

defp wrap?(field_metadata, context), 
  do: context[:using_values_wrapper?) and wrapper?(field_metadata)

defp wrapper?(field_metadata), 
  do: String.ends_with?(field_metada[:module), "Value") 
    and (scalar?(metadata[field]) or enum?(metadata[field])) 

I'm assuming we have all metadata info during encoding/decoding

What do you think? Is it achieavable?

[whatyouhide]

I think this would be brittle since folks might not only use Google's wrappers and might want a similar behaviour in other situations too. I think this could be achieved with a plugin where you mark the type as a wrapper right in the .proto definition, but not sure that that would be a clean solution either.

[ulissesalmeida]

@whatyouhide Interesting, I'm introducing the message level options here:

https://github.com/tony612/protobuf-elixir/pull/91/files#diff-b29046f70a5e38390cd6f773e215597cR32

It could be something like:

message MyEnumValue {
  option (elixirpb.message).value_wrapper = true;
  MyEnum value = 1;
}

The only downside that I see is the users will have to manually put this option in all google protobuf messages. If for some reason they want to update their google protobuf messages isn't that automatic anymore. They will have to remember to put that option. A solution maybe is adding this option to original google protobuf files 🤔 But I don't know how the google team is receptible with custom language options their source.

[whatyouhide]

I don't think Google would appreciate that 😄 But yeah, the option you showed is the only solution that I like so far. FWIW, we copied Google's Protobuf wrappers in our own collection of Protobuf schemas to have control over them, so for example, in our use case, the option you suggested would work great.

[ulissesalmeida]

@whatyouhide I implemented a the ValueWrappers feature there:
coingaming#1

right now it's a global option from command line, but it's very straight forward to add a message option too.

However, this PR depends on encodable/decodable protocols, because that PR creates a distinction between modules and type annotations in the package metadata. If you have a time to give a try ❤️

[whatyouhide]

Hey @ulissesalmeida, we're trying to clean up issues so I’m closing this one since we likely won't pick it up until we have time for a major release in the future. The context and discussion here are super helpful though, so thank you for all the input! Hopefully we'll come back to this soon enough 😌 💟