protoc-gen-swagger: support all well-known wrapper types #695

jriecken · 2018-07-07T00:26:51Z

There were a few well-known wrapper types missing from the wkSchemas map, in specific: UInt32Value, UInt64Value and BytesValue. This change handles them (and maps them to the same swagger types as the non-wrapped values)

This also fixes the mapping of Int64Value. The Int64Value handling maps the value to a swagger integer. The documentation for Int64Value indicates that it should be mapped to a JSON string (also the mapping for normal int64 in protoc-gen-swagger maps it to a string, so this was inconsistent.)

googlebot · 2018-07-07T00:26:53Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here (e.g. I signed it!) and we'll verify it.

What to do if you already signed the CLA

Individual signers

It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
The email used to register you as an authorized contributor must also be attached to your GitHub account.

jriecken · 2018-07-07T00:30:06Z

I signed it!

googlebot · 2018-07-07T00:30:07Z

CLAs look good, thanks!

johanbrandhorst · 2018-07-07T13:25:00Z

Hi @jriecken! Thanks for your contribution! The idea looks good, unfortunately, it looks like your changes caused an issue with the marshal_jsonpb tests:

marshal_jsonpb_test.go:274: strings.Contains("{\"singleNested\":{},\"uuid\":\"6EC2446F-7E89-4127-B3E6-5C05E6BECBA7\",\"nested\":[{\"name\":\"foo\",\"amount\":12345}],\"uint64Value\":\"18446744073709551615\",\"repeatedStringValue\":[],\"oneofString\":\"bar\",\"mapValue\":{\"a\":\"ONE\",\"b\":\"ZERO\"},\"mappedStringValue\":{},\"mappedNestedValue\":{},\"timestampValue\":\"1970-01-01T00:00:00Z\",\"repeatedEnumValue\":[]}", "ONE") = true; want false

If I had to guess, I think you might need to add these new types to to the test as well. Other than that, looks good!

jriecken · 2018-07-07T15:41:19Z

It looks like this was caused by an upstream bugfix in github.com/golang/protobuf/jsonpb that was made yesterday: golang/protobuf#645

The JSON generated by that marshal_jsonpb test before that fix looks like:

{"singleNested":{},"uuid":"6EC2446F-7E89-4127-B3E6-5C05E6BECBA7","nested":[{"name":"foo","amount":12345}],"uint64Value":"18446744073709551615"
,"repeatedStringValue":[],"oneofString":"bar","mapValue":{"a":1,"b":0},"mappedStringValue":{},"mappedNestedValue":{},"timestampValue":"1970-01-01T00:00:00Z","repeatedEnumValue":[
]}

and after:

{"singleNested":{},"uuid":"6EC2446F-7E89-4127-B3E6-5C05E6BECBA7","nested":[{"name":"foo","amount":12345}],"uint64Value":"18446744073709551615"
,"repeatedStringValue":[],"oneofString":"bar","mapValue":{"a":"ONE","b":"ZERO"},"mappedStringValue":{},"mappedNestedValue":{},"timestampValue":"1970-01-01T00:00:00Z","repeatedEnu
mValue":[]}

Note how the mapValue map's values changed from the numeric representation of the enum ("mapValue":{"a":1,"b":0}) to the string one ("mapValue":{"a":"ONE","b":"ZERO"}). This makes sense as the test is setting EnumsAsInts to 0 (i.e. output them as strings).

I'll update the test to match the fixed behavior from jsonpb

jriecken · 2018-07-07T15:57:29Z

One of the browser tests was also assuming integer enum values - fixed it too

johanbrandhorst · 2018-07-07T16:28:55Z

Looks like we still need some regeneration of files.

jriecken · 2018-07-07T16:32:48Z

One of the travis runs failed because protoc generated a file descriptor var that was slightly different (fileDescriptor_wrappers_5850bd48b0187754 vs fileDescriptor_wrappers_0cd12813574dca64) and the git diff failed. The other 5 non-bazel test runs did not have this same issue.

The Bazel version of the build also failed (it must somehow be pulling in a different version of jsonpb as it output the old numeric enum values?)

I feel like I'm very quickly going down a rabbit hole in grpc-gateway's build process...

johanbrandhorst · 2018-07-07T16:40:04Z

I think the first failure should be fixed with a regeneration of the protofiles (I guess it's sha-dependent?). I admit I'm mostly ignorant of the build process. @achew22 @tmc could you please help?

jriecken · 2018-07-07T16:52:10Z

I was using the latest version of protoc instead of 3.1 - regenerating with 3.1 created the same var name.

jriecken · 2018-07-07T17:37:55Z

I did a little more digging and the Bazel Go rules pin the golang/protobuf dependency to this version: https://github.com/golang/protobuf/releases/tag/v1.1.0 (see https://github.com/bazelbuild/rules_go/blob/7e07b18263c08c8eca31207e1f5b28aa71539a72/go/private/repositories.bzl#L75)

The non-Bazel build uses go get (i..e pulling from master of golang/protobuf, which has the change made yesterday that caused the marshal_jsonpb test to break in the first place)

johanbrandhorst · 2018-07-07T18:00:44Z

So it seems we have a mismatch of versions between that pinned by Bazel and that fetched in the normal test runs. Presumably this means we cannot get a successful build unless we change one of the two. I'm not comfortable with making the call here but I would assume we'd be more keen to pin versions to releases, so assuming that, how do we pin the non-Bazel tests? This might well not be fixable until something like #689 is done.

johanbrandhorst · 2018-07-08T15:26:12Z

@jriecken It looks to me like the Bazel rules used are v0.10.3:

grpc-gateway/WORKSPACE

Line 5 in 5948c50

    
           url = "https://github.com/bazelbuild/rules_go/releases/download/0.10.3/rules_go-0.10.3.tar.gz",

. I'm working on a PR which will add parity between Bazel pins and Gopkg.toml.

johanbrandhorst · 2018-07-08T15:26:50Z

Release v0.10.3 of rules_go uses golang/protobuf v1.0.0: https://github.com/bazelbuild/rules_go/blob/f676870c5caf8df559a51e7aa005d2ece148a03b/go/private/repositories.bzl#L72

johanbrandhorst · 2018-07-08T15:43:38Z

#696 should hopefully fix these dependency inconsistencies. Then this PR should be easy to merge.

johanbrandhorst · 2018-07-23T21:26:23Z

@jriecken could you rebase on master please? The fix for #696 has been merged

There were a few well-known wrapper types missing from the wkSchemas map. In specific UInt32Value, UInt64Value and BytesValue. This change handles them (and maps them to the same swagger types as the non-wrapped values) This also fixes the mapping of Int64Value. The Int64Value handling maps the value to a swagger integer. The documentation for Int64Value indicates that it should be mapped to a JSON string (also the mapping for normal int64 in protoc-gen-swagger maps it to a string, so this was inconsistent.)

codecov-io · 2018-07-23T22:04:22Z

Codecov Report

Merging #695 into master will decrease coverage by 0.11%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master     #695      +/-   ##
==========================================
- Coverage   56.54%   56.43%   -0.12%     
==========================================
  Files          30       30              
  Lines        3040     3046       +6     
==========================================
  Hits         1719     1719              
- Misses       1152     1158       +6     
  Partials      169      169

Impacted Files	Coverage Δ
protoc-gen-swagger/genswagger/template.go	`38.18% <0%> (-0.29%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d724c4f...bf62347. Read the comment docs.

jriecken · 2018-07-23T22:06:19Z

Done

johanbrandhorst · 2018-07-24T06:00:56Z

Great, this LGTM!

mayank-dixit · 2018-11-14T05:43:29Z

@johanbrandhorst @jriecken
With this change, can this be done:

service Greeter {
  rpc SayHello (HelloRequest) returns (HelloReply) {
    option (google.api.http) = {
      get: "/say/{strVal}"
    };
  }
}

message HelloRequest {
  google.protobuf.StringValue strVal = 5;
}

Right now when I compile using protoc:

protoc -I/usr/local/include -I. \
  -I$GOPATH/src \
  -I$GOPATH/src/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis \
  --grpc-gateway_out=logtostderr=true:. \
  helloworld/helloworld.proto

I get the error:
--grpc-gateway_out: aggregate type TYPE_MESSAGE in parameter of Greeter.SayHello: strVal

johanbrandhorst · 2018-11-14T10:22:52Z

That should absolutely work, I can't see why that is erroring. Could you raise an issue please?

mayank-dixit · 2018-11-14T11:29:13Z

That should absolutely work, I can't see why that is erroring. Could you raise an issue please?

Created: #808

googlebot added the cla: no label Jul 7, 2018

googlebot added cla: yes and removed cla: no labels Jul 7, 2018

johanbrandhorst self-requested a review July 8, 2018 15:12

johanbrandhorst mentioned this pull request Jul 8, 2018

Add explicit dependency versions #696

Merged

jriecken force-pushed the all-well-known-wrapper-types branch from f4d2348 to bf62347 Compare July 23, 2018 21:43

johanbrandhorst approved these changes Jul 24, 2018

View reviewed changes

johanbrandhorst merged commit 209d7ce into grpc-ecosystem:master Jul 24, 2018

johanbrandhorst mentioned this pull request Sep 9, 2018

Handle wrapper proto types as primitives #412

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

protoc-gen-swagger: support all well-known wrapper types #695

protoc-gen-swagger: support all well-known wrapper types #695

jriecken commented Jul 7, 2018

googlebot commented Jul 7, 2018

jriecken commented Jul 7, 2018

googlebot commented Jul 7, 2018

johanbrandhorst commented Jul 7, 2018

jriecken commented Jul 7, 2018 •

edited

Loading

jriecken commented Jul 7, 2018

johanbrandhorst commented Jul 7, 2018

jriecken commented Jul 7, 2018

johanbrandhorst commented Jul 7, 2018

jriecken commented Jul 7, 2018 •

edited

Loading

jriecken commented Jul 7, 2018

johanbrandhorst commented Jul 7, 2018

johanbrandhorst commented Jul 8, 2018

johanbrandhorst commented Jul 8, 2018

johanbrandhorst commented Jul 8, 2018

johanbrandhorst commented Jul 23, 2018

codecov-io commented Jul 23, 2018

jriecken commented Jul 23, 2018

johanbrandhorst commented Jul 24, 2018

mayank-dixit commented Nov 14, 2018

johanbrandhorst commented Nov 14, 2018

mayank-dixit commented Nov 14, 2018

protoc-gen-swagger: support all well-known wrapper types #695

protoc-gen-swagger: support all well-known wrapper types #695

Conversation

jriecken commented Jul 7, 2018

googlebot commented Jul 7, 2018

What to do if you already signed the CLA

Individual signers

Corporate signers

jriecken commented Jul 7, 2018

googlebot commented Jul 7, 2018

johanbrandhorst commented Jul 7, 2018

jriecken commented Jul 7, 2018 • edited Loading

jriecken commented Jul 7, 2018

johanbrandhorst commented Jul 7, 2018

jriecken commented Jul 7, 2018

johanbrandhorst commented Jul 7, 2018

jriecken commented Jul 7, 2018 • edited Loading

jriecken commented Jul 7, 2018

johanbrandhorst commented Jul 7, 2018

johanbrandhorst commented Jul 8, 2018

johanbrandhorst commented Jul 8, 2018

johanbrandhorst commented Jul 8, 2018

johanbrandhorst commented Jul 23, 2018

codecov-io commented Jul 23, 2018

Codecov Report

jriecken commented Jul 23, 2018

johanbrandhorst commented Jul 24, 2018

mayank-dixit commented Nov 14, 2018

johanbrandhorst commented Nov 14, 2018

mayank-dixit commented Nov 14, 2018

jriecken commented Jul 7, 2018 •

edited

Loading

jriecken commented Jul 7, 2018 •

edited

Loading