Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to add Custom vanity to grpc-gateway #395

Closed
Tommy-42 opened this issue Mar 15, 2018 · 13 comments
Closed

How to add Custom vanity to grpc-gateway #395

Tommy-42 opened this issue Mar 15, 2018 · 13 comments

Comments

@Tommy-42
Copy link

Tommy-42 commented Mar 15, 2018

Hello guys,

I am currently using gogoproto with a custom vanity
CustomNameID same one as swarm-kit

What does this CustomNameID? it basically change my_id to myID instead of myId.

so our protoc-gen-custom cmd looks like :

package main

import (
    "github.com/gogo/protobuf/protoc-gen-gogo/descriptor"
    "github.com/gogo/protobuf/vanity"
    "github.com/gogo/protobuf/vanity/command"

    // custom govalidator import
    validator_plugin "myrepo/govalidator/validator"
)

func main() {
    req := command.Read()
    files := req.GetProtoFile()
    files = vanity.FilterFiles(files, vanity.NotGoogleProtobufDescriptorProto)

    req.ProtoFile = files

    for _, opt := range []func(*descriptor.FileDescriptorProto){
        vanity.TurnOffGoGettersAll,
        vanity.TurnOffGoStringerAll,
        vanity.TurnOnMarshalerAll,
        vanity.TurnOnStringerAll,
        vanity.TurnOnUnmarshalerAll,
        vanity.TurnOnSizerAll,
        CustomNameID, // custom added here
    } {
        vanity.ForEachFile(files, opt)
    }

    // generate default models
    resp := command.Generate(req)
    command.Write(resp)
    
    // generate go validator plugin
    resp = command.GeneratePlugin(req, validator_plugin.NewPlugin(), ".validator.pb.go")
    command.Write(resp)
}    

my problem is, I am currently trying to add grpc-gateway to my project but grpc-gateway output field without our CustomNameID.

Does anyone has ever encountered this problem ?

Is there a way to add grpc-gateway to this custom protoc-gen-custom to be able to use vanity on it, like we used with our govalidator ?

Thanks you all

@awalterschulze
Copy link
Member

I don't see how adding grpc-gateway to the plugin list is going to solve your problem, but I also think I don't fully understand your problem yet.

Could you show an example snippet of how grpc-gateway outputs the field without the CustomNameID and why that breaks things?

@Tommy-42
Copy link
Author

Tommy-42 commented Mar 19, 2018

Yes sure,

Output with protoc + our customname we have

`OwnerID:` + fmt.Sprintf("%v", this.OwnerID) + `,`,

With grpc-gateway we have :

`OwnerId:` + fmt.Sprintf("%v", this.OwnerId) + `,`,

I am currently trying to use in my proto

import "github.com/gogo/protobuf/gogoproto/gogo.proto";

message foo {
  string owner_id = 1 [(gogoproto.customname) = "OwnerID"];
}

( without succes yet, still digging what did I forgot again )

@awalterschulze
Copy link
Member

Where is this output happening?

@Tommy-42
Copy link
Author

Tommy-42 commented Mar 19, 2018

for the first one into the pb.go

the second one into the pb.gw.go

@awalterschulze
Copy link
Member

I really need more info.
Please help.

I don't have a lot of time to spend on these issues. Please try and make it as easy for me as possible to help you.

@Tommy-42
Copy link
Author

Sorry, I have been kind busy since my answer.

Ok so :

I have a proto file with a message, let say :

message Contact {
  string contact_id = 1;
}

If I generate the code using protoc the output will be something like ( pb.go) :

func (this *Contact) String() string {
	if this == nil {
		return "nil"
	}
	s := strings.Join([]string{`&Contact{`,
		`ContactId:` + fmt.Sprintf("%v", this.ContactId) + `,`,
		`}`,
	}, "")
	return s
}

Note that ContactId I uppercase d lowercase

so what we do is :

We have a custom generator :

package main

import (
    "github.com/gogo/protobuf/protoc-gen-gogo/descriptor"
    "github.com/gogo/protobuf/vanity"
    "github.com/gogo/protobuf/vanity/command"

    // custom govalidator import
    validator_plugin "myrepo/govalidator/validator"
)

func main() {
    req := command.Read()
    files := req.GetProtoFile()
    files = vanity.FilterFiles(files, vanity.NotGoogleProtobufDescriptorProto)

    req.ProtoFile = files

    for _, opt := range []func(*descriptor.FileDescriptorProto){
        vanity.TurnOffGoGettersAll,
        vanity.TurnOffGoStringerAll,
        vanity.TurnOnMarshalerAll,
        vanity.TurnOnStringerAll,
        vanity.TurnOnUnmarshalerAll,
        vanity.TurnOnSizerAll,
        CustomNameID, // custom added here
    } {
        vanity.ForEachFile(files, opt)
    }

    // generate default models
    resp := command.Generate(req)
    command.Write(resp)
    
    // generate go validator plugin
    resp = command.GeneratePlugin(req, validator_plugin.NewPlugin(), ".validator.pb.go")
    command.Write(resp)
}   

we added one more custom vanity check here : CustomNameID

Which basically search for *Id and replace them to *ID

It work fine when generating the code ( pb.go) but ...

If we want to use grpc-gateway ( pb.gw.go ) like :
protoc .... --grpc-gateway_out= .....

the vanity check doesn't apply and we have on one hand :

pb.go : ContactID

and on the other hand :

pb.gw.go : ContactId

Resulting in error where the variable are not found.

thanks for your time and your patience

@awalterschulze
Copy link
Member

Could you paste a little code showing the output in pb.gw.go where ContactId is referenced?

@Tommy-42
Copy link
Author

func request_GetContact_0(ctx context.Context, marshaler runtime.Marshaler, client Client, req *http.Request, pathParams map[string]string) (proto.Message, runtime.ServerMetadata, error) {
	var protoReq GetContactRequest
	var metadata runtime.ServerMetadata

	var (
		val string
		ok  bool
		err error
		_   = err
	)

	val, ok = pathParams["id"]
	if !ok {
		return nil, metadata, status.Errorf(codes.InvalidArgument, "missing parameter %s", "id")
	}

	// HERE
	protoReq.Id, err = runtime.String(val)

	if err != nil {
		return nil, metadata, status.Errorf(codes.InvalidArgument, "type mismatch, parameter: %s, error: %v", "id", err)
	}

	msg, err := client.GetContact(ctx, &protoReq, grpc.Header(&metadata.HeaderMD), grpc.Trailer(&metadata.TrailerMD))
	return msg, metadata, err

}

@awalterschulze
Copy link
Member

Ok so that is just Id and not ContactId, but I think I get it now.

I assume swarmkit is not using grpc-gateway?

Could you generate your grpc-gateway as a separate service? without CustomNameID, because I don't think grpc-gateway is going to support that. But then you should still be able to generate a separate service with your normal gRPC code that can interact on a binary level with your gateway using your vanity binary.

Otherwise you could also wait for golang/protobuf#555
which should deprecate gogoproto.customname and should be supported by grpc-gateway in some future.

@Tommy-42
Copy link
Author

yes not quite the same but you got the idea :) !( contact_id is no more generated and I couldnt get you the exact output )

No swarm-kit does not use it :( , I think I'll have to wait golang/protobuf#555

I will just remove the vanity filter and go back to Id instead of ID.

If I found something interesting, as a workaround or something. I will try to share it :)

Thanks for the help

@awalterschulze
Copy link
Member

How about having your grpc-gateway as a separate service with fieldname Id that communicates over the wire with your grpc service where the fieldname is generated as ID. That should work?

@awalterschulze
Copy link
Member

Can I close this?

@Tommy-42
Copy link
Author

Yes, sorry for the delay

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants