Skip to content

build: temporary unbreak for make proto#5632

Merged
sougou merged 2 commits intovitessio:masterfrom
planetscale:ss-unbreak-protoc
Dec 31, 2019
Merged

build: temporary unbreak for make proto#5632
sougou merged 2 commits intovitessio:masterfrom
planetscale:ss-unbreak-protoc

Conversation

@sougou
Copy link
Copy Markdown
Contributor

@sougou sougou commented Dec 31, 2019

Due to an import pathing restriction in protoc, we still need the
source tree to represent the import names in the packages.
This workaround makes the build work for those who have their
github branch checked out this way.

Signed-off-by: Sugu Sougoumarane ssougou@gmail.com

@sougou sougou requested a review from morgo December 31, 2019 17:25
Due to an import pathing restriction in protoc, we still need the
source tree to represent the import names in the packages.
This workaround makes the build work for those who have their
github branch checked out this way.

Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
$(PROTOC_COMMAND) -Iproto $< --python_out=py/vtproto --grpc_python_out=py/vtproto

# TODO(sougou): find a better way around this temp hack.
VTTOP=$(VTROOT)/../../..
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the contributor's guide, we are asking people to checkout to ~/vitess. This will break anyone who follows that.
What error are you getting from make proto?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Run as is, you'll get this error:

~/...vitess> make proto
Tue Dec 31 12:30:45 PST 2019: Compiling proto definitions
/bin/bash: line 1: cd: /home/sougou/dev/src/vitess.io/vitess/src: No such file or directory
[more of the same]

If you fix the cd command, then you'll get something like this:

~/...vitess> make proto
Tue Dec 31 12:32:38 PST 2019: Compiling proto definitions
vitess.io/vitess/proto: warning: directory does not exist.
vitess.io/vitess/proto/tabletmanagerservice.proto: No such file or directory

The .proto files, there is this line:

option go_package = "vitess.io/vitess/go/vt/proto/tabletmanagerdata";

This makes protoc want to generate the file at vitess.io/vitess/proto relative to the current path. We need to figure out a way to make it generate the file under vt/proto instead. I haven't found a way yet.

If we change the go_package to go/vt/proto/tabletmanagerdata, then it generates the wrong package names in the generated files (without thevitess.io/vitess prefix).

Essentially, protos won't build with the current Makefile at all. This change at least allows you to build it if you have vitess checked out under src/vitess.io/vitess.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we potentially use a symlink from vitess.io/vitess/proto to vt/proto instead? .. at least until we find a better way to do this.

@sougou
Copy link
Copy Markdown
Contributor Author

sougou commented Dec 31, 2019

Tried the symlink. Didn't help:

~/...vitess> mkdir -p vitess.io/vitess
~/...vitess> ln -s go/vt/proto vitess.io/vitess/proto
~/...vitess> git status
On branch ss-unbreak-protoc
Your branch is behind 'origin/ss-unbreak-protoc' by 1 commit, and can be fast-forwarded.
  (use "git pull" to update your local branch)

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   Makefile
        modified:   go.mod

Untracked files:
  (use "git add <file>..." to include in what will be committed)

        vitess.io/

no changes added to commit (use "git add" and/or "git commit -a")
~/...vitess> make proto
Tue Dec 31 14:24:11 PST 2019: Compiling proto definitions
vitess.io/vitess/proto: warning: directory does not exist.
vitess.io/vitess/proto/tabletmanagerservice.proto: No such file or directory
[snip]

@sougou sougou closed this Dec 31, 2019
@sougou sougou reopened this Dec 31, 2019
The protoc generated code was not matching the goimports standard.
I've added a goimports step after the build.
This caused another breakage because time.proto did not match its
package name of vttime. I've fixed that also.

Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
@sougou
Copy link
Copy Markdown
Contributor Author

sougou commented Dec 31, 2019

I've pushed an extra commit to fix the codegen of protoc, which was no compliant with goimports.

Copy link
Copy Markdown
Contributor

@morgo morgo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. This is temporary, but at least it is working again. We can spend some more time figuring out a more permanent fix.

@sougou sougou merged commit 5a9da55 into vitessio:master Dec 31, 2019
@sougou sougou deleted the ss-unbreak-protoc branch December 31, 2019 23:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants