Multiple protobufs with same filename are not compiled

[stetra]

It seems that if there are multiple .proto files with the same filename, only the last added one will get compiled.

Suppose I have two protobuf files, both named foo.proto:

protobuf
└── util
    ├── v1
    │   └── foo.proto
    └── v2
        └── foo.proto

If my build.gradle is configured like so:

dependencies {
    protobuf fileTree("protobuf/util/v1")
    protobuf fileTree("protobuf/util/v2")
}

Then when running gradle build only protobuf/util/v2/foo.proto is compiled and sources are generated under build/generated/source/proto/. However if I reverse the order of the dependencies:

dependencies {
    protobuf fileTree("protobuf/util/v2")
    protobuf fileTree("protobuf/util/v1")
}

Then only protobuf/util/v1/foo.proto is compiled. I have created an example repo at https://github.com/stetra/protobuf-gradle-plugin-example

Is this the expected behavior? How can I ensure both foo.proto protobufs are compiled without having to rename them?

[zpencer]

In your repo, if you move your proto directory to src/main/proto then everything works as expected.
Looking at the output of gradlew build --debug:

13:20:21.418 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationExecutor] Build operation 'Resolve files of :protobuf' completed
13:20:21.418 [DEBUG] [org.gradle.api.Task] Extracting protos from /tmp/protobuf-gradle-plugin-example/protobuf/util/v1/foo.proto to /tmp/protobuf-gradle-plugin-example/build/extracted-protos/main
13:20:21.419 [DEBUG] [org.gradle.api.Task] Extracting protos from /tmp/protobuf-gradle-plugin-example/protobuf/util/v2/foo.proto to /tmp/protobuf-gradle-plugin-example/build/extracted-protos/main

Each .proto file was added individually, and they get copied to the destination overwriting each other. `

Alternatively you can use this config if you must use the non default location:

plugins {
    id "com.google.protobuf" version "0.8.6"
    id "java"
}

sourceCompatibility = 1.8
targetCompatibility = 1.8

repositories {
    mavenCentral()
}

dependencies {
    compile 'com.google.protobuf:protobuf-java:3.6.0'
    
}

protobuf {
    protoc {
        // The protobuf compiler, should be compatible with protobuf-java
        artifact = 'com.google.protobuf:protoc:3.6.0'
    }
}


sourceSets {
  main {
    proto {
      srcDir 'protobuf/'
    }
  }
}

[zhangkun83]

Having just protobuf fileTree("protobuf/util") still copies individual files into the destination directory. It should have copied the directory tree. This is indeed a bug.

The problem is in ProtobufExtract.groovy:

  @TaskAction
  void extract() {
    inputs.files.each { file ->
      logger.debug "Extracting protos from ${file} to ${destDir}"
      if (file.isDirectory()) {
        project.copy {
          includeEmptyDirs(false)
          from(file.path) {
            include '**/*.proto'
          }
          into(destDir)
        }

It appears when a directory is given in the build script, it's expanded into individual files when given to inputs. The code above never sees the original directory.

[zhangkun83]

I think we incorrectly recommended the use of:

protobuf fileTree("ext/")

With this syntax Gradle will collect all files under the given directory and emit them as individual files, leaving the plugin unable to tell what the root directory is, thus dumping the files under the same directory.

If we just pass the directory to the configuration, the current implementation will correctly handle the copying of a whole directory, preserving its structure:

protobuf files("ext/")

I am going to make change the tests to have the latter syntax, and add warnings if the former syntax is detected.

/cc @ejona86

[ejona86]

SGTM