file_descriptor_proto_data changes each time

[Xynnn007]

Hi all, I'm working on RB(reproducible building) of kata-agents

Some of code are based on rust-protobuf in <kata-containers>/src/agent/protocols
I build the project via cargo build in dir <kata-containers>/src/agent/protocols for many times, but each time the field
file_descriptor_proto_data in the result health.rs file differs.

Source code shows how health.rs is generated

...
   let protos = vec![
        "protos/types.proto",
        "protos/agent.proto",
        "protos/health.proto",
        "protos/google/protobuf/empty.proto",
        "protos/oci.proto",
    ];

    Codegen::new()
        .out_dir("src")
        .inputs(&protos)
        .include("protos")
        .rust_protobuf()
        .customize(Customize {
            async_server: true,
            ..Default::default()
        })
        .run()
        .expect("Gen codes failed.");
...

Here Codegen refers to ttrpc-codegen = "0.2.0", which calls related functions from rust-protobuf.
The build logs are like below

$cargo build
  ...
   Compiling protocols v0.1.0 (/root/kata/src/agent/protocols)
    Finished dev [unoptimized + debuginfo] target(s) in 8.64s
$mv src/health.rs src/health1.rs
$cargo clean
$cargo build
  ...
   Compiling protocols v0.1.0 (/root/kata/src/agent/protocols)
    Finished dev [unoptimized + debuginfo] target(s) in 30.77s
$diff src/health.rs src/health1.rs
611,612c611,612
<     sion\x18\x02\x20\x01(\tB\0:\0B\x10\xb8\xe2\x1e\x01\xf8\xe1\x1e\x01\xa8\
<     \xe2\x1e\x01\xc0\xe2\x1e\x01b\x06proto3\
---
>     sion\x18\x02\x20\x01(\tB\0:\0B\x10\xf8\xe1\x1e\x01\xc0\xe2\x1e\x01\xa8\
>     \xe2\x1e\x01\xb8\xe2\x1e\x01b\x06proto3\

the snapshot may help to understand details

I wonder whether here some hash functions or random sources are involved.
And to help deterministic/reproducible build, can you help to ensure that we can get same result .rs files from .proto in the same environment?

Thank you!

[stepancheg]

This is a significant issue. Can you provide a minimal example to reproduce the issue? Or at least, some .proto file to reproduce the issue?

[Xynnn007]

This is a significant issue. Can you provide a minimal example to reproduce the issue? Or at least, some .proto file to reproduce the issue?

Thank you for your response! I made a git repo for the test to reproduce the issue. Besides, my cargo&rustc is rustc 1.53.0 (53cb7b09b 2021-06-17).

[stepancheg]

UnknownFields is deterministic now, see commit a00fe63.

Tested in your repo with these changes:

diff --git a/Cargo.toml b/Cargo.toml
index 5a7d567..198116d 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -7,6 +7,8 @@ edition = "2018"
 [dependencies]
 #ttrpc = { version = "0.5.0", features = ["async"] }
 #protobuf = "=2.14.0"
+protobuf = { path = "../../rust-protobuf/protobuf" }
 
 [build-dependencies]
-ttrpc-codegen = "0.2.0"
+#protoc-rust = "=2.14.0"
+protoc-rust = { path = "../../rust-protobuf/protoc-rust" }
diff --git a/build.rs b/build.rs
index 74390c7..1fd69fb 100644
--- a/build.rs
+++ b/build.rs
@@ -3,7 +3,7 @@
 // SPDX-License-Identifier: Apache-2.0
 //
 
-use ttrpc_codegen::{Codegen, Customize};
+use protoc_rust::{Codegen, Customize};
 
 fn main() {
     let protos = vec![
@@ -18,11 +18,6 @@ fn main() {
         .out_dir("src")
         .inputs(&protos)
         .include("protos")
-        .rust_protobuf()
-        .customize(Customize {
-            async_server: true,
-            ..Default::default()
-        })
         .run()
         .expect("Gen codes failed.");
 }

[stepancheg]

Going to publish new version with fixes soon.

Thank you for the repro command, sorry for the delay in fixing.

[stepancheg]

Published 2.25.0.