Flaky test: scenario-tests - OCaml node does not have valid peers

[dannywillems]

See https://github.com/o1-labs/openmina/actions/runs/16937900537/job/48002067413

Error: OCaml node 0 doesn't have valid peers
thread 'rust_as_seed' panicked at /home/runner/work/openmina/openmina/node/testing/src/scenarios/multi_node/connection_discovery.rs:76:9:
OCaml node 0 doesn't have valid peers
stack backtrace:
   0:     0x55fb734577fa - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h14b23c1989cbd5c2
   1:     0x55fb73485ee3 - core::fmt::write::h346b5eee5ed4d7cc
   2:     0x55fb73453253 - std::io::Write::write_fmt::h88e4c2f3647229f3
   3:     0x55fb73457642 - std::sys::backtrace::BacktraceLock::print::h48db11f3fd4983ff
   4:     0x55fb73458c7d - std::panicking::default_hook::{{closure}}::h65db976b9c0d8674
   5:     0x55fb73458a87 - std::panicking::default_hook::h6eb3a1192db1ae36
   6:     0x55fb71614c17 - test::test_main::{{closure}}::h8312a87e3ad01bef
   7:     0x55fb715916cf - connection_discovery_rust_as_seed::rust_as_seed::{{closure}}::{{closure}}::h30290f7f6db06a21
   8:     0x55fb73459328 - std::panicking::rust_panic_with_hook::h370ee1901241e459
   9:     0x55fb734590b6 - std::panicking::begin_panic_handler::{{closure}}::heabfe92676d6d073
  10:     0x55fb73457cd9 - std::sys::backtrace::__rust_end_short_backtrace::h6e22d229d4fdf49e
  11:     0x55fb73458d7c - rust_begin_unwind
  12:     0x55fb71520650 - core::panicking::panic_fmt::hfae270fab21da3e6
  13:     0x55fb71590bf3 - connection_discovery_rust_as_seed::rust_as_seed::{{closure}}::h31fc2e1a3bbe23c2
  14:     0x55fb7159ded6 - tokio::runtime::scheduler::current_thread::Context::enter::h67a380bc694eb6e8
  15:     0x55fb7158e05b - tokio::runtime::context::scoped::Scoped<T>::set::h5b7b36f2b27773f7
  16:     0x55fb7159e24d - tokio::runtime::scheduler::current_thread::CoreGuard::block_on::h8fa7536523e97354
  17:     0x55fb7158e84a - tokio::runtime::context::runtime::enter_runtime::hb814629f726d910d
  18:     0x55fb715b86a5 - tokio::runtime::runtime::Runtime::block_on::hf967d088d30f8dd3
  19:     0x55fb7158715e - core::ops::function::FnOnce::call_once::h770469bf7a384de3
  20:     0x55fb716197ab - test::__rust_begin_short_backtrace::h812692078d2a7065
  21:     0x55fb71619128 - test::run_test::{{closure}}::h427e5ada87ee2030
  22:     0x55fb715dcbfa - std::sys::backtrace::__rust_begin_short_backtrace::ha8680ce782bf69f7
  23:     0x55fb715e0554 - core::ops::function::FnOnce::call_once{{vtable.shim}}::hbc1380969ae25c01
  24:     0x55fb7345ec2b - std::sys::pal::unix::thread::Thread::new::thread_start::h0feaf4a9a4b2ecde
  25:     0x7f4ced663aa4 - <unknown>
  26:     0x7f4ced6f0a34 - clone
  27:                0x0 - <unknown>


failures:
    rust_as_seed

[dannywillems]

2025-09-30T08:31:57.428374Z  WARN mina_core::network: time: "1759221117428362287", kind: "network", "no network config initialized, using default config", config: "devnet"
2025-09-30T08:31:57.458230Z  INFO mina_tree::proofs::verifiers: Verifier index is loaded, time: "1759221117458226592"
2025-09-30T08:31:57.483970Z  INFO mina_tree::proofs::verifiers: Verifier index is loaded, time: "1759221117483966129"
2025-09-30T08:31:57.484107Z  INFO mina_node_testing::cluster: Adding Rust node 0 with config: max_peers=100, snark_worker=None, block_producer=false, time: "1759221117484107651"
2025-09-30T08:31:57.484246Z  INFO mina_node_testing::cluster: Using deterministic peer ID for node 0, time: "1759221117484246772"
2025-09-30T08:31:57.484386Z  INFO mina_node_testing::cluster: Assigned ports for Rust node 0: HTTP=11000, LibP2P=11001, time: "1759221117484386905"
2025-09-30T08:31:57.484399Z  INFO mina_node_testing::cluster: Rust node 0 configured as seed node (no initial peers), time: "1759221117484399354"
2025-09-30T08:31:57.668541Z  INFO mina_node_testing::cluster: Successfully created Rust node 0 at ports HTTP=11000, LibP2P=11001, time: "1759221117668537751"
2025-09-30T08:31:57.669627Z  INFO Server::run{addr: 0.0.0.0:11000}: warp::server: listening on http://0.0.0.0:11000
2025-09-30T08:31:57.806801Z  INFO mina_core::log::action: time: "11", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "TransitionFrontierGenesisInject", summary: "Inject genesis block into the transition frontier"
2025-09-30T08:31:57.806901Z  INFO mina_core::log::action: time: "12", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "BlockProducerBestTipUpdate"
2025-09-30T08:31:57.809743Z  INFO mina_core::log::action: time: "13", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "TransactionPoolBestTipChanged"
2025-09-30T08:31:57.809768Z  INFO node::transaction_pool::transaction_pool_effects: time: "1759221117809768147", kind: "TransactionPoolEffectfulFetchAccounts", summary: "fetching accounts for tx pool"
2025-09-30T08:31:57.820228Z  INFO mina_core::log::action: time: "15", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "TransactionPoolBestTipChangedWithAccounts"
2025-09-30T08:31:57.820326Z  INFO p2p::p2p_state: time: "1759221117820323210", kind: "P2pState new", summary: "Current node's id: 12D3KooWEyoppNCUx8Yx66oV9fJnriXwCcXwDDUA2kj6vnc6iDEp", peer_id_str: "12D3KooWEyoppNCUx8Yx66oV9fJnriXwCcXwDDUA2kj6vnc6iDEp"
2025-09-30T08:31:57.820418Z  INFO mina_core::log::action: time: "16", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pInitializeInitialize", summary: "Initializes p2p layer", chain_id: 29936104443aaf264a7f0192ac64b1c7173198c1ed404c1bcff5e562e05eb7f6
2025-09-30T08:31:57.820664Z  INFO mina_node_testing::cluster: Adding OCaml node 0 with 1 initial peers, block_producer=false, time: "1759221117820663211"
2025-09-30T08:31:57.820680Z  INFO mina_node_testing::node::ocaml::config: Attempting to find local Mina binary: mina, time: "1759221117820679721"
2025-09-30T08:31:57.821289Z  INFO mina_node_testing::node::ocaml::config: Local Mina binary not found, falling back to Docker, time: "1759221117821287253"
2025-09-30T08:31:57.821310Z  INFO mina_node_testing::node::ocaml::config: Pulling default Docker image: gcr.io/o1labs-192920/mina-daemon:3.3.0-alpha1-6929a7e-noble-devnet, time: "1759221117821310249"
3.3.0-alpha1-6929a7e-noble-devnet: Pulling from o1labs-192920/mina-daemon
Digest: sha256:b779080f5543702cc84a9da687bb9773c34b47b0a2df4f2f61a4795d42340117
Status: Image is up to date for gcr.io/o1labs-192920/mina-daemon:3.3.0-alpha1-6929a7e-noble-devnet
gcr.io/o1labs-192920/mina-daemon:3.3.0-alpha1-6929a7e-noble-devnet
2025-09-30T08:31:58.624150Z  INFO mina_node_testing::node::ocaml::config: Successfully pulled Docker image, using DockerDefault, time: "1759221118624141102"
2025-09-30T08:31:58.624466Z  INFO mina_node_testing::cluster: Assigned ports for OCaml node 0: LibP2P=11002, GraphQL=11003, Client=11004, time: "1759221118624463285"
2025-09-30T08:31:58.624920Z  INFO mina_node_testing::node::ocaml::config: Using default Docker image: gcr.io/o1labs-192920/mina-daemon:3.3.0-alpha1-6929a7e-noble-devnet, time: "1759221118624917898"
2025-09-30T08:31:58.624967Z  INFO mina_node_testing::node::ocaml::config: Configuring Docker container: name=mina_testing_ocaml_14147e-1, image=gcr.io/o1labs-192920/mina-daemon:3.3.0-alpha1-6929a7e-noble-devnet, uid=1000, mount=/tmp/t14147e-1, time: "1759221118624966975"
2025-09-30T08:31:58.625004Z  INFO mina_node_testing::node::ocaml::config: Added 2 environment variables to Docker container, time: "1759221118625004119"
2025-09-30T08:31:58.625019Z  INFO mina_node_testing::node::ocaml::config: Docker command configured for container: mina_testing_ocaml_14147e-1, time: "1759221118625019149"
2025-09-30T08:31:58.625109Z  INFO mina_node_testing::node::ocaml: Spawning OCaml daemon process, time: "1759221118625108184"
2025-09-30T08:31:58.626079Z  INFO mina_node_testing::node::ocaml: OCaml daemon process started with PID: 1316081, time: "1759221118626075884"
2025-09-30T08:31:58.626269Z  INFO mina_node_testing::cluster: Successfully started OCaml node 0 with keypair index 0, time: "1759221118626269144"
[localhost:11002] 2025-09-30 08:31:59 UTC [Info] Mina daemon is booting up; built with commit "6929a7ecd1b5cf977ceb674baa6e32764576c109"
[localhost:11002]   
[localhost:11002] 2025-09-30 08:31:59 UTC [Info] Booting may take several seconds, please wait
[localhost:11002]   
[localhost:11002] Using libp2p keypair private-key password from environment variable MINA_LIBP2P_PASS
[localhost:11002] 2025-09-30 08:31:59 UTC [Info] Reading configuration files $config_files
[localhost:11002]   config_files: [
[localhost:11002]   "/var/lib/coda/config_6929a7ec.json", "/tmp/t14147e-1/.config/daemon.json"
[localhost:11002] ]
[localhost:11002] 2025-09-30 08:31:59 UTC [Info] Initializing with runtime configuration. Ledger name: null
[localhost:11002]   
[localhost:11002] 2025-09-30 08:31:59 UTC [Info] Using the constraint constants from the configuration file
[localhost:11002]   
[localhost:11002] 2025-09-30 08:32:02 UTC [Error] Verification failure: downloaded $path and expected SHA3-256 = $expected_hash but it had $computed_hash
[localhost:11002]   path: "/tmp/s3_cache_dir/genesis_ledger_7f8e3c72be57f296e86e1083379c7a09c6131b1567510cf0019eb4aff8b87813.tar.gz"
[localhost:11002]   expected_hash: "2c35c7d89fdf62a8eebea52dc6cd10732d6e66982b83ee9c2a5a139c3ea6725f"
[localhost:11002]   computed_hash: "e2609d4d6a09656577468145a29c89bdc5f857b1b04a0b64c2d0e9f92937d909"
[localhost:11002] 2025-09-30 08:32:02 UTC [Info] Initializing with runtime configuration. Ledger source: $name
[localhost:11002]   name: "jy1wjiJgTkzXr7yL8r5x4ikaNJuikibsRMnkjdH6uqGCsDmR2sf"
[localhost:11002] 2025-09-30 08:32:02 UTC [Error] Could not send error report: Node_error_service was not configured
[localhost:11002]   
[localhost:11002] 2025-09-30 08:32:02 UTC [Fatal] Unhandled top-level exception: $exn
[localhost:11002] Generating crash report
[localhost:11002]   exn: {
[localhost:11002]   "sexp": [
[localhost:11002]     "monitor.ml.Error",
[localhost:11002]     [ "Failure", "Tarball hash mismatch" ],
[localhost:11002]     [
[localhost:11002]       "Raised at Stdlib.failwith in file \"stdlib.ml\", line 29, characters 17-33",
[localhost:11002]       "Called from Async_kernel__Deferred1.M.map.(fun) in file \"src/deferred1.ml\", line 17, characters 40-45",
[localhost:11002]       "Called from Async_kernel__Job_queue.run_job in file \"src/job_queue.ml\" (inlined), line 128, characters 2-5",
[localhost:11002]       "Called from Async_kernel__Job_queue.run_jobs in file \"src/job_queue.ml\", line 169, characters 6-47",
[localhost:11002]       "Caught by monitor try_with_join_or_error at file \"src/lib/genesis_ledger_helper/genesis_ledger_helper.ml\", line 559, characters 41-41"
[localhost:11002]     ]
[localhost:11002]   ],
[localhost:11002]   "backtrace": [
[localhost:11002]     "Raised at Base__Error.raise in file \"src/error.ml\" (inlined), line 8, characters 14-30",
[localhost:11002]     "Called from Mina_cli_entrypoint.load_config_files.(fun) in file \"src/app/cli/src/cli_entrypoint/mina_cli_entrypoint.ml\", line 139, characters 8-23",
[localhost:11002]     "Called from Async_kernel__Deferred1.M.map.(fun) in file \"src/deferred1.ml\", line 17, characters 40-45",
[localhost:11002]     "Called from Async_kernel__Job_queue.run_job in file \"src/job_queue.ml\" (inlined), line 128, characters 2-5",
[localhost:11002]     "Called from Async_kernel__Job_queue.run_jobs in file \"src/job_queue.ml\", line 169, characters 6-47"
[localhost:11002]   ]
[localhost:11002] }
[localhost:11002] 2025-09-30 08:32:02 UTC [Warn] Shutdown before Mina instance was created, not saving a visualization
[localhost:11002]   
[localhost:11002] 
[localhost:11002] 
[localhost:11002]   ☠  Mina Daemon crashed.
[localhost:11002]    The Mina Protocol developers would like to know why!
[localhost:11002] 
[localhost:11002]     Please:
[localhost:11002]       Open an issue:
[localhost:11002]         <https://github.com/MinaProtocol/mina/issues/new>
[localhost:11002] 
[localhost:11002]       Briefly describe what you were doing and attach the crash report /tmp/t14147e-1/.config/coda_crash_report_2025-09-30_08-32-02.713008.tar.gz
[localhost:11002] 
[localhost:11002]     
[localhost:11002] Mina process exited with status code 1
2025-09-30T08:32:28.627227Z  INFO mina_node_testing::cluster: Adding OCaml node 1 with 1 initial peers, block_producer=false, time: "1759221148627215350"
2025-09-30T08:32:28.627422Z  INFO mina_node_testing::cluster: Assigned ports for OCaml node 1: LibP2P=11005, GraphQL=11006, Client=11007, time: "1759221148627422095"
2025-09-30T08:32:28.627583Z  INFO mina_node_testing::node::ocaml::config: Using default Docker image: gcr.io/o1labs-192920/mina-daemon:3.3.0-alpha1-6929a7e-noble-devnet, time: "1759221148627583265"
2025-09-30T08:32:28.627599Z  INFO mina_node_testing::node::ocaml::config: Configuring Docker container: name=mina_testing_ocaml_14147e-2, image=gcr.io/o1labs-192920/mina-daemon:3.3.0-alpha1-6929a7e-noble-devnet, uid=1000, mount=/tmp/t14147e-2, time: "1759221148627599066"
2025-09-30T08:32:28.627619Z  INFO mina_node_testing::node::ocaml::config: Added 2 environment variables to Docker container, time: "1759221148627619587"
2025-09-30T08:32:28.627626Z  INFO mina_node_testing::node::ocaml::config: Docker command configured for container: mina_testing_ocaml_14147e-2, time: "1759221148627626348"
2025-09-30T08:32:28.627663Z  INFO mina_node_testing::node::ocaml: Spawning OCaml daemon process, time: "1759221148627663626"
2025-09-30T08:32:28.628257Z  INFO mina_node_testing::node::ocaml: OCaml daemon process started with PID: 1316757, time: "1759221148628256786"
2025-09-30T08:32:28.628343Z  INFO mina_node_testing::cluster: Successfully started OCaml node 1 with keypair index 1, time: "1759221148628343395"
[localhost:11005] 2025-09-30 08:32:29 UTC [Info] Mina daemon is booting up; built with commit "6929a7ecd1b5cf977ceb674baa6e32764576c109"
[localhost:11005]   
[localhost:11005] 2025-09-30 08:32:29 UTC [Info] Booting may take several seconds, please wait
[localhost:11005]   
[localhost:11005] Using libp2p keypair private-key password from environment variable MINA_LIBP2P_PASS
[localhost:11005] 2025-09-30 08:32:29 UTC [Info] Reading configuration files $config_files
[localhost:11005]   config_files: [
[localhost:11005]   "/var/lib/coda/config_6929a7ec.json", "/tmp/t14147e-2/.config/daemon.json"
[localhost:11005] ]
[localhost:11005] 2025-09-30 08:32:29 UTC [Info] Initializing with runtime configuration. Ledger name: null
[localhost:11005]   
[localhost:11005] 2025-09-30 08:32:29 UTC [Info] Using the constraint constants from the configuration file
[localhost:11005]   
[localhost:11005] 2025-09-30 08:32:33 UTC [Error] Verification failure: downloaded $path and expected SHA3-256 = $expected_hash but it had $computed_hash
[localhost:11005]   path: "/tmp/s3_cache_dir/genesis_ledger_7f8e3c72be57f296e86e1083379c7a09c6131b1567510cf0019eb4aff8b87813.tar.gz"
[localhost:11005]   expected_hash: "2c35c7d89fdf62a8eebea52dc6cd10732d6e66982b83ee9c2a5a139c3ea6725f"
[localhost:11005]   computed_hash: "e2609d4d6a09656577468145a29c89bdc5f857b1b04a0b64c2d0e9f92937d909"
[localhost:11005] 2025-09-30 08:32:33 UTC [Info] Initializing with runtime configuration. Ledger source: $name
[localhost:11005]   name: "jy1wjiJgTkzXr7yL8r5x4ikaNJuikibsRMnkjdH6uqGCsDmR2sf"
[localhost:11005] 2025-09-30 08:32:33 UTC [Error] Could not send error report: Node_error_service was not configured
[localhost:11005]   
[localhost:11005] 2025-09-30 08:32:33 UTC [Fatal] Unhandled top-level exception: $exn
[localhost:11005] Generating crash report
[localhost:11005]   exn: {
[localhost:11005]   "sexp": [
[localhost:11005]     "monitor.ml.Error",
[localhost:11005]     [ "Failure", "Tarball hash mismatch" ],
[localhost:11005]     [
[localhost:11005]       "Raised at Stdlib.failwith in file \"stdlib.ml\", line 29, characters 17-33",
[localhost:11005]       "Called from Async_kernel__Deferred1.M.map.(fun) in file \"src/deferred1.ml\", line 17, characters 40-45",
[localhost:11005]       "Called from Async_kernel__Job_queue.run_job in file \"src/job_queue.ml\" (inlined), line 128, characters 2-5",
[localhost:11005]       "Called from Async_kernel__Job_queue.run_jobs in file \"src/job_queue.ml\", line 169, characters 6-47",
[localhost:11005]       "Caught by monitor try_with_join_or_error at file \"src/lib/genesis_ledger_helper/genesis_ledger_helper.ml\", line 559, characters 41-41"
[localhost:11005]     ]
[localhost:11005]   ],
[localhost:11005]   "backtrace": [
[localhost:11005]     "Raised at Base__Error.raise in file \"src/error.ml\" (inlined), line 8, characters 14-30",
[localhost:11005]     "Called from Mina_cli_entrypoint.load_config_files.(fun) in file \"src/app/cli/src/cli_entrypoint/mina_cli_entrypoint.ml\", line 139, characters 8-23",
[localhost:11005]     "Called from Async_kernel__Deferred1.M.map.(fun) in file \"src/deferred1.ml\", line 17, characters 40-45",
[localhost:11005]     "Called from Async_kernel__Job_queue.run_job in file \"src/job_queue.ml\" (inlined), line 128, characters 2-5",
[localhost:11005]     "Called from Async_kernel__Job_queue.run_jobs in file \"src/job_queue.ml\", line 169, characters 6-47"
[localhost:11005]   ]
[localhost:11005] }
[localhost:11005] 2025-09-30 08:32:33 UTC [Warn] Shutdown before Mina instance was created, not saving a visualization
[localhost:11005]   
[localhost:11005] 
[localhost:11005] 
[localhost:11005]   ☠  Mina Daemon crashed.
[localhost:11005]    The Mina Protocol developers would like to know why!
[localhost:11005] 
[localhost:11005]     Please:
[localhost:11005]       Open an issue:
[localhost:11005]         <https://github.com/MinaProtocol/mina/issues/new>
[localhost:11005] 
[localhost:11005]       Briefly describe what you were doing and attach the crash report /tmp/t14147e-2/.config/coda_crash_report_2025-09-30_08-32-33.845541.tar.gz
[localhost:11005] 
[localhost:11005]     
[localhost:11005] Mina process exited with status code 1
test rust_as_seed has been running for over 60 seconds
^C
➜  mina-rust git:(develop) cargo test --release             --package=mina-node-testing             --package=cli -- --exact rust_as_seed --nocapture
warning: profiles for the non root package will be ignored, specify profiles at the workspace root:
package:   /home/fok/codes/o1-labs/mina-rust/ledger/Cargo.toml
workspace: /home/fok/codes/o1-labs/mina-rust/Cargo.toml
warning: profiles for the non root package will be ignored, specify profiles at the workspace root:
package:   /home/fok/codes/o1-labs/mina-rust/tools/fuzzing/Cargo.toml
workspace: /home/fok/codes/o1-labs/mina-rust/Cargo.toml
    Finished `release` profile [optimized] target(s) in 0.93s
     Running unittests src/lib.rs (target/release/deps/cli-29a288105a4be41e)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 10 filtered out; finished in 0.00s

     Running unittests src/main.rs (target/release/deps/mina-2d9a05f2a2423ab6)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 10 filtered out; finished in 0.00s

     Running tests/bootstrap.rs (target/release/deps/bootstrap-a273c22fc5708296)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.00s

     Running unittests src/lib.rs (target/release/deps/mina_node_testing-e353608af43b318b)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.00s

     Running unittests src/main.rs (target/release/deps/mina_node_testing-4fdcf460f8487950)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests src/bin/runner.rs (target/release/deps/runner-cea9648cce92113c)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/common.rs (target/release/deps/common-dbd5a0baa4b7ea61)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/connection_discovery_ocaml_to_rust.rs (target/release/deps/connection_discovery_ocaml_to_rust-caef04b38267f46d)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.00s

     Running tests/connection_discovery_ocaml_to_rust_via_seed.rs (target/release/deps/connection_discovery_ocaml_to_rust_via_seed-742df0634049a5af)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.00s

     Running tests/connection_discovery_rust_as_seed.rs (target/release/deps/connection_discovery_rust_as_seed-250ce2822d73321e)

running 1 test
2025-09-30T08:33:22.995570Z  WARN mina_core::network: time: "1759221202995555616", kind: "network", "no network config initialized, using default config", config: "devnet"
2025-09-30T08:33:23.026299Z  INFO mina_tree::proofs::verifiers: Verifier index is loaded, time: "1759221203026290727"
2025-09-30T08:33:23.059247Z  INFO mina_tree::proofs::verifiers: Verifier index is loaded, time: "1759221203059239082"
2025-09-30T08:33:23.059403Z  INFO mina_node_testing::cluster: Adding Rust node 0 with config: max_peers=100, snark_worker=None, block_producer=false, time: "1759221203059402956"
2025-09-30T08:33:23.059551Z  INFO mina_node_testing::cluster: Using deterministic peer ID for node 0, time: "1759221203059550989"
2025-09-30T08:33:23.059720Z  INFO mina_node_testing::cluster: Assigned ports for Rust node 0: HTTP=11000, LibP2P=11001, time: "1759221203059720571"
2025-09-30T08:33:23.059733Z  INFO mina_node_testing::cluster: Rust node 0 configured as seed node (no initial peers), time: "1759221203059733123"
2025-09-30T08:33:23.255432Z  INFO mina_node_testing::cluster: Successfully created Rust node 0 at ports HTTP=11000, LibP2P=11001, time: "1759221203255428360"
2025-09-30T08:33:23.256708Z  INFO Server::run{addr: 0.0.0.0:11000}: warp::server: listening on http://0.0.0.0:11000
2025-09-30T08:33:23.399775Z  INFO mina_core::log::action: time: "11", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "TransitionFrontierGenesisInject", summary: "Inject genesis block into the transition frontier"
2025-09-30T08:33:23.399837Z  INFO mina_core::log::action: time: "12", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "BlockProducerBestTipUpdate"
2025-09-30T08:33:23.402214Z  INFO mina_core::log::action: time: "13", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "TransactionPoolBestTipChanged"
2025-09-30T08:33:23.402235Z  INFO node::transaction_pool::transaction_pool_effects: time: "1759221203402234975", kind: "TransactionPoolEffectfulFetchAccounts", summary: "fetching accounts for tx pool"
2025-09-30T08:33:23.411085Z  INFO mina_core::log::action: time: "15", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "TransactionPoolBestTipChangedWithAccounts"
2025-09-30T08:33:23.411186Z  INFO p2p::p2p_state: time: "1759221203411183126", kind: "P2pState new", summary: "Current node's id: 12D3KooWEyoppNCUx8Yx66oV9fJnriXwCcXwDDUA2kj6vnc6iDEp", peer_id_str: "12D3KooWEyoppNCUx8Yx66oV9fJnriXwCcXwDDUA2kj6vnc6iDEp"
2025-09-30T08:33:23.411291Z  INFO mina_core::log::action: time: "16", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pInitializeInitialize", summary: "Initializes p2p layer", chain_id: 29936104443aaf264a7f0192ac64b1c7173198c1ed404c1bcff5e562e05eb7f6
2025-09-30T08:33:23.411528Z  INFO mina_node_testing::cluster: Adding OCaml node 0 with 1 initial peers, block_producer=false, time: "1759221203411526637"
2025-09-30T08:33:23.411548Z  INFO mina_node_testing::node::ocaml::config: Attempting to find local Mina binary: mina, time: "1759221203411548339"
2025-09-30T08:33:23.412197Z  INFO mina_node_testing::node::ocaml::config: Local Mina binary not found, falling back to Docker, time: "1759221203412190976"
2025-09-30T08:33:23.412229Z  INFO mina_node_testing::node::ocaml::config: Pulling default Docker image: gcr.io/o1labs-192920/mina-daemon:3.3.0-alpha1-6929a7e-noble-devnet, time: "1759221203412228759"
3.3.0-alpha1-6929a7e-noble-devnet: Pulling from o1labs-192920/mina-daemon
Digest: sha256:b779080f5543702cc84a9da687bb9773c34b47b0a2df4f2f61a4795d42340117
Status: Image is up to date for gcr.io/o1labs-192920/mina-daemon:3.3.0-alpha1-6929a7e-noble-devnet
gcr.io/o1labs-192920/mina-daemon:3.3.0-alpha1-6929a7e-noble-devnet
2025-09-30T08:33:24.350124Z  INFO mina_node_testing::node::ocaml::config: Successfully pulled Docker image, using DockerDefault, time: "1759221204350115328"
2025-09-30T08:33:24.350405Z  INFO mina_node_testing::cluster: Assigned ports for OCaml node 0: LibP2P=11002, GraphQL=11003, Client=11004, time: "1759221204350402998"
2025-09-30T08:33:24.350795Z  INFO mina_node_testing::node::ocaml::config: Using default Docker image: gcr.io/o1labs-192920/mina-daemon:3.3.0-alpha1-6929a7e-noble-devnet, time: "1759221204350792544"
2025-09-30T08:33:24.350827Z  INFO mina_node_testing::node::ocaml::config: Configuring Docker container: name=mina_testing_ocaml_142034-1, image=gcr.io/o1labs-192920/mina-daemon:3.3.0-alpha1-6929a7e-noble-devnet, uid=1000, mount=/tmp/t142034-1, time: "1759221204350826686"
2025-09-30T08:33:24.350859Z  INFO mina_node_testing::node::ocaml::config: Added 2 environment variables to Docker container, time: "1759221204350858723"
2025-09-30T08:33:24.350874Z  INFO mina_node_testing::node::ocaml::config: Docker command configured for container: mina_testing_ocaml_142034-1, time: "1759221204350873951"
2025-09-30T08:33:24.350990Z  INFO mina_node_testing::node::ocaml: Spawning OCaml daemon process, time: "1759221204350989183"
2025-09-30T08:33:24.351861Z  INFO mina_node_testing::node::ocaml: OCaml daemon process started with PID: 1319110, time: "1759221204351857317"
2025-09-30T08:33:24.352090Z  INFO mina_node_testing::cluster: Successfully started OCaml node 0 with keypair index 0, time: "1759221204352088774"
[localhost:11002] 2025-09-30 08:33:25 UTC [Info] Mina daemon is booting up; built with commit "6929a7ecd1b5cf977ceb674baa6e32764576c109"
[localhost:11002]   
[localhost:11002] 2025-09-30 08:33:25 UTC [Info] Booting may take several seconds, please wait
[localhost:11002]   
[localhost:11002] Using libp2p keypair private-key password from environment variable MINA_LIBP2P_PASS
[localhost:11002] 2025-09-30 08:33:26 UTC [Info] Reading configuration files $config_files
[localhost:11002]   config_files: [
[localhost:11002]   "/var/lib/coda/config_6929a7ec.json", "/tmp/t142034-1/.config/daemon.json"
[localhost:11002] ]
[localhost:11002] 2025-09-30 08:33:26 UTC [Info] Initializing with runtime configuration. Ledger name: null
[localhost:11002]   
[localhost:11002] 2025-09-30 08:33:26 UTC [Info] Using the constraint constants from the configuration file
[localhost:11002]   
[localhost:11002] 2025-09-30 08:33:30 UTC [Info] Loaded genesis ledger from $ledger_file
[localhost:11002]   ledger_file: "/tmp/t142034-1/.config/genesis/genesis_ledger_7f8e3c72be57f296e86e1083379c7a09c6131b1567510cf0019eb4aff8b87813"
[localhost:11002] 2025-09-30 08:33:42 UTC [Info] Daemon will use chain id 29936104443aaf264a7f0192ac64b1c7173198c1ed404c1bcff5e562e05eb7f6
[localhost:11002]   
[localhost:11002] 2025-09-30 08:33:42 UTC [Info] Daemon running protocol version 3.0.0
[localhost:11002]   
[localhost:11002] 2025-09-30 08:33:42 UTC [Info] Creating daemon with commit id: 6929a7ecd1b5cf977ceb674baa6e32764576c109
[localhost:11002]   
2025-09-30T08:33:54.354085Z  INFO mina_node_testing::cluster: Adding OCaml node 1 with 1 initial peers, block_producer=false, time: "1759221234354077539"
2025-09-30T08:33:54.354386Z  INFO mina_node_testing::cluster: Assigned ports for OCaml node 1: LibP2P=11005, GraphQL=11006, Client=11007, time: "1759221234354383853"
2025-09-30T08:33:54.354838Z  INFO mina_node_testing::node::ocaml::config: Using default Docker image: gcr.io/o1labs-192920/mina-daemon:3.3.0-alpha1-6929a7e-noble-devnet, time: "1759221234354834544"
2025-09-30T08:33:54.354873Z  INFO mina_node_testing::node::ocaml::config: Configuring Docker container: name=mina_testing_ocaml_142034-2, image=gcr.io/o1labs-192920/mina-daemon:3.3.0-alpha1-6929a7e-noble-devnet, uid=1000, mount=/tmp/t142034-2, time: "1759221234354871647"
2025-09-30T08:33:54.354914Z  INFO mina_node_testing::node::ocaml::config: Added 2 environment variables to Docker container, time: "1759221234354913648"
2025-09-30T08:33:54.354932Z  INFO mina_node_testing::node::ocaml::config: Docker command configured for container: mina_testing_ocaml_142034-2, time: "1759221234354931661"
2025-09-30T08:33:54.355076Z  INFO mina_node_testing::node::ocaml: Spawning OCaml daemon process, time: "1759221234355074615"
2025-09-30T08:33:54.358998Z  INFO mina_node_testing::node::ocaml: OCaml daemon process started with PID: 1319891, time: "1759221234358989413"
2025-09-30T08:33:54.359164Z  INFO mina_node_testing::cluster: Successfully started OCaml node 1 with keypair index 1, time: "1759221234359163685"
[localhost:11005] 2025-09-30 08:33:55 UTC [Info] Mina daemon is booting up; built with commit "6929a7ecd1b5cf977ceb674baa6e32764576c109"
[localhost:11005]   
[localhost:11005] 2025-09-30 08:33:55 UTC [Info] Booting may take several seconds, please wait
[localhost:11005]   
[localhost:11005] Using libp2p keypair private-key password from environment variable MINA_LIBP2P_PASS
[localhost:11005] 2025-09-30 08:33:56 UTC [Info] Reading configuration files $config_files
[localhost:11005]   config_files: [
[localhost:11005]   "/var/lib/coda/config_6929a7ec.json", "/tmp/t142034-2/.config/daemon.json"
[localhost:11005] ]
[localhost:11005] 2025-09-30 08:33:56 UTC [Info] Initializing with runtime configuration. Ledger name: null
[localhost:11005]   
[localhost:11005] 2025-09-30 08:33:56 UTC [Info] Using the constraint constants from the configuration file
[localhost:11005]   
[localhost:11005] 2025-09-30 08:33:59 UTC [Info] Loaded genesis ledger from $ledger_file
[localhost:11005]   ledger_file: "/tmp/t142034-2/.config/genesis/genesis_ledger_7f8e3c72be57f296e86e1083379c7a09c6131b1567510cf0019eb4aff8b87813"
[localhost:11002] 2025-09-30 08:34:04 UTC [Info] Starting a new prover process
[localhost:11002]   
[localhost:11005] 2025-09-30 08:34:14 UTC [Info] Daemon will use chain id 29936104443aaf264a7f0192ac64b1c7173198c1ed404c1bcff5e562e05eb7f6
[localhost:11005]   
[localhost:11005] 2025-09-30 08:34:14 UTC [Info] Daemon running protocol version 3.0.0
[localhost:11005]   
[localhost:11005] 2025-09-30 08:34:14 UTC [Info] Creating daemon with commit id: 6929a7ecd1b5cf977ceb674baa6e32764576c109
[localhost:11005]   
test rust_as_seed has been running for over 60 seconds
[localhost:11005] 2025-09-30 08:34:46 UTC [Info] Starting a new prover process
[localhost:11005]   
[localhost:11002] 2025-09-30 08:36:55 UTC [Info] Daemon started process of kind "Prover" with pid 123
[localhost:11002]   
[localhost:11002] 2025-09-30 08:36:55 UTC [Info] Starting a new verifier process
[localhost:11002]   
[localhost:11002] 2025-09-30 08:37:16 UTC [Info] Daemon started process of kind "Verifier" with pid 149
[localhost:11002]   
[localhost:11002] 2025-09-30 08:37:16 UTC [Info] Starting a new vrf-evaluator process
[localhost:11002]   
[localhost:11002] 2025-09-30 08:37:17 UTC [Info] Daemon started process of kind "Vrf_evaluator" with pid 174
[localhost:11002]   
[localhost:11002] 2025-09-30 08:37:17 UTC [Info] Custom child process "libp2p_helper" started with pid 181
[localhost:11002]   
[localhost:11002] 2025-09-30 08:37:17 UTC [Info] libp2p peer ID this session is $peer_id
[localhost:11002]   peer_id: "12D3KooWKG1ZakBYEirEWdFYSstTEvxzTxCyuyaebojKthiESfWi"
[localhost:11002] 2025-09-30 08:37:17 UTC [Error] libp2p_helper: failed to set log level debug for routing/record: error: No such logger
[localhost:11002]   
[localhost:11002] 2025-09-30 08:37:17 UTC [Info] Starting transition router
[localhost:11002]   
[localhost:11002] 2025-09-30 08:37:17 UTC [Info] Initializing transition router
[localhost:11002]   
[localhost:11002] 2025-09-30 08:37:17 UTC [Warn] GCLOUD_KEYFILE environment variable not set. Must be set to use upload_blocks_to_gcloud
[localhost:11002]   
[localhost:11002] 2025-09-30 08:37:17 UTC [Info] Mina daemon is connecting
[localhost:11002]   
[localhost:11002] 2025-09-30 08:37:17 UTC [Info] Initializing plugins
[localhost:11002]   
[localhost:11002] 2025-09-30 08:37:17 UTC [Info] Not running uptime service, no URL given
[localhost:11002]   
[localhost:11002] 2025-09-30 08:37:17 UTC [Info] Stopping daemon after 10620 mins and when there are no blocks to be produced
[localhost:11002]   
[localhost:11002] 2025-09-30 08:37:17 UTC [Info] Attempted to turn on snark worker, but snark worker key is set to none
[localhost:11002]   
[localhost:11002] 2025-09-30 08:37:17 UTC [Info] Daemon ready. Clients can now connect
[localhost:11002]   
[localhost:11002] 2025-09-30 08:37:17 UTC [Info] Created GraphQL server at: http://localhost:11003/graphql
[localhost:11002]   
[-] no FUZZER conf specified
2025-09-30T08:37:17.481986Z  WARN p2p::service_impl::mio: time: "1759221437481967666", summary: "Increasing buffer size to 256kb"
2025-09-30T08:37:17.535850Z  INFO mina_core::log::action: time: "330", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pNetworkKademliaUpdateRoutingTable", summary: "Update routing table with peer addresses", peer_id: 2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j, addrs: ["/ip4/127.0.0.1/tcp/11002", "/ip4/127.0.0.1/tcp/11002", "/ip4/192.168.68.132/tcp/11002"]
2025-09-30T08:37:17.537203Z  WARN p2p::service_impl::mio: time: "1759221437537197413", summary: "Increasing buffer size to 1535kb"
2025-09-30T08:37:17.537387Z  WARN node::event_source::event_source_effects: time: "404", kind: "P2pChannelEvent::Opened", peer_id: "2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j", error: "channel not supported"
2025-09-30T08:37:17.537514Z  WARN node::event_source::event_source_effects: time: "405", kind: "P2pChannelEvent::Opened", peer_id: "2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j", error: "channel not supported"
2025-09-30T08:37:17.537739Z  WARN node::event_source::event_source_effects: time: "417", kind: "P2pChannelEvent::Opened", peer_id: "2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j", error: "channel not supported"
2025-09-30T08:37:17.548010Z  INFO mina_core::log::action: time: "434", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pNetworkKademliaStartBootstrap", summary: "Perform local node's Kademlia bootstrap", key: PeerId(2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ)
2025-09-30T08:37:17.591607Z  WARN p2p::service_impl::mio: time: "1759221437591596485", summary: "Increasing buffer size to 1791kb"
[localhost:11002] 2025-09-30 08:37:17 UTC [Info] Mina daemon is listening
[localhost:11002]   
2025-09-30T08:37:17.735163Z  WARN mina_core::log::action: time: "936", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pNetworkSchedulerError", summary: "Fatal connection error", addr: 127.0.0.1:11002 (incoming: true), error: kademlia incoming stream: error converting protobuf message: unsupported RPC kind: GET_PROVIDERS
2025-09-30T08:37:17.816248Z  WARN p2p::connection::incoming::p2p_connection_incoming_reducer: time: "1316", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, summary: "rejecting incoming connection as duplicate", peer_id: 2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j
2025-09-30T08:37:17.833717Z  WARN p2p::connection::incoming::p2p_connection_incoming_reducer: time: "1417", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, summary: "rejecting incoming connection as duplicate", peer_id: 2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j
2025-09-30T08:37:17.910350Z  INFO mina_core::log::action: time: "1614", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pNetworkKademliaUpdateRoutingTable", summary: "Update routing table with peer addresses", peer_id: 2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j, addrs: ["/ip4/192.168.68.132/tcp/11002"]
2025-09-30T08:37:17.910765Z  WARN node::event_source::event_source_effects: time: "1705", kind: "P2pChannelEvent::Opened", peer_id: "2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j", error: "channel not supported"
2025-09-30T08:37:17.910803Z  WARN node::event_source::event_source_effects: time: "1706", kind: "P2pChannelEvent::Opened", peer_id: "2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j", error: "channel not supported"
2025-09-30T08:37:17.910887Z  WARN node::event_source::event_source_effects: time: "1718", kind: "P2pChannelEvent::Opened", peer_id: "2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j", error: "channel not supported"
2025-09-30T08:37:17.913282Z  WARN p2p::service_impl::mio: time: "1759221437913272147", summary: "Increasing buffer size to 1791kb"
2025-09-30T08:37:17.944196Z  WARN mina_core::log::action: time: "2032", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pNetworkSchedulerError", summary: "Fatal connection error", addr: 10.110.200.16:11002 (incoming: true), error: kademlia incoming stream: error converting protobuf message: unsupported RPC kind: GET_PROVIDERS
2025-09-30T08:37:18.071987Z  INFO mina_core::log::action: time: "2703", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pNetworkKademliaUpdateRoutingTable", summary: "Update routing table with peer addresses", peer_id: 2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j, addrs: ["/ip4/192.168.68.132/tcp/11002"]
2025-09-30T08:37:18.072744Z  WARN node::event_source::event_source_effects: time: "2783", kind: "P2pChannelEvent::Opened", peer_id: "2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j", error: "channel not supported"
2025-09-30T08:37:18.072873Z  WARN node::event_source::event_source_effects: time: "2784", kind: "P2pChannelEvent::Opened", peer_id: "2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j", error: "channel not supported"
2025-09-30T08:37:18.073062Z  WARN node::event_source::event_source_effects: time: "2796", kind: "P2pChannelEvent::Opened", peer_id: "2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j", error: "channel not supported"
2025-09-30T08:37:18.127781Z  WARN p2p::service_impl::mio: time: "1759221438127765752", summary: "Increasing buffer size to 2047kb"
2025-09-30T08:37:18.188713Z  WARN mina_core::log::action: time: "3121", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pNetworkSchedulerError", summary: "Fatal connection error", addr: 10.110.200.16:11002 (incoming: true), error: peer reset yamux stream
2025-09-30T08:37:22.449413Z  WARN p2p::connection::incoming::p2p_connection_incoming_reducer: time: "3520", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, summary: "rejecting incoming connection as duplicate", peer_id: 2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j
2025-09-30T08:37:22.477794Z  WARN p2p::connection::incoming::p2p_connection_incoming_reducer: time: "3659", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, summary: "rejecting incoming connection as duplicate", peer_id: 2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j
[localhost:11005] 2025-09-30 08:37:22 UTC [Info] Daemon started process of kind "Prover" with pid 124
[localhost:11005]   
[localhost:11005] 2025-09-30 08:37:22 UTC [Info] Starting a new verifier process
[localhost:11005]   
2025-09-30T08:37:22.491046Z  INFO mina_core::log::action: time: "3835", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pNetworkKademliaUpdateRoutingTable", summary: "Update routing table with peer addresses", peer_id: 2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j, addrs: ["/ip4/127.0.0.1/tcp/11002", "/ip4/127.0.0.1/tcp/11002", "/ip4/192.168.68.132/tcp/11002"]
2025-09-30T08:37:22.491336Z  WARN node::event_source::event_source_effects: time: "3909", kind: "P2pChannelEvent::Opened", peer_id: "2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j", error: "channel not supported"
2025-09-30T08:37:22.491361Z  WARN node::event_source::event_source_effects: time: "3910", kind: "P2pChannelEvent::Opened", peer_id: "2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j", error: "channel not supported"
2025-09-30T08:37:22.491424Z  WARN node::event_source::event_source_effects: time: "3922", kind: "P2pChannelEvent::Opened", peer_id: "2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j", error: "channel not supported"
[localhost:11002] 2025-09-30 08:37:27 UTC [Error] error sending message on stream "1": $error
[localhost:11002]   error: { "string": "only wrote 0 out of 9 bytes error: libp2p error: stream reset" }
[localhost:11002] 2025-09-30 08:37:27 UTC [Warn] failed to reset stream (this means it was probably closed successfully): $error
[localhost:11002]   error: { "string": "internal RPC error: unknown stream_idx" }
[localhost:11002] 2025-09-30 08:37:27 UTC [Error] error sending message on stream "2": $error
[localhost:11002]   error: { "string": "only wrote 0 out of 9 bytes error: libp2p error: stream reset" }
[localhost:11002] 2025-09-30 08:37:27 UTC [Warn] failed to reset stream (this means it was probably closed successfully): $error
[localhost:11002]   error: { "string": "internal RPC error: unknown stream_idx" }
[localhost:11002] 2025-09-30 08:37:28 UTC [Error] error sending message on stream "3": $error
[localhost:11002]   error: { "string": "only wrote 0 out of 9 bytes error: libp2p error: stream reset" }
[localhost:11002] 2025-09-30 08:37:28 UTC [Warn] failed to reset stream (this means it was probably closed successfully): $error
[localhost:11002]   error: { "string": "internal RPC error: unknown stream_idx" }
[localhost:11005] 2025-09-30 08:37:35 UTC [Info] Daemon started process of kind "Verifier" with pid 149
[localhost:11005]   
[localhost:11005] 2025-09-30 08:37:35 UTC [Info] Starting a new vrf-evaluator process
[localhost:11005]   
[localhost:11005] 2025-09-30 08:37:36 UTC [Info] Daemon started process of kind "Vrf_evaluator" with pid 174
[localhost:11005]   
[localhost:11005] 2025-09-30 08:37:36 UTC [Info] Custom child process "libp2p_helper" started with pid 181
[localhost:11005]   
[localhost:11005] 2025-09-30 08:37:36 UTC [Info] libp2p peer ID this session is $peer_id
[localhost:11005]   peer_id: "12D3KooWQoSDqpenkrjCnKCCLnwEMRVRqUfZJXqD8Y8awEnBAJZX"
[localhost:11005] 2025-09-30 08:37:36 UTC [Error] libp2p_helper: failed to set log level debug for routing/record: error: No such logger
[localhost:11005]   
[localhost:11005] 2025-09-30 08:37:36 UTC [Info] Starting transition router
[localhost:11005]   
[localhost:11005] 2025-09-30 08:37:36 UTC [Info] Initializing transition router
[localhost:11005]   
[localhost:11005] 2025-09-30 08:37:36 UTC [Warn] GCLOUD_KEYFILE environment variable not set. Must be set to use upload_blocks_to_gcloud
[localhost:11005]   
[localhost:11005] 2025-09-30 08:37:36 UTC [Info] Mina daemon is connecting
[localhost:11005]   
[localhost:11005] 2025-09-30 08:37:36 UTC [Info] Initializing plugins
[localhost:11005]   
[localhost:11005] 2025-09-30 08:37:36 UTC [Info] Not running uptime service, no URL given
[localhost:11005]   
[localhost:11005] 2025-09-30 08:37:36 UTC [Info] Stopping daemon after 10320 mins and when there are no blocks to be produced
[localhost:11005]   
[localhost:11005] 2025-09-30 08:37:36 UTC [Info] Attempted to turn on snark worker, but snark worker key is set to none
[localhost:11005]   
[localhost:11005] 2025-09-30 08:37:36 UTC [Info] Daemon ready. Clients can now connect
[localhost:11005]   
[localhost:11005] 2025-09-30 08:37:36 UTC [Info] Created GraphQL server at: http://localhost:11006/graphql
[localhost:11005]   
2025-09-30T08:37:36.338092Z  INFO mina_core::log::action: time: "4678", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pNetworkKademliaUpdateRoutingTable", summary: "Update routing table with peer addresses", peer_id: 2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP, addrs: ["/ip4/127.0.0.1/tcp/11005", "/ip4/127.0.0.1/tcp/11005", "/ip4/192.168.68.132/tcp/11005"]
2025-09-30T08:37:36.338362Z  WARN node::event_source::event_source_effects: time: "4752", kind: "P2pChannelEvent::Opened", peer_id: "2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP", error: "channel not supported"
2025-09-30T08:37:36.338385Z  WARN node::event_source::event_source_effects: time: "4753", kind: "P2pChannelEvent::Opened", peer_id: "2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP", error: "channel not supported"
2025-09-30T08:37:36.338447Z  WARN node::event_source::event_source_effects: time: "4765", kind: "P2pChannelEvent::Opened", peer_id: "2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP", error: "channel not supported"
[localhost:11002] 2025-09-30 08:37:36 UTC [Info] Already connected to enough peers, start initialization
[localhost:11002]   
[localhost:11005] 2025-09-30 08:37:36 UTC [Info] Mina daemon is listening
[localhost:11005]   
[localhost:11005] 2025-09-30 08:37:36 UTC [Info] Already connected to enough peers, start initialization
[localhost:11005]   
[localhost:11002] 2025-09-30 08:37:36 UTC [Info] Persistent_frontier.Instance.check_database took 49.591us
[localhost:11002]   
[localhost:11002] 2025-09-30 08:37:36 UTC [Info] Persistent frontier database does not exist
[localhost:11002]   
[localhost:11002] 2025-09-30 08:37:36 UTC [Info] Resetting transition frontier database to new root
[localhost:11002]   
[localhost:11002] 2025-09-30 08:37:36 UTC [Info] Requesting peers for their best tip to do initialization
[localhost:11002]   
[localhost:11005] 2025-09-30 08:37:36 UTC [Info] Persistent_frontier.Instance.check_database took 62.227us
[localhost:11005]   
[localhost:11005] 2025-09-30 08:37:36 UTC [Info] Persistent frontier database does not exist
[localhost:11005]   
[localhost:11005] 2025-09-30 08:37:36 UTC [Info] Resetting transition frontier database to new root
[localhost:11005]   
[localhost:11005] 2025-09-30 08:37:36 UTC [Info] Requesting peers for their best tip to do initialization
[localhost:11005]   
2025-09-30T08:37:36.526485Z  WARN mina_core::log::action: time: "5351", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pNetworkSchedulerError", summary: "Fatal connection error", addr: 127.0.0.1:11005 (incoming: true), error: kademlia incoming stream: error converting protobuf message: unsupported RPC kind: GET_PROVIDERS
2025-09-30T08:37:36.621560Z  WARN p2p::connection::incoming::p2p_connection_incoming_reducer: time: "5787", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, summary: "rejecting incoming connection as duplicate", peer_id: 2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP
2025-09-30T08:37:36.647341Z  INFO mina_core::log::action: time: "6022", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pNetworkKademliaUpdateRoutingTable", summary: "Update routing table with peer addresses", peer_id: 2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP, addrs: ["/ip4/127.0.0.1/tcp/11005", "/ip4/127.0.0.1/tcp/11005", "/ip4/192.168.68.132/tcp/11005"]
2025-09-30T08:37:36.647550Z  WARN node::event_source::event_source_effects: time: "6095", kind: "P2pChannelEvent::Opened", peer_id: "2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP", error: "channel not supported"
2025-09-30T08:37:36.647573Z  WARN node::event_source::event_source_effects: time: "6096", kind: "P2pChannelEvent::Opened", peer_id: "2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP", error: "channel not supported"
2025-09-30T08:37:36.647638Z  WARN node::event_source::event_source_effects: time: "6108", kind: "P2pChannelEvent::Opened", peer_id: "2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP", error: "channel not supported"
2025-09-30T08:37:36.701506Z  WARN mina_core::log::action: time: "6301", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pNetworkSchedulerError", summary: "Fatal connection error", addr: 127.0.0.1:11005 (incoming: true), error: kademlia incoming stream: error converting protobuf message: unsupported RPC kind: GET_PROVIDERS
2025-09-30T08:37:36.757600Z  WARN p2p::connection::incoming::p2p_connection_incoming_reducer: time: "6684", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, summary: "rejecting incoming connection as duplicate", peer_id: 2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP
2025-09-30T08:37:36.806524Z  INFO mina_core::log::action: time: "6992", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pNetworkKademliaUpdateRoutingTable", summary: "Update routing table with peer addresses", peer_id: 2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP, addrs: ["/ip4/127.0.0.1/tcp/11005", "/ip4/127.0.0.1/tcp/11005", "/ip4/192.168.68.132/tcp/11005"]
2025-09-30T08:37:36.806985Z  WARN node::event_source::event_source_effects: time: "7070", kind: "P2pChannelEvent::Opened", peer_id: "2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP", error: "channel not supported"
2025-09-30T08:37:36.807036Z  WARN node::event_source::event_source_effects: time: "7071", kind: "P2pChannelEvent::Opened", peer_id: "2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP", error: "channel not supported"
2025-09-30T08:37:36.807124Z  WARN node::event_source::event_source_effects: time: "7083", kind: "P2pChannelEvent::Opened", peer_id: "2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP", error: "channel not supported"
2025-09-30T08:37:36.907365Z  WARN mina_core::log::action: time: "7405", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pNetworkSchedulerError", summary: "Fatal connection error", addr: 127.0.0.1:11005 (incoming: true), error: peer reset yamux stream
2025-09-30T08:37:36.988984Z  WARN p2p::connection::incoming::p2p_connection_incoming_reducer: time: "7841", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, summary: "rejecting incoming connection as duplicate", peer_id: 2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP
2025-09-30T08:37:37.000859Z  WARN p2p::connection::incoming::p2p_connection_incoming_reducer: time: "7930", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, summary: "rejecting incoming connection as duplicate", peer_id: 2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP
2025-09-30T08:37:37.015126Z  INFO mina_core::log::action: time: "8103", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pNetworkKademliaUpdateRoutingTable", summary: "Update routing table with peer addresses", peer_id: 2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP, addrs: ["/ip4/127.0.0.1/tcp/11005", "/ip4/127.0.0.1/tcp/11005", "/ip4/192.168.68.132/tcp/11005"]
2025-09-30T08:37:37.016073Z  WARN node::event_source::event_source_effects: time: "8179", kind: "P2pChannelEvent::Opened", peer_id: "2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP", error: "channel not supported"
2025-09-30T08:37:37.016149Z  WARN node::event_source::event_source_effects: time: "8180", kind: "P2pChannelEvent::Opened", peer_id: "2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP", error: "channel not supported"
2025-09-30T08:37:37.016296Z  WARN node::event_source::event_source_effects: time: "8192", kind: "P2pChannelEvent::Opened", peer_id: "2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP", error: "channel not supported"
2025-09-30T08:37:37.123524Z  WARN mina_core::log::action: time: "8459", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pNetworkSchedulerError", summary: "Fatal connection error", addr: 127.0.0.1:11005 (incoming: true), error: peer reset yamux stream
2025-09-30T08:37:41.303816Z  WARN p2p::connection::incoming::p2p_connection_incoming_reducer: time: "8981", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, summary: "rejecting incoming connection as duplicate", peer_id: 2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP
2025-09-30T08:37:41.321817Z  WARN p2p::connection::incoming::p2p_connection_incoming_reducer: time: "9075", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, summary: "rejecting incoming connection as duplicate", peer_id: 2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP
2025-09-30T08:37:41.334814Z  INFO mina_core::log::action: time: "9255", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pNetworkKademliaUpdateRoutingTable", summary: "Update routing table with peer addresses", peer_id: 2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP, addrs: ["/ip4/192.168.68.132/tcp/11005"]
2025-09-30T08:37:41.335149Z  WARN node::event_source::event_source_effects: time: "9331", kind: "P2pChannelEvent::Opened", peer_id: "2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP", error: "channel not supported"
2025-09-30T08:37:41.335179Z  WARN node::event_source::event_source_effects: time: "9332", kind: "P2pChannelEvent::Opened", peer_id: "2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP", error: "channel not supported"
2025-09-30T08:37:41.335254Z  WARN node::event_source::event_source_effects: time: "9344", kind: "P2pChannelEvent::Opened", peer_id: "2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP", error: "channel not supported"
[localhost:11005] 2025-09-30 08:37:43 UTC [Info] Fast forward successful
[localhost:11005]   
[localhost:11005] 2025-09-30 08:37:43 UTC [Info] Loaded full frontier and extensions
[localhost:11005]   
[localhost:11005] 2025-09-30 08:37:43 UTC [Info] Successfully loaded frontier
[localhost:11005]   
[localhost:11005] 2025-09-30 08:37:43 UTC [Error] error sending message on stream "1": $error
[localhost:11005]   error: { "string": "only wrote 0 out of 9 bytes error: libp2p error: stream reset" }
[localhost:11005] 2025-09-30 08:37:43 UTC [Error] error sending message on stream "3": $error
[localhost:11005]   error: {
[localhost:11005]   "string": "only wrote 0 out of 15 bytes error: libp2p error: stream reset"
[localhost:11005] }
[localhost:11005] 2025-09-30 08:37:43 UTC [Warn] RPC call error for "get_best_tip"
[localhost:11005]   
[localhost:11005] 2025-09-30 08:37:43 UTC [Error] error sending message on stream "9": $error
[localhost:11005]   error: {
[localhost:11005]   "string": "only wrote 0 out of 15 bytes error: libp2p error: stream reset"
[localhost:11005] }
[localhost:11005] 2025-09-30 08:37:43 UTC [Error] error sending message on stream "8": $error
[localhost:11005]   error: {
[localhost:11005]   "string": "only wrote 0 out of 15 bytes error: libp2p error: stream reset"
[localhost:11005] }
[localhost:11005] 2025-09-30 08:37:43 UTC [Error] error sending message on stream "10": $error
[localhost:11005]   error: {
[localhost:11005]   "string": "only wrote 0 out of 15 bytes error: libp2p error: stream reset"
[localhost:11005] }
[localhost:11005] 2025-09-30 08:37:43 UTC [Warn] failed to reset stream (this means it was probably closed successfully): $error
[localhost:11005]   error: { "string": "internal RPC error: unknown stream_idx" }
[localhost:11005] 2025-09-30 08:37:43 UTC [Warn] failed to reset stream (this means it was probably closed successfully): $error
[localhost:11005]   error: { "string": "internal RPC error: unknown stream_idx" }
[localhost:11005] 2025-09-30 08:37:43 UTC [Warn] failed to reset stream (this means it was probably closed successfully): $error
[localhost:11005]   error: { "string": "internal RPC error: unknown stream_idx" }
[localhost:11005] 2025-09-30 08:37:43 UTC [Warn] failed to reset stream (this means it was probably closed successfully): $error
[localhost:11005]   error: { "string": "internal RPC error: unknown stream_idx" }
[localhost:11002] 2025-09-30 08:37:43 UTC [Info] Fast forward successful
[localhost:11002]   
[localhost:11002] 2025-09-30 08:37:43 UTC [Info] Loaded full frontier and extensions
[localhost:11002]   
[localhost:11002] 2025-09-30 08:37:43 UTC [Info] Successfully loaded frontier
[localhost:11002]   
2025-09-30T08:37:43.547762Z  INFO mina_core::log::action: time: "9841", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pDisconnectionInit", summary: "Initialize disconnection", peer_id: 2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j, reason: failed to receive message from channel: unimplemented rpc __Versioned_rpc.Menu:1
2025-09-30T08:37:47.472379Z  WARN p2p::connection::incoming::p2p_connection_incoming_reducer: time: "10383", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, summary: "rejecting incoming connection as duplicate", peer_id: 2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j
2025-09-30T08:37:47.483152Z  WARN p2p::connection::incoming::p2p_connection_incoming_reducer: time: "10456", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, summary: "rejecting incoming connection as duplicate", peer_id: 2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j
2025-09-30T08:37:47.519732Z  INFO mina_core::log::action: time: "10661", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pNetworkKademliaUpdateRoutingTable", summary: "Update routing table with peer addresses", peer_id: 2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j, addrs: ["/ip4/192.168.0.11/tcp/57128", "/ip4/192.168.68.132/tcp/11002"]
2025-09-30T08:37:47.519950Z  WARN node::event_source::event_source_effects: time: "10737", kind: "P2pChannelEvent::Opened", peer_id: "2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j", error: "channel not supported"
2025-09-30T08:37:47.519971Z  WARN node::event_source::event_source_effects: time: "10738", kind: "P2pChannelEvent::Opened", peer_id: "2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j", error: "channel not supported"
2025-09-30T08:37:47.520038Z  WARN node::event_source::event_source_effects: time: "10750", kind: "P2pChannelEvent::Opened", peer_id: "2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j", error: "channel not supported"
[localhost:11005] 2025-09-30 08:37:52 UTC [Info] Mina daemon is bootstrapping
[localhost:11005]   
[localhost:11005] 2025-09-30 08:37:52 UTC [Info] Successfully loaded frontier, but failed downloaded best tip from network
[localhost:11005]   
[localhost:11005] 2025-09-30 08:37:52 UTC [Info] Local state already in sync
[localhost:11005]   
[localhost:11005] 2025-09-30 08:37:52 UTC [Info] Starting transition frontier controller phase
[localhost:11005]   
[localhost:11005] 2025-09-30 08:37:52 UTC [Info] Mina daemon is synced
[localhost:11005]   
[localhost:11005] 2025-09-30 08:37:52 UTC [Info] Updating new available work took 0.04601478576660156 ms
[localhost:11005]   
[localhost:11002] 2025-09-30 08:37:52 UTC [Error] error sending message on stream "4": $error
[localhost:11002]   error: { "string": "only wrote 0 out of 9 bytes error: libp2p error: stream reset" }
[localhost:11002] 2025-09-30 08:37:52 UTC [Warn] failed to reset stream (this means it was probably closed successfully): $error
[localhost:11002]   error: { "string": "internal RPC error: unknown stream_idx" }
[localhost:11002] 2025-09-30 08:37:53 UTC [Error] error sending message on stream "6": $error
[localhost:11002]   error: { "string": "only wrote 0 out of 9 bytes error: libp2p error: stream reset" }
[localhost:11002] 2025-09-30 08:37:53 UTC [Warn] RPC call error for "get_best_tip"
[localhost:11002]   
[localhost:11002] 2025-09-30 08:37:53 UTC [Info] Mina daemon is bootstrapping
[localhost:11002]   
[localhost:11002] 2025-09-30 08:37:53 UTC [Info] Successfully loaded frontier, but failed downloaded best tip from network
[localhost:11002]   
[localhost:11002] 2025-09-30 08:37:53 UTC [Info] Local state already in sync
[localhost:11002]   
[localhost:11002] 2025-09-30 08:37:53 UTC [Info] Starting transition frontier controller phase
[localhost:11002]   
[localhost:11002] 2025-09-30 08:37:53 UTC [Info] Mina daemon is synced
[localhost:11002]   
[localhost:11002] 2025-09-30 08:37:53 UTC [Info] Updating new available work took 0.0324249267578125 ms
[localhost:11002]   
2025-09-30T08:38:23.497862Z  WARN mina_core::log::action: time: "12206", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pNetworkSchedulerError", summary: "Fatal connection error", addr: 10.110.200.16:11005 (incoming: true), error: peer reset yamux stream
2025-09-30T08:38:26.310970Z  WARN p2p::connection::incoming::p2p_connection_incoming_reducer: time: "12659", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, summary: "rejecting incoming connection as duplicate", peer_id: 2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP
2025-09-30T08:38:26.325902Z  WARN p2p::connection::incoming::p2p_connection_incoming_reducer: time: "12757", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, summary: "rejecting incoming connection as duplicate", peer_id: 2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP
2025-09-30T08:38:26.364501Z  INFO mina_core::log::action: time: "12958", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pNetworkKademliaUpdateRoutingTable", summary: "Update routing table with peer addresses", peer_id: 2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP, addrs: ["/ip4/127.0.0.1/tcp/11005", "/ip4/127.0.0.1/tcp/11005", "/ip4/192.168.0.11/tcp/38884", "/ip4/192.168.68.132/tcp/11005"]
2025-09-30T08:38:26.364646Z  WARN node::event_source::event_source_effects: time: "13032", kind: "P2pChannelEvent::Opened", peer_id: "2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP", error: "channel not supported"
2025-09-30T08:38:26.364658Z  WARN node::event_source::event_source_effects: time: "13033", kind: "P2pChannelEvent::Opened", peer_id: "2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP", error: "channel not supported"
2025-09-30T08:38:26.364692Z  WARN node::event_source::event_source_effects: time: "13045", kind: "P2pChannelEvent::Opened", peer_id: "2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP", error: "channel not supported"
2025-09-30T08:38:27.573272Z  WARN mina_core::log::action: time: "13459", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pNetworkSchedulerError", summary: "Fatal connection error", addr: 10.110.200.16:11002 (incoming: true), error: peer reset yamux stream
2025-09-30T08:38:32.451234Z  WARN p2p::connection::incoming::p2p_connection_incoming_reducer: time: "13892", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, summary: "rejecting incoming connection as duplicate", peer_id: 2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j
2025-09-30T08:38:32.478687Z  WARN p2p::connection::incoming::p2p_connection_incoming_reducer: time: "14056", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, summary: "rejecting incoming connection as duplicate", peer_id: 2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j
2025-09-30T08:38:32.506878Z  INFO mina_core::log::action: time: "14261", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pNetworkKademliaUpdateRoutingTable", summary: "Update routing table with peer addresses", peer_id: 2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j, addrs: ["/ip4/127.0.0.1/tcp/11002", "/ip4/127.0.0.1/tcp/11002", "/ip4/192.168.0.11/tcp/57128", "/ip4/192.168.68.132/tcp/11002"]
2025-09-30T08:38:32.507199Z  WARN node::event_source::event_source_effects: time: "14337", kind: "P2pChannelEvent::Opened", peer_id: "2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j", error: "channel not supported"
2025-09-30T08:38:32.507236Z  WARN node::event_source::event_source_effects: time: "14338", kind: "P2pChannelEvent::Opened", peer_id: "2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j", error: "channel not supported"
2025-09-30T08:38:32.507322Z  WARN node::event_source::event_source_effects: time: "14350", kind: "P2pChannelEvent::Opened", peer_id: "2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j", error: "channel not supported"
2025-09-30T08:39:06.418108Z  WARN mina_core::log::action: time: "15741", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pNetworkSchedulerError", summary: "Fatal connection error", addr: 127.0.0.1:11005 (incoming: true), error: peer reset yamux stream
2025-09-30T08:39:11.281882Z  WARN p2p::connection::incoming::p2p_connection_incoming_reducer: time: "16179", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, summary: "rejecting incoming connection as duplicate", peer_id: 2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP
2025-09-30T08:39:11.315547Z  WARN p2p::connection::incoming::p2p_connection_incoming_reducer: time: "16344", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, summary: "rejecting incoming connection as duplicate", peer_id: 2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP
2025-09-30T08:39:11.328229Z  INFO mina_core::log::action: time: "16524", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pNetworkKademliaUpdateRoutingTable", summary: "Update routing table with peer addresses", peer_id: 2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP, addrs: ["/ip4/127.0.0.1/tcp/11005", "/ip4/127.0.0.1/tcp/11005", "/ip4/192.168.0.11/tcp/38884", "/ip4/192.168.68.132/tcp/11005"]
2025-09-30T08:39:11.328424Z  WARN node::event_source::event_source_effects: time: "16600", kind: "P2pChannelEvent::Opened", peer_id: "2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP", error: "channel not supported"
2025-09-30T08:39:11.328444Z  WARN node::event_source::event_source_effects: time: "16601", kind: "P2pChannelEvent::Opened", peer_id: "2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP", error: "channel not supported"
2025-09-30T08:39:11.328495Z  WARN node::event_source::event_source_effects: time: "16613", kind: "P2pChannelEvent::Opened", peer_id: "2c43BLoehdYqBqAQDhhVYF8jcJb12BghBMk8sfSuAutwoWuC3pP", error: "channel not supported"
2025-09-30T08:39:12.562139Z  WARN mina_core::log::action: time: "17027", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pNetworkSchedulerError", summary: "Fatal connection error", addr: 127.0.0.1:11002 (incoming: true), error: peer reset yamux stream
2025-09-30T08:39:17.489981Z  WARN p2p::connection::incoming::p2p_connection_incoming_reducer: time: "17570", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, summary: "rejecting incoming connection as duplicate", peer_id: 2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j
2025-09-30T08:39:17.496374Z  WARN p2p::connection::incoming::p2p_connection_incoming_reducer: time: "17635", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, summary: "rejecting incoming connection as duplicate", peer_id: 2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j
2025-09-30T08:39:17.545909Z  INFO mina_core::log::action: time: "17840", node_id: 2awmjihpsd9TkdaqEkvrJ2UguAsoigRtgoiKei4AxpA5EVzr6JZ, kind: "P2pNetworkKademliaUpdateRoutingTable", summary: "Update routing table with peer addresses", peer_id: 2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j, addrs: ["/ip4/192.168.0.11/tcp/57128", "/ip4/192.168.68.132/tcp/11002"]
2025-09-30T08:39:17.546154Z  WARN node::event_source::event_source_effects: time: "17916", kind: "P2pChannelEvent::Opened", peer_id: "2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j", error: "channel not supported"
2025-09-30T08:39:17.546176Z  WARN node::event_source::event_source_effects: time: "17917", kind: "P2pChannelEvent::Opened", peer_id: "2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j", error: "channel not supported"
2025-09-30T08:39:17.546243Z  WARN node::event_source::event_source_effects: time: "17929", kind: "P2pChannelEvent::Opened", peer_id: "2bRmtMF4xhF57ERCL74b8PQ3WJ7Jo4au4vdn5ooyVV6Jmt1dF7j", error: "channel not supported"
2025-09-30T08:39:36.453822Z  INFO mina_node_testing::node::ocaml::config: Cleaning up Docker container: mina_testing_ocaml_142034-1 (image: gcr.io/o1labs-192920/mina-daemon:3.3.0-alpha1-6929a7e-noble-devnet), time: "1759221576453814908"
2025-09-30T08:39:36.453865Z  INFO mina_node_testing::node::ocaml::config: Stopping Docker container: mina_testing_ocaml_142034-1, time: "1759221576453865778"
mina_testing_ocaml_142034-1
2025-09-30T08:39:36.694833Z  INFO mina_node_testing::node::ocaml::config: Successfully stopped Docker container: mina_testing_ocaml_142034-1, time: "1759221576694821839"
2025-09-30T08:39:36.694872Z  INFO mina_node_testing::node::ocaml::config: Removing Docker container: mina_testing_ocaml_142034-1, time: "1759221576694872250"
mina_testing_ocaml_142034-1
2025-09-30T08:39:37.061681Z  INFO mina_node_testing::node::ocaml::config: Successfully removed Docker container: mina_testing_ocaml_142034-1, time: "1759221577061675072"
2025-09-30T08:39:37.070629Z  INFO mina_node_testing::node::ocaml::config: Cleaning up Docker container: mina_testing_ocaml_142034-2 (image: gcr.io/o1labs-192920/mina-daemon:3.3.0-alpha1-6929a7e-noble-devnet), time: "1759221577070617522"
2025-09-30T08:39:37.070696Z  INFO mina_node_testing::node::ocaml::config: Stopping Docker container: mina_testing_ocaml_142034-2, time: "1759221577070696236"
mina_testing_ocaml_142034-2
2025-09-30T08:39:37.279450Z  INFO mina_node_testing::node::ocaml::config: Successfully stopped Docker container: mina_testing_ocaml_142034-2, time: "1759221577279438544"
2025-09-30T08:39:37.279497Z  INFO mina_node_testing::node::ocaml::config: Removing Docker container: mina_testing_ocaml_142034-2, time: "1759221577279495486"
mina_testing_ocaml_142034-2
2025-09-30T08:39:37.615211Z  INFO mina_node_testing::node::ocaml::config: Successfully removed Docker container: mina_testing_ocaml_142034-2, time: "1759221577615202456"
test rust_as_seed ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 378.53s

     Running tests/connection_discovery_rust_to_ocaml.rs (target/release/deps/connection_discovery_rust_to_ocaml-ee5080ee5f8ee809)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.00s

     Running tests/connection_discovery_rust_to_ocaml_via_seed.rs (target/release/deps/connection_discovery_rust_to_ocaml_via_seed-10256d35071cbc3d)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.00s

     Running tests/multi_node_initial_joining.rs (target/release/deps/multi_node_initial_joining-c078f1154b91fc39)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.00s

     Running tests/multi_node_peer_discovery.rs (target/release/deps/multi_node_peer_discovery-6d515ab32b05541e)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.00s

     Running tests/multi_node_propagate_block.rs (target/release/deps/multi_node_propagate_block-3300712d92224650)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.00s

     Running tests/node_libp2p_only.rs (target/release/deps/node_libp2p_only-1a79fe7edd70e9d3)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.00s

     Running tests/node_libp2p_with_rust_to_rust_webrtc.rs (target/release/deps/node_libp2p_with_rust_to_rust_webrtc-d0953b9665a469b5)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/p2p_basic_connections.rs (target/release/deps/p2p_basic_connections-572de7c22fac6771)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 5 filtered out; finished in 0.00s

     Running tests/p2p_basic_incoming.rs (target/release/deps/p2p_basic_incoming-265a986dcdb3e53a)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 2 filtered out; finished in 0.00s

     Running tests/p2p_basic_outgoing.rs (target/release/deps/p2p_basic_outgoing-de1aea099fe59697)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 8 filtered out; finished in 0.00s

     Running tests/p2p_kad.rs (target/release/deps/p2p_kad-6457ebd0bc8db41e)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.00s

     Running tests/p2p_pubsub.rs (target/release/deps/p2p_pubsub-8cc415dd1997440e)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.00s

     Running tests/p2p_signaling.rs (target/release/deps/p2p_signaling-fec1d9962ab293fd)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/record_replay.rs (target/release/deps/record_replay-01b8d4f0aad9fd06)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 2 filtered out; finished in 0.00s

     Running tests/single_node.rs (target/release/deps/single_node-6f41f81d617b0a02)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 4 filtered out; finished in 0.00s

   Doc-tests cli

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 5 filtered out; finished in 0.00s

   Doc-tests mina_node_testing

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 4 filtered out; finished in 0.00s

[0xMimir]

The seems to pass now everytime, here it has passed 20 without failing