grin-wallet owner_api creating large quantity of threads

[fentanyluser]

the grin-wallet process is generating over 10,000+ threads over time and does not stop until all memory is exhausted resulting in a stack trace. rpc is being used normally to check for transactions etc. issue was not apparent until the 3.0 release.

thread 'tokio-runtime-worker-1' panicked at 'failed to spawn worker thread: Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }': tokio-threadpool-0.1.17/src/pool/mod.rs:390stack backtrace:
0: grin_util::logger::send_panic_to_log::{{closure}}
1: std::panicking::rust_panic_with_hook
at src/libstd/panicking.rs:468
2: std::panicking::continue_panic_fmt
at src/libstd/panicking.rs:373
3: std::panicking::begin_panic_fmt
at src/libstd/panicking.rs:328
4: tokio_threadpool::pool::Pool::spawn_thread
5: tokio_threadpool::pool::Pool::signal_work
6: tokio_threadpool::worker::Worker::run
7: tokio_reactor::with_default
8: tokio::runtime::builder::Builder::build::{{closure}}
9: std::thread::local::LocalKey::with
10: std::thread::local::LocalKey::with
11: std::sys_common::backtrace::__rust_begin_short_backtrace
12: std::panicking::try::do_call
13: __rust_maybe_catch_panic
at src/libpanic_unwind/lib.rs:78
14: core::ops::function::FnOnce::call_once{{vtable.shim}}
15: <alloc::boxed::Box as core::ops::function::FnOnce>::call_once
at /rustc/73528e339aae0f17a15ffa49a8ac608f50c6cf14/src/liballoc/boxed.rs:942
16: <alloc::boxed::Box as core::ops::function::FnOnce>::call_once
at /rustc/73528e339aae0f17a15ffa49a8ac608f50c6cf14/src/liballoc/boxed.rs:942
std::sys_common::thread::start_thread
at src/libstd/sys_common/thread.rs:13
std::sys::unix::thread::Thread::new::thread_start
at src/libstd/sys/unix/thread.rs:79
17: start_thread
18: clone

Thread 'tokio-runtime-worker-1' panicked with message:
"failed to spawn worker thread: Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }"

[jaspervdm]

Thanks for reporting, I'm PRing a fix momentarily

[jaspervdm]

Re-opening issue as according to @fentanyluser the problem is still present. Going to see if there are other places where we create threads and don't clean them up.

[fentanyluser]

to reproduce the issue, start up grin-wallet owner_api and run a command such as

while true; do curl --user user:pass -i -X POST --data-binary '{"jsonrpc":"2.0","id":"curltext","method":"retrieve_summary_info","params":[true, 1]}' -H 'content-type:text/plain;' http://localhost:3420/v2/owner; done

and check the number of threads in /proc/pid/status

[jaspervdm]

I can't reproduce using latest master, the number of threads is stable on macOS, according to the Activity Monitor

[fentanyluser]

OK looks like some was changed in v3.0.1-beta.1, was using v3.0.0 and applied commit to that. Can confirm that it is working now using master.