CI spuriously failed due to panic in tokio::rt_threaded_alt many_multishot_futures

[NobodyXu]

CI spuriously failed due to same error as #5885 (comment)

Stack backtrace:

--- STDERR:              tokio::rt_threaded_alt many_multishot_futures ---
thread 'tokio-runtime-worker' panicked at 'explicit panic', tokio/src/runtime/scheduler/multi_thread_alt/queue.rs:149:13
stack backtrace:
   0: rust_begin_unwind
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/panicking.rs:593:5
   1: core::panicking::panic_fmt
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/panicking.rs:67:14
   2: core::panicking::panic
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/panicking.rs:117:5
   3: tokio::runtime::scheduler::multi_thread_alt::queue::Local<T>::push_back
             at ./src/runtime/scheduler/multi_thread_alt/queue.rs:149:13
   4: tokio::runtime::scheduler::multi_thread_alt::worker::Worker::next_remote_task_batch_synced
             at ./src/runtime/scheduler/multi_thread_alt/worker.rs:799:9
   5: tokio::runtime::scheduler::multi_thread_alt::worker::Worker::do_park
             at ./src/runtime/scheduler/multi_thread_alt/worker.rs:1154:29
   6: tokio::runtime::scheduler::multi_thread_alt::worker::Worker::park
             at ./src/runtime/scheduler/multi_thread_alt/worker.rs:1125:30
   7: tokio::runtime::scheduler::multi_thread_alt::worker::Worker::next_task
             at ./src/runtime/scheduler/multi_thread_alt/worker.rs:707:46
   8: tokio::runtime::scheduler::multi_thread_alt::worker::Worker::run
             at ./src/runtime/scheduler/multi_thread_alt/worker.rs:570:35
   9: tokio::runtime::scheduler::multi_thread_alt::worker::run::{{closure}}::{{closure}}
             at ./src/runtime/scheduler/multi_thread_alt/worker.rs:502:23
  10: tokio::runtime::context::scoped::Scoped<T>::set
             at ./src/runtime/context/scoped.rs:40:9
  11: tokio::runtime::context::set_scheduler::{{closure}}
             at ./src/runtime/context.rs:176:26
  12: std::thread::local::LocalKey<T>::try_with
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/thread/local.rs:270:16
  13: std::thread::local::LocalKey<T>::with
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/thread/local.rs:246:9
  14: tokio::runtime::context::set_scheduler
             at ./src/runtime/context.rs:176:9
  15: tokio::runtime::scheduler::multi_thread_alt::worker::run::{{closure}}
             at ./src/runtime/scheduler/multi_thread_alt/worker.rs:498:9
  16: tokio::runtime::context::runtime::enter_runtime
             at ./src/runtime/context/runtime.rs:65:16
  17: tokio::runtime::scheduler::multi_thread_alt::worker::run
             at ./src/runtime/scheduler/multi_thread_alt/worker.rs:487:5
  18: tokio::runtime::scheduler::multi_thread_alt::worker::create::{{closure}}
             at ./src/runtime/scheduler/multi_thread_alt/worker.rs:334:49
  19: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
             at ./src/runtime/blocking/task.rs:42:21
  20: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
             at /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tracing-0.1.37/src/instrument.rs:272:9
  21: tokio::runtime::task::core::Core<T,S>::poll::{{closure}}
             at ./src/runtime/task/core.rs:334:17
  22: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at ./src/loom/std/unsafe_cell.rs:16:9
  23: tokio::runtime::task::core::Core<T,S>::poll
             at ./src/runtime/task/core.rs:323:13
  24: tokio::runtime::task::harness::poll_future::{{closure}}
             at ./src/runtime/task/harness.rs:485:19
  25: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/panic/unwind_safe.rs:271:9
  26: std::panicking::try::do_call
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/panicking.rs:500:40
  27: __rust_try
  28: std::panicking::try
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/panicking.rs:464:19
  29: std::panic::catch_unwind
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/panic.rs:142:14
  30: tokio::runtime::task::harness::poll_future
             at ./src/runtime/task/harness.rs:473:18
  31: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at ./src/runtime/task/harness.rs:208:27
  32: tokio::runtime::task::harness::Harness<T,S>::poll
             at ./src/runtime/task/harness.rs:153:15
  33: tokio::runtime::task::raw::poll
             at ./src/runtime/task/raw.rs:276:5
  34: tokio::runtime::task::raw::RawTask::poll
             at ./src/runtime/task/raw.rs:200:18
  35: tokio::runtime::task::UnownedTask<S>::run
             at ./src/runtime/task/mod.rs:437:9
  36: tokio::runtime::blocking::pool::Task::run
             at ./src/runtime/blocking/pool.rs:159:9
  37: tokio::runtime::blocking::pool::Inner::run
             at ./src/runtime/blocking/pool.rs:513:17
  38: tokio::runtime::blocking::pool::Spawner::spawn_thread::{{closure}}
             at ./src/runtime/blocking/pool.rs:471:13
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
worker thread panicking; aborting process

   Canceling due to test failure: 1 tests still running
        PASS [   0.073s] tokio::rt_threaded_alt many_oneshot_futures
        PASS [  12.033s] tokio::rt_threaded many_multishot_futures
------------
     Summary [  31.677s] 709/1164 tests run: 708 passed, 1 failed, 21 skipped
     SIGABRT [   6.435s] tokio::rt_threaded_alt many_multishot_futures

Originally posted by @NobodyXu in #5887 (comment)

[carllerche]

I've been digging into this, but I still have not figured out the cause.