Integrate TaskManager in reth main

[mattsse]

Describe the feature

The tasks crate provides TaskManager and TaskExecutor

The TaskExecutor is used for spawning new tasks (can be critical tasks, e.g. network tasks, or regular, like p2p session tasks)

The TaskManager is notified when a critical task has panicked, which can be used as an exit condition.

It's intended that the TaskManager is the main entry point when launching the node.

For example, this will run until a critical task has panicked

let panicked = manager.next().await;

// this will terminate _all_ tasks spawned via an executor
drop(manager)

The shutdown process needs some additional work, I guess ideally we want to listen for ctrl-c -> shutdown tasks

https://github.com/paradigmxyz/reth/blob/main/crates/tasks/src/lib.rs

Additional context

ref #807

[literallymarvellous]

I would like to take this.

Currently, cargo run --bin reth node is returning a Could not load config error.

[mattsse]

appreciate it, but this would do more harm atm, since there's a bunch of work underway for the CLI

blocked by
#1017
#963

If you want to take on something that's slightly more challenging but can be done incrementally #1037 would be a good start

I believe the config error is due to #1046

[literallymarvellous]

Alright, I'll take 1037.

[onbjerg]

Should no longer be blocked. One note though is that I want this to be optional, i.e. we should try to not make it a requirement to use the task manager for crate consumers

[literallymarvellous]

Alright, will still like to take this.

Concerning making TaskManager optional, how would that work? Does reth main run without the task manager by default, maybe have a cli flag for the task manager functionality?

[mattsse]

same as:

reth/crates/net/network/src/config.rs

Lines 70 to 71 in e8d7c05

	/// The executor to use for spawning tasks.
	pub executor: Option<TaskExecutor>,

I think we need a helper type for Option<TaskExecutor>
perhaps MaybeTaskManager(Option<TaskExecutor>)

[onbjerg]

Blocked by #1079 now, sorry

maybe have a cli flag for the task manager functionality?

No, we should always use the task manager in the CLI. The reason it should be optional is in the case where you are using the crates yourself and you would like to use your own runtime. We should strive to not use tokio::task and tokio::spawn at all, if possible

Ideally we would simply get the things we can spawn, and then spawn them in the CLI. E.g. instead of start_network existing, we should probably just have it return the tasks we need to spawn (NetworkManager etc) and then spawn them ourselves

[literallymarvellous]

oh, okay. I get it now.