fix(huntsman): Use tokio::net::TcpListener to enable SO_REUSEADDR on gRPC servers.

[sitaowang1998]

Description

Background

Tonic uses std::net::TcpListener by default, which does not set SO_REUSEADDR when opening the socket.

TCP has a complicated state transition on close down. After a server actively closes its socket listening on a port, the port will enter a TIME_WAIT state and timeout by default 60 seconds for the port to go back to CLOSED state. This is to ensure that stray/duplicate packet still on transmission will timeout on arrival and won't be delivered to newly connected listener.

However, this 60-second wait is too long for a fast crash-recovered server. It tries to bind to a port in TIME_WAIT, and fails if SO_REUSEADDR is not set. It crashes again, and docker compose will restart it after exponential backoff. Thus, it is likely that the crashed server will only become healthy after 60 second TIME_WAIT timeout + 40 second docker compose backoff, which is way too long for critical services like storage server and scheduler.

Fix

This PR fixes the problem by using the tokio::net::TcpListener instead of std::net::TcpListener, the default TCP listener used by tonic. std::net::TcpListener does not set SO_REUSEADDR, while the tokio::net::TcpListener has it on.

This PR:

• In two grpc_server.rs in scheduler and storage, creates a tokio::net::TcpListener and pass it to tonic.
• In two Cargo.toml in aforementioned components, adds net feature for tokio.

Checklist

• The PR satisfies the contribution guidelines.
• This is a breaking change and that has been indicated in the PR title, OR this isn't a
  breaking change.
• Necessary docs have been updated, OR no docs need to be updated.

Validation performed

• The e2e failure recovery test succeeds on another branch, with an unrelated fix that will be put up in another PR.
• GitHub workflow pass.

Summary by CodeRabbit

• Bug Fixes
  • Improved gRPC server startup by explicitly binding to the configured network address.
  • Added clearer error logging when a server cannot bind to its listening address.
  • Updated shutdown handling to work reliably with incoming network connections.

[coderabbitai]

Walkthrough

Both scheduler and storage gRPC servers now explicitly bind TCP listeners, convert them to Tonic TcpIncoming streams, and use incoming-based shutdown serving. Tokio networking support is enabled in both components.

Changes

gRPC TCP listener serving

Layer / File(s)	Summary
Scheduler listener integration components/spider-scheduler/Cargo.toml, components/spider-scheduler/src/bin/grpc_server.rs	Enables Tokio’s net feature and changes the scheduler server to use an explicitly bound TcpListener with TcpIncoming and incoming shutdown handling.
Storage listener integration components/spider-storage/Cargo.toml, components/spider-storage/src/bin/grpc_server.rs	Enables Tokio’s net feature and changes the storage server to use an explicitly bound TcpListener with TcpIncoming and incoming shutdown handling.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

• y-scope/spider#369: Updates the scheduler gRPC server implementation in the same entrypoint.

Suggested reviewers: linzhihao-723

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: switching gRPC servers to tokio::net::TcpListener to enable SO_REUSEADDR.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[LinZhihao-723]

Directly modified the PR title. Notice that the scope should not be spider-grpc-server since we don't have any crates or namespaces of that name.