fix(spider-client): Make the gRPC client's request futures Send.

[sitaowang1998]

Description

Since #396 routed the client's gRPC calls through the spider-utils retry helper, each call site passed an async closure (AsyncFnMut) to call_with_retry. The future such a closure returns is an anonymous, unnameable type, so there was no way to require it to be Send — and in practice it wasn't. As a result, a SpiderClient call could not be tokio::spawned on a multi-threaded runtime, which is the common way callers drive concurrent jobs.

This PR makes every future returned by the spider-client public API Send, and adds compile-time assertions so the guarantee cannot silently regress.

The change has three parts:

1. spider-utils retry helper — bound the returned future as Send. execute_with_retry and call_with_retry now take GrpcCall: FnMut() -> FutureType with FutureType: Future<Output = ...> + Send, replacing the previous AsyncFnMut() -> Result<...>. Naming the future via an explicit type parameter is what lets us attach the + Send bound; an async closure's future cannot be named, so it cannot be bounded. This is the mechanism behind the whole PR. The retry loop still reconstructs a fresh future per attempt — the closure is a future producer, not a single future — so retry semantics are unchanged. The helper's only callers are the two spider-client files updated in this same PR, so no other code is affected.

2. spider-client call sites — plain closures returning owned futures. Each gRPC call now uses a plain move || closure that acquires a pooled client, builds the request, and returns an owned async move { ... } future, rather than an async closure that borrows captured state. Request construction was additionally moved inside the closure for every call so each retry attempt mints its own owned request; Copy id fields (JobId, ResourceGroupId) are used directly, while owned fields (Vec<u8>, String) are cloned per attempt — the same per-attempt clone the previous code already performed, just relocated. This keeps the produced futures free of borrowed, non-Send state.

3. SpiderClient-level compile-time assertions. client.rs now asserts, at compile time and zero runtime cost, that SpiderClient and SpiderClientBuilder are Send + Sync, and that every public async method (submit_job, start_job, cancel_job, get_job_state, get_job_outputs, get_job_error, add_resource_group, verify_resource_group) returns a Send future. The type-level check alone does not cover the futures, so both are needed: a future that captures a non-Send value across an .await would break spawnability while the handle stays Send + Sync.

Note on the retry unit tests: they switched their shared invocation counter from Cell<usize> to AtomicUsize. This is a direct consequence of the new FutureType: Send bound — the test closures capture the counter by shared reference across an .await, and &T is Send only when T: Sync. Cell is !Sync, so it no longer satisfies the bound; AtomicUsize is Sync and is the minimal fix.

Checklist

• The PR satisfies the contribution guidelines.
• This is a breaking change and that has been indicated in the PR title, OR this isn't a
  breaking change.
• Necessary docs have been updated, OR no docs need to be updated.

Validation performed

• Ensure all workflows pass.
• Add compile-time assertion to make sure the request APIs are Send.

Summary by CodeRabbit

• Bug Fixes
  • Improved reliability for job submission, starting, cancellation, status checks, outputs, and error retrieval by re-establishing gRPC call context on retries.
  • Resource group creation and verification now handle transient connection issues more consistently during retries.
• Tests
  • Expanded and strengthened retry unit tests to confirm the expected number of retry attempts.

[coderabbitai]

Walkthrough

The retry helpers now accept futures returned by synchronous closures. Job and resource-group gRPC operations acquire a client from a cloned connection pool within each retry attempt, while compile-time checks validate client and future sendability.

Changes

gRPC retry migration

Layer / File(s)	Summary
Retry helper contract and validation components/spider-utils/src/grpc/retry.rs	Retry helpers use FnMut closures returning Send futures, and tests track and assert retry attempts with AtomicUsize.
Client sendability checks components/spider-client/src/client.rs	Compile-time checks assert SpiderClient and SpiderClientBuilder are Send + Sync, and that public async method futures are Send.
Job and resource-group RPC retry wiring components/spider-client/src/grpc/job.rs, components/spider-client/src/grpc/resource_group.rs	Job and resource-group RPCs acquire clients inside retry closures and construct request data within each attempt.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant SpiderClient
  participant call_with_retry
  participant ConnectionPool
  participant GrpcClient
  SpiderClient->>call_with_retry: submit or invoke RPC
  call_with_retry->>ConnectionPool: get_client() per attempt
  ConnectionPool-->>GrpcClient: client
  GrpcClient-->>call_with_retry: RPC result
  call_with_retry-->>SpiderClient: result or retry

Loading

Possibly related PRs

• y-scope/spider#360: Refactors pooled gRPC client acquisition around retry attempts.
• y-scope/spider#363: Adds the related spider-client gRPC job and resource-group clients.
• y-scope/spider#396: Introduces the retry helper and call_with_retry integration used by these wrappers.

Suggested reviewers: linzhihao-723

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately captures the main change: making SpiderClient gRPC request futures Send.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@components/spider-client/src/grpc/job.rs`:
- Line 125: Replace each per-attempt `let request = request;` binding in
`components/spider-client/src/grpc/job.rs` at lines 125, 154, 183, 214, and 243
with a clone, matching the existing handling in `submit_job`; no other retry
logic changes are needed.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: d6b9cbc2-04c5-43ba-be0c-a2ea7fe28d26

📥 Commits

Reviewing files that changed from the base of the PR and between 1474f09 and 8f74be0.

📒 Files selected for processing (3)

• components/spider-client/src/grpc/job.rs
• components/spider-client/src/grpc/resource_group.rs
• components/spider-utils/src/grpc/retry.rs

[LinZhihao-723]

Some extra polishing.
Directly modified the PR title and PR description.