fix(runtime): cancel TCP request readers while the input buffer is full

[bojiang-li]

Summary

When a bidirectional engine retains its request input but stops consuming it, handle_request_reader can block in bytes_tx.send(data).await. Its outer cancellation branches are no longer polled, so a local context kill or stop leaves the TCP reader and its channel senders alive until the consumer drains or drops the receiver.

Poll the existing kill/stop futures while forwarding each input frame. Either local signal exits the reader and drops the pending frame, matching the existing request-reader cancellation behavior. Receiver drop, FIFO delivery, upstream cancellation accounting, and wire protocol remain unchanged. Unlike local context cancellation, a Stop/Kill frame behind input data on the same socket still follows that data in wire order.

The production caller is Ingress<ManyIn<T>, ManyOut<U>>::parse_and_build_request: a capacity-8 decoded-input forwarder can retain the capacity-64 transport receiver while the engine stops consuming. This change closes the transport reader's senders; it does not claim to cancel every higher-layer forwarder or release GPU resources.

Five real loopback-TCP tests use a read-completion notification and a full capacity-1 channel to establish the blocked send without sleeps. They cover local kill/stop, consumer drop, healthy drain, and ordered remote kill. Existing control-message tests distinguish remote Stop and Kill context state and cancellation accounting. The notification wrapper exists only in the test module; production reuses the existing cancellation futures.

Related: #15001 fixes the opposite, response-receive direction, where graceful Stop must preserve pending response data. This request-input path already exits on either local Stop or Kill. Reviewed #9991, #11149, #9674, #14849 and #13786; none covers this full request-channel wait.

Validation

• Original before/after investigation: six targeted cases, 4 passed / 2 failed before the production fix; only local kill and stop timed out. The subsequent review follow-up removes the redundant full-channel RemoteStop case and its helper branch. The retained full-channel RemoteKill case covers FIFO ordering; existing direct Stop/Kill control tests retain their distinct state and counter checks.
• Current follow-up: cargo test -p dynamo-runtime --lib pipeline::network::tcp::client::tests -- --nocapture: 40 passed, including all five retained full-channel cases and both direct Stop/Kill control tests.
• cargo fmt --all -- --check, cargo clippy -p dynamo-runtime --lib --tests -- -D warnings -A unused-variables, changed-file pre-commit, and git diff --check: passed. The Clippy allowance is for the existing macOS-only join_reuseport unused parameter in quic_response.rs.
• The follow-up changes only test code; the production portion of client.rs is byte-identical to the previous PR head.
• Upstream Pre Merge CI passes for the current head aef7484f8066edbd809c68ac06a9d0873c12eab0. The runtime library reports 805 passed, 0 failed, 2 ignored; all five retained full-channel tests and the existing Stop/Kill control-message tests explicitly pass. The Pre Merge run also completes doc tests, additional feature checks, benchmark compilation, Clippy, and pre-commit. The separate full GPU/deployment pipelines have not been triggered for this head; they require maintainer approval.

AI-assisted investigation and implementation. CPU validation on macOS only; no GPU or end-to-end model-serving claim. The initial sandbox run failed before TCP setup because loopback listening was denied; the results above come from the permitted local-network run.

Summary by CodeRabbit

• Bug Fixes

  • Local context cancellation can now interrupt blocked data sends.
  • Improved handling of downstream disconnects, normal draining, and upstream control messages while preserving message order.

• Tests

  • Added coverage for full-channel scenarios, including forwarding behavior, channel closure, cancellation state, and metrics.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[github-actions]

👋 Hi bojiang-li! Thank you for contributing to ai-dynamo/dynamo.

Just a reminder: The NVIDIA Test Github Validation CI runs an essential subset of the testing framework to quickly catch errors.Your PR reviewers may elect to test the changes comprehensively before approving your changes.

🚀

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 78c2733d-794b-498c-a8c6-262ff1def24e

📥 Commits

Reviewing files that changed from the base of the PR and between 7831037 and c61b20f.

📒 Files selected for processing (1)

• lib/runtime/src/pipeline/network/tcp/client.rs

Included review availability: Your plan provides up to 12 included reviews per hour; 10 remain after this review.

---

Walkthrough

The TCP request reader now allows local kill and stop signals to interrupt blocked sends. Tests cover full-channel behavior for local and remote cancellation, receiver closure, and normal draining.

Changes

TCP request cancellation handling

Layer / File(s)	Summary
Reader cancellation and scenario validation lib/runtime/src/pipeline/network/tcp/client.rs	handle_request_reader selects between data sends and local kill or stop signals. Tests cover six full-channel scenarios, FIFO delivery, cancellation state and counts, task termination, channel closure, and sender cleanup.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Merge Risk: ⚪ Minimal · up to c61b2

The cancellation change is mergeable based on the available evidence.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 63.64% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 11 functions across 1 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly and concisely describes the main change: enabling TCP request readers to cancel while blocked by a full input buffer.
Description check	✅ Passed	The description provides a detailed overview, implementation details, reviewer context, related issue information, and validation results. It does not use the template headings exactly and does not in…

• Fix all pre-merge checks with AI

---

_{Comment @coderabbitai help to get the list of available commands.}