Skip to content

fix(http): retry when a response body drops mid-stream - #14253

Merged
jdx merged 1 commit into
jdx:mainfrom
JamBalaya56562:fix-http-retry-body
Oct 10, 2026
Merged

jdx merged 1 commit into
jdx:mainfrom
JamBalaya56562:fix-http-retry-body

Conversation

@JamBalaya56562

@JamBalaya56562 JamBalaya56562 commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

A request whose response body dropped partway through failed on the first try, even with http_retries set. mise already counts a broken body read as transient, and #14049 extended that to an HTTP/2 stream reset during the read. But the retry loop ended once the response headers arrived, and callers read the body afterwards, outside it:

let resp = self.get_async(url).await?; // retried
let json = resp.json().await?;         // not retried

So a version list or release page cut off mid-transfer failed with error decoding response body, with no retry.

The retry loop now takes a reader and runs it inside each attempt. These callers read the whole body inside the retry:

  • json / json_headers / json_headers_with_headers
  • get_text and everything built on its request, including the cached variants
  • get_bytes
  • a version list read with version_regex. It used to fetch the response and read it outside the loop, so it bypassed get_text's HTML check. It now goes through get_text with a new allow_html(), which keeps the HTML pages that regex parsing expects.

Callers that need the response itself, such as HEAD, streamed downloads and error-body inspection, pass the response through unchanged, so their behavior is the same. get_html is unchanged too: it checks Content-Type before reading, and reading inside the retry would download bodies it currently discards.

This takes over #12974 by @Binlogo, which pr-closer closed after 7 days of failing checks. The HTTP client has since moved to crates/mise-util/src/http.rs, so this ports the change and does not replay it. They are credited as co-author. The original description gives background: a 27 MB GitHub releases page in mise-versions that drops mid-stream. As it says, retrying alone will not always rescue a response that large.

Review threads on #12974:

  • Greptile: the version_regex path was still unretried. Fixed, as described above.
  • CodeRabbit: json_headers resolves auth headers before checking offline mode. Not changed here. That order predates this PR: get_async resolved the headers before get_async_with_headers checked offline mode. It is a separate fix.

Validation

  • New unit tests in crates/mise-util/src/http/tests/retry.rs:
    • get_text, json and get_bytes each get a response cut off before its Content-Length on the first attempt and a full one on the second. Each returns the second body after two connections.
    • allow_html() returns an HTML page, which is still rejected without it.
  • cargo test -p mise-util --lib -- http::tests::retry passes all 33 tests with this change, the four new ones among them.
  • The three retry tests, run against main's code, fail with error decoding response body, the symptom this fixes.

AI-assisted — Tool: Claude Code; model: anthropic/claude-opus-5-5; version: unavailable.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Bug Fixes
    • Text, JSON, and byte requests now retry transient errors encountered while reading response bodies, not just errors encountered when sending requests.
    • Updated authentication settings are retained across retry attempts.
  • New Features
    • Text requests can accept HTML responses when enabled.
    • Version lists using a custom pattern can now be fetched from HTML pages.

@coderabbitai

coderabbitai Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Important

Review skipped

Review was skipped as selected files did not have any reviewable changes.

⚙️ Run configuration
  • Configuration used: Repository YAML (base), Central YAML (inherited), Organization UI (inherited)
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 928cfa17-2aea-44cd-8c2d-e0e14a24ecaa

📥 Commits

Reviewing files that changed from the base of the PR and between 6805afe and c13e392.


You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository YAML (base), Central YAML (inherited), Organization UI (inherited)
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: f0b00f8c-1fdb-4649-b345-bc951f38fbe8

📥 Commits

Reviewing files that changed from the base of the PR and between 5c6e084 and 6805afe.


📒 Files selected for processing (1)
  • crates/mise-util/src/http.rs

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 7 remain after this review.



📝 Walkthrough

Walkthrough

HTTP request paths now retry errors that occur while reading response bodies. Text requests gain an allow_html() option. The regex-configured version-fetch path uses this option.

Changes

HTTP response handling

Layer / File(s) Summary
Shared retry execution
crates/mise-util/src/http.rs
A shared retry operation invokes a response reader during each request attempt. The HTTPS-fallback request wrapper uses the shared retry state and attempt options.
Byte and JSON body retries
crates/mise-util/src/http.rs, crates/mise-util/src/http/tests/retry.rs
Byte and JSON request paths read response bodies inside the retry operation. Tests cover retries after truncated bodies.
Text retries and HTML acceptance
crates/mise-util/src/http.rs, crates/mise-util/src/http/tests/retry.rs, src/backend/version_list.rs
Text requests read response bodies inside retries and add allow_html(). The regex-configured version-fetch path enables HTML acceptance. Tests cover text body retries and HTML acceptance or rejection.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Sequence Diagram(s)

sequenceDiagram
  participant TextRequest
  participant send_with_retries
  participant HTTPClient
  TextRequest->>send_with_retries: send request with response reader
  send_with_retries->>HTTPClient: send request
  HTTPClient-->>send_with_retries: response
  send_with_retries->>send_with_retries: read response body
  send_with_retries->>HTTPClient: retry after body-read error
Loading

Suggested reviewers: jdx


Merge Risk: ⚪ Minimal · up to 6805a

Response-body failures can now be retried, while the inspected JSON and version-list paths retain their existing behavior. No material issue remains before merge.

Pre-merge checks | Passed 4 | Failed 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 42.86% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 21 functions across 3 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main change: retrying HTTP requests when response-body reads fail mid-stream.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@JamBalaya56562
JamBalaya56562 force-pushed the fix-http-retry-body branch 2 times, most recently from 5c6e084 to 6805afe Compare October 9, 2026 18:41
Co-authored-by: Binlogo <binboy@live.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jdx
jdx merged commit bd4494e into jdx:main Oct 10, 2026
43 checks passed
@JamBalaya56562
JamBalaya56562 deleted the fix-http-retry-body branch October 10, 2026 12:38
jdx added a commit that referenced this pull request Oct 10, 2026
)

<!-- entire-trail-link-start -->
https://entire.io/gh/jdx/mise/trails/637
<!-- entire-trail-link-end -->

`test_text_request_allow_html_returns_the_page` (added in #14253) times
out on every run, which fails `unit-macos` on main (run 38058688044) and
reproduces locally on Linux.

Its second half checks that an HTML page is still rejected without
`allow_html()`. For an `http://` URL, `TextRequest::send` retries as
`https://` first. That retry sends a TLS hello to the plain-HTTP canned
server, which drains until `\r\n\r\n` and so blocks until the 2s client
timeout. nextest's CI ceiling is 1s, so both attempts are killed.

The test now uses a small inline listener for that half: it answers the
first connection with the HTML and drops the second, so the https retry
fails immediately. The test goes from timing out to 0.03s, and all 34
`http::tests::retry` tests pass. No production code changes.

*AI-assisted — Tool: Claude Code; model: anthropic/claude-sonnet-5-5;
version: 2.1.293.*

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Low Risk**
> Test-only change in retry.rs; no runtime HTTP behavior is modified.
> 
> **Overview**
> Fixes a flaky/timeout failure in
**`test_text_request_allow_html_returns_the_page`** on the assertion
that HTML is rejected without **`allow_html()`**.
> 
> The second request still uses an **`http://`** URL (so the client’s
**https** retry runs), but the test no longer uses
**`spawn_canned_server`** with two canned responses. It binds a one-off
**`TcpListener`** that serves HTML on the first connection and leaves
the second connection hanging without blocking until the client
timeout—matching how a TLS hello on plain HTTP used to stall the canned
server under nextest’s short CI deadline.
> 
> Comments in the test document that behavior. **No production HTTP code
changes.**
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
a59cf87. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Tests**
* Updated coverage for HTTP-to-HTTPS retry behavior when an HTML
response is followed by a dropped connection.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
mise-en-dev added a commit that referenced this pull request Oct 11, 2026
<!-- entire-trail-link-start -->
https://entire.io/gh/jdx/mise/trails/628
<!-- entire-trail-link-end -->

### 🚀 Features

- **(runtime-symlinks)** add a global link to the version global config
selects by @jdx in [#14271](#14271)

### 🐛 Bug Fixes

- **(history)** skip rewriting unchanged checkpoint metadata on index
rebuild by @jdx in [#14249](#14249)
- **(http)** retry when a response body drops mid-stream by
@JamBalaya56562 in [#14253](#14253)
- **(lock)** decode percent-encoded release tags when checking a locked
URL by @jdx in [#14252](#14252)
- **(schema)** accept outputs = { auto = false } on tasks by
@JamBalaya56562 in [#14265](#14265)

### 📚 Documentation

- **(dotfiles)** add a recommended setup and a readable modes-and-keys
table by @jdx in [#14267](#14267)

### 🧪 Testing

- **(http)** stop the allow_html test waiting out the https retry by
@jdx in [#14285](#14285)

### 📦 Registry

- test ansible, azure and dvc against python 3.14 by @jdx in
[#14268](#14268)

## 📦 Aqua Registry Updates

### New Packages (2)

- [`runxhq/runx`](https://github.com/runxhq/runx)
- [`takumin/passt-bin`](https://github.com/takumin/passt-bin)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants