test(e2e): use local HTTP server instead of httpbin.org for tool-stub tests by jdx · Pull Request #6488 · jdx/mise

jdx · 2025-09-30T14:52:06Z

Summary

Replace external httpbin.org dependency with a local Python HTTP server for the generate tool-stub e2e tests
Add tool_stub_test_server.py: A lightweight HTTP server that mocks the httpbin.org endpoints needed for testing
Update all test URLs to use the local server instead of external dependencies

Benefits

More reliable: No external network dependencies or rate limiting
Faster: No network latency from external requests
Consistent: Follows the same pattern used in other e2e tests (test_http_compressed_binaries, test_task_standalone)

Test plan

Verified test passes locally with mise run test:e2e test_generate_tool_stub
All 19 test cases pass with the local HTTP server
Server properly handles cleanup on exit

🤖 Generated with Claude Code

Note

Replace httpbin.org dependencies in tool-stub e2e tests with a local Python HTTP server, updating test URLs and adding a mock server script.

E2E Tests (e2e/generate/test_generate_tool_stub):
- Start a local HTTP server on a dynamic port; add cleanup via trap.
- Replace https://httpbin.org/... with http://127.0.0.1:$SERVER_PORT/... across tests; update corresponding assertions.
- Slow tests: switch hello-stub URL to local /json; retain real GitHub release test.
New Helper: e2e/helpers/scripts/tool_stub_test_server.py
- Minimal HTTP server exposing /status/200, /status/201, /status/202, and /json endpoints; silent logging.

^{Written by Cursor Bugbot for commit 8a3f960. This will update automatically on new commits. Configure here.}

… tests Replace external httpbin.org dependency with a local Python HTTP server for the generate tool-stub e2e tests. This makes the tests more reliable, faster, and eliminates external network dependencies. Changes: - Add tool_stub_test_server.py: A lightweight HTTP server that mocks the httpbin.org endpoints (/status/200, /status/201, /status/202, /json) needed for tool-stub testing - Update test_generate_tool_stub to start the local server on a dynamic port before running tests - Replace all httpbin.org URLs with local server URLs (http://127.0.0.1:$SERVER_PORT) The test server follows the same pattern used in other e2e tests (test_http_compressed_binaries, test_task_standalone) for consistency. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

Copilot

Pull Request Overview

This PR replaces external httpbin.org dependencies with a local Python HTTP server for the generate tool-stub e2e tests. This change improves test reliability by eliminating external network dependencies and potential rate limiting issues.

Key changes:

Added a local HTTP test server that mocks httpbin.org endpoints
Updated all test URLs to use the local server instead of external httpbin.org
Implemented proper server lifecycle management with cleanup on exit

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
e2e/helpers/scripts/tool_stub_test_server.py	New Python HTTP server that mocks httpbin.org endpoints needed for testing
e2e/generate/test_generate_tool_stub	Updated test script to use local server instead of external httpbin.org URLs

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-09-30T14:52:57Z

e2e/helpers/scripts/tool_stub_test_server.py

+
+def start_server(port):
+    """Start the HTTP test server"""
+    with socketserver.TCPServer(("127.0.0.1", port), ToolStubTestHandler) as httpd:


TCPServer doesn't handle multiple concurrent connections well by default. Consider using socketserver.ThreadingTCPServer instead to handle concurrent requests during testing.

Suggested change

with socketserver.TCPServer(("127.0.0.1", port), ToolStubTestHandler) as httpd:

with socketserver.ThreadingTCPServer(("127.0.0.1", port), ToolStubTestHandler) as httpd:

Copilot · 2025-09-30T14:52:57Z

e2e/generate/test_generate_tool_stub

+
+# Ensure cleanup on exit
+cleanup() {
+	kill "$SERVER_PID" 2>/dev/null || true


The cleanup function should verify the server process actually terminates. Consider adding a wait period or using kill -0 to check if the process is still running after the kill attempt.

Suggested change

kill "$SERVER_PID" 2>/dev/null || true

kill "$SERVER_PID" 2>/dev/null || true

# Wait up to 5 seconds for process to terminate

for i in {1..10}; do

if ! kill -0 "$SERVER_PID" 2>/dev/null; then

break

fi

sleep 0.5

done

if kill -0 "$SERVER_PID" 2>/dev/null; then

echo "Warning: server process $SERVER_PID did not terminate after kill" >&2

fi

Copilot · 2025-09-30T14:52:58Z

e2e/generate/test_generate_tool_stub

+# Wait for server to start
+sleep 1


The fixed sleep duration may be unreliable on slower systems. Consider implementing a proper health check by attempting to connect to the server or checking for a specific response before proceeding with tests.

Suggested change

# Wait for server to start

sleep 1

# Wait for server to start (health check)

for i in {1..20}; do

if curl -s "http://127.0.0.1:$SERVER_PORT/status/200" >/dev/null; then

break

fi

sleep 0.5

done

# If still not up, fail

if ! curl -s "http://127.0.0.1:$SERVER_PORT/status/200" >/dev/null; then

echo "Error: Test server did not start in time" >&2

exit 1

fi

cursor · 2025-09-30T14:54:38Z

e2e/generate/test_generate_tool_stub

 # Test 1: Generate tool stub with checksum and size detection
 # Using a small, reliable file for testing
-assert_succeed "mise generate tool-stub hello-stub --url 'https://httpbin.org/json'"
+assert_succeed "mise generate tool-stub hello-stub --url 'http://127.0.0.1:$SERVER_PORT/json'"


Bug: Test Script Variable Scope Issue

The slow tests section runs as a separate script, so the SERVER_PORT variable defined in the first script is not available. This results in an undefined SERVER_PORT when constructing URLs, causing tests to fail.

github-actions · 2025-09-30T15:05:41Z

Hyperfine Performance

`mise x -- echo`

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`mise-2025.9.24 x -- echo`	15.3 ± 0.5	14.4	20.0	1.00
`mise x -- echo`	15.5 ± 0.4	14.4	17.0	1.01 ± 0.04

`mise env`

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`mise-2025.9.24 env`	14.7 ± 0.4	14.1	19.1	1.00
`mise env`	14.9 ± 0.4	14.1	17.8	1.01 ± 0.04

`mise hook-env`

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`mise-2025.9.24 hook-env`	14.6 ± 0.3	13.9	15.9	1.00
`mise hook-env`	14.7 ± 0.3	13.9	16.8	1.01 ± 0.03

`mise ls`

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`mise-2025.9.24 ls`	13.5 ± 0.4	12.7	15.7	1.00
`mise ls`	13.6 ± 0.3	12.9	14.7	1.01 ± 0.04

`xtasks/test/perf`

Command	mise-2025.9.24	mise	Variance
install (cached)	132ms	✅ 78ms	+69%
ls (cached)	53ms	53ms	+0%
bin-paths (cached)	55ms	56ms	-1%
task-ls (cached)	368ms	376ms	-2%

✅ Performance improvement: install cached is 69%

@jdx

### 🐛 Bug Fixes - **(auto-install)** support installing non-active backend versions by @jdx in [#6484](#6484) - **(task)** prevent hang when tasks with multiple dependencies fail by @stempler in [#6481](#6481) ### 🧪 Testing - **(e2e)** use local HTTP server instead of httpbin.org for tool-stub tests by @jdx in [#6488](#6488) ### New Contributors - @stempler made their first contribution in [#6481](#6481)  --- > [!NOTE] > Bumps to 2025.9.25 with changelog updates (bug fixes, testing), adds `mdsf` to the aqua registry, and updates completions and packaging metadata. > > - **Release v2025.9.25** > - Update versions in `Cargo.toml`, `Cargo.lock`, `default.nix`, `packaging/rpm/mise.spec`, and `README.md`. > - Refresh shell completion cache spec filenames in `completions/_mise`, `completions/mise.bash`, and `completions/mise.fish`. > - Expand `CHANGELOG.md` with 2025.9.25 entry (bug fixes and testing notes). > - **Registry** > - Add `hougesen/mdsf` package: `crates/aqua-registry/aqua-registry/pkgs/hougesen/mdsf/registry.yaml`. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit 6debaab. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup>  Co-authored-by: mise-en-dev <release@mise.jdx.dev>

Copilot AI review requested due to automatic review settings September 30, 2025 14:52

jdx enabled auto-merge (squash) September 30, 2025 14:52

Copilot AI reviewed Sep 30, 2025

View reviewed changes

cursor bot reviewed Sep 30, 2025

View reviewed changes

jdx merged commit 8d4a4a4 into main Sep 30, 2025
27 checks passed

jdx deleted the test/use-local-http-server-for-tool-stub-test branch September 30, 2025 15:03

jdx mentioned this pull request Sep 30, 2025

chore: release 2025.9.25 #6479

Merged

BrewTestBot mentioned this pull request Sep 30, 2025

mise 2025.9.25 Homebrew/homebrew-core#246370

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

test(e2e): use local HTTP server instead of httpbin.org for tool-stub tests#6488

test(e2e): use local HTTP server instead of httpbin.org for tool-stub tests#6488
jdx merged 1 commit intomainfrom
test/use-local-http-server-for-tool-stub-test

jdx commented Sep 30, 2025 •

edited by cursor bot

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Sep 30, 2025

Uh oh!

Copilot AI Sep 30, 2025

Uh oh!

Copilot AI Sep 30, 2025

Uh oh!

cursor bot Sep 30, 2025

Uh oh!

Uh oh!

github-actions bot commented Sep 30, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	with socketserver.TCPServer(("127.0.0.1", port), ToolStubTestHandler) as httpd:
	with socketserver.ThreadingTCPServer(("127.0.0.1", port), ToolStubTestHandler) as httpd:

-	kill "$SERVER_PID" 2>/dev/null || true
+	kill "$SERVER_PID" 2>/dev/null || true
+	# Wait up to 5 seconds for process to terminate
+	for i in {1..10}; do
+		if ! kill -0 "$SERVER_PID" 2>/dev/null; then
+			break
+		fi
+		sleep 0.5
+	done
+	if kill -0 "$SERVER_PID" 2>/dev/null; then
+		echo "Warning: server process $SERVER_PID did not terminate after kill" >&2
+	fi

-# Wait for server to start
-sleep 1
+# Wait for server to start (health check)
+for i in {1..20}; do
+    if curl -s "http://127.0.0.1:$SERVER_PORT/status/200" >/dev/null; then
+        break
+    fi
+    sleep 0.5
+done
+# If still not up, fail
+if ! curl -s "http://127.0.0.1:$SERVER_PORT/status/200" >/dev/null; then
+    echo "Error: Test server did not start in time" >&2
+    exit 1
+fi

Uh oh!

Conversation

jdx commented Sep 30, 2025 • edited by cursor bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Benefits

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Sep 30, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 30, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 30, 2025

Choose a reason for hiding this comment

Uh oh!

cursor bot Sep 30, 2025

Choose a reason for hiding this comment

Bug: Test Script Variable Scope Issue

Uh oh!

Uh oh!

github-actions bot commented Sep 30, 2025

Hyperfine Performance

mise x -- echo

mise env

mise hook-env

mise ls

xtasks/test/perf

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jdx commented Sep 30, 2025 •

edited by cursor bot

Loading

`mise x -- echo`

`mise env`

`mise hook-env`

`mise ls`

`xtasks/test/perf`