Skip to content

test(e2e): use local HTTP server instead of httpbin.org for tool-stub tests #6488

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jdx merged 1 commit into from
Sep 30, 2025
Merged

test(e2e): use local HTTP server instead of httpbin.org for tool-stub tests #6488

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 43 additions & 24 deletions e2e/generate/test_generate_tool_stub
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,35 +6,54 @@
# Disable GPG verification to avoid test failures
export MISE_GPG_VERIFY=false

# Find available port
find_available_port() {
python3 -c "import socket; s=socket.socket(); s.bind(('',0)); print(s.getsockname()[1]); s.close()"
}

# Start local HTTP test server
SERVER_PORT=$(find_available_port)
python3 "${TEST_ROOT}/helpers/scripts/tool_stub_test_server.py" "$SERVER_PORT" &
SERVER_PID=$!

# Wait for server to start
sleep 1
Comment on lines +19 to +20
Copy link

Copilot AI Sep 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fixed sleep duration may be unreliable on slower systems. Consider implementing a proper health check by attempting to connect to the server or checking for a specific response before proceeding with tests.

Suggested change
# Wait for server to start
sleep 1
# Wait for server to start (health check)
for i in {1..20}; do
if curl -s "http://127.0.0.1:$SERVER_PORT/status/200" >/dev/null; then
break
fi
sleep 0.5
done
# If still not up, fail
if ! curl -s "http://127.0.0.1:$SERVER_PORT/status/200" >/dev/null; then
echo "Error: Test server did not start in time" >&2
exit 1
fi

Copilot uses AI. Check for mistakes.

# Ensure cleanup on exit
cleanup() {
kill "$SERVER_PID" 2>/dev/null || true
Copy link

Copilot AI Sep 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cleanup function should verify the server process actually terminates. Consider adding a wait period or using kill -0 to check if the process is still running after the kill attempt.

Suggested change
kill "$SERVER_PID" 2>/dev/null || true
kill "$SERVER_PID" 2>/dev/null || true
# Wait up to 5 seconds for process to terminate
for i in {1..10}; do
if ! kill -0 "$SERVER_PID" 2>/dev/null; then
break
fi
sleep 0.5
done
if kill -0 "$SERVER_PID" 2>/dev/null; then
echo "Warning: server process $SERVER_PID did not terminate after kill" >&2
fi

Copilot uses AI. Check for mistakes.
}
trap cleanup EXIT

# Create test project directory
mkdir -p generate_tool_stub_test
cd generate_tool_stub_test

# Test 1: Basic tool stub generation with URL (skip download for speed)
assert_succeed "mise generate tool-stub test-tool-basic --url 'https://httpbin.org/status/200' --skip-download"
assert_succeed "mise generate tool-stub test-tool-basic --url 'http://127.0.0.1:$SERVER_PORT/status/200' --skip-download"

# Verify the generated stub exists and is executable
assert_succeed "test -x test-tool-basic"

# Verify the generated stub contains expected content
assert_contains "cat test-tool-basic" "#!/usr/bin/env -S mise tool-stub"
assert_contains "cat test-tool-basic" 'url = "https://httpbin.org/status/200"'
assert_contains "cat test-tool-basic" 'url = "http://127.0.0.1:'"$SERVER_PORT"'/status/200"'

# Test 2: Tool stub generation with specific version
assert_succeed "mise generate tool-stub versioned-tool --version '1.2.3' --url 'https://httpbin.org/status/200' --skip-download"
assert_succeed "mise generate tool-stub versioned-tool --version '1.2.3' --url 'http://127.0.0.1:$SERVER_PORT/status/200' --skip-download"

assert_contains "cat versioned-tool" 'version = "1.2.3"'

# Test 3: Tool stub generation with platform-specific URLs
assert_succeed "mise generate tool-stub platform-tool --platform-url 'linux-x64:https://httpbin.org/status/200' --platform-url 'darwin-arm64:https://httpbin.org/status/201' --skip-download"
assert_succeed "mise generate tool-stub platform-tool --platform-url 'linux-x64:http://127.0.0.1:$SERVER_PORT/status/200' --platform-url 'darwin-arm64:http://127.0.0.1:$SERVER_PORT/status/201' --skip-download"

assert_contains "cat platform-tool" "[platforms.linux-x64]"
assert_contains "cat platform-tool" 'url = "https://httpbin.org/status/200"'
assert_contains "cat platform-tool" 'url = "http://127.0.0.1:'"$SERVER_PORT"'/status/200"'
assert_contains "cat platform-tool" "[platforms.darwin-arm64]"
assert_contains "cat platform-tool" 'url = "https://httpbin.org/status/201"'
assert_contains "cat platform-tool" 'url = "http://127.0.0.1:'"$SERVER_PORT"'/status/201"'

# Test 4: Tool stub generation with custom binary path
assert_succeed "mise generate tool-stub custom-bin-tool --url 'https://httpbin.org/status/200' --bin 'bin/custom-binary' --skip-download"
assert_succeed "mise generate tool-stub custom-bin-tool --url 'http://127.0.0.1:$SERVER_PORT/status/200' --bin 'bin/custom-binary' --skip-download"

assert_contains "cat custom-bin-tool" 'bin = "bin/custom-binary"'

Expand All @@ -45,18 +64,18 @@ assert_fail "mise generate tool-stub missing-url --skip-download"
assert_fail "mise generate tool-stub invalid-platform --platform-url 'invalid-format' --skip-download"

# Test 7: Valid platform names should work (no restriction)
assert_succeed "mise generate tool-stub custom-platform-tool --platform-url 'my-platform:https://httpbin.org/status/200' --skip-download"
assert_succeed "mise generate tool-stub custom-platform-tool --platform-url 'my-platform:http://127.0.0.1:$SERVER_PORT/status/200' --skip-download"

# Test 8: Verify generated stub content is well-formed TOML
echo '#!/usr/bin/env -S mise tool-stub

version = "1.0.0"
url = "https://httpbin.org/status/200"' >simple-stub
url = "http://127.0.0.1:'"$SERVER_PORT"'/status/200"' >simple-stub
chmod +x simple-stub

# Verify the stub file contains valid TOML content
assert_contains "cat simple-stub" 'version = "1.0.0"'
assert_contains "cat simple-stub" 'url = "https://httpbin.org/status/200"'
assert_contains "cat simple-stub" 'url = "http://127.0.0.1:'"$SERVER_PORT"'/status/200"'

# Test 9: Help command
assert_succeed "mise generate tool-stub --help"
Expand All @@ -67,35 +86,35 @@ assert_contains "mise generate --help" "tool-stub"

# Test 11: Test appending platforms to existing stub
# First create a stub with one platform
assert_succeed "mise generate tool-stub append-test --platform-url 'linux-x64:https://httpbin.org/status/200' --skip-download"
assert_succeed "mise generate tool-stub append-test --platform-url 'linux-x64:http://127.0.0.1:$SERVER_PORT/status/200' --skip-download"
assert_contains "cat append-test" "[platforms.linux-x64]"
assert_contains "cat append-test" 'url = "https://httpbin.org/status/200"'
assert_contains "cat append-test" 'url = "http://127.0.0.1:'"$SERVER_PORT"'/status/200"'

# Now append another platform to the same stub
assert_succeed "mise generate tool-stub append-test --platform-url 'darwin-arm64:https://httpbin.org/status/201' --skip-download"
assert_succeed "mise generate tool-stub append-test --platform-url 'darwin-arm64:http://127.0.0.1:$SERVER_PORT/status/201' --skip-download"

# Verify both platforms exist
assert_contains "cat append-test" "[platforms.linux-x64]"
assert_contains "cat append-test" 'url = "https://httpbin.org/status/200"'
assert_contains "cat append-test" 'url = "http://127.0.0.1:'"$SERVER_PORT"'/status/200"'
assert_contains "cat append-test" "[platforms.darwin-arm64]"
assert_contains "cat append-test" 'url = "https://httpbin.org/status/201"'
assert_contains "cat append-test" 'url = "http://127.0.0.1:'"$SERVER_PORT"'/status/201"'

# Test 12: Test updating existing platform URL
assert_succeed "mise generate tool-stub update-test --platform-url 'linux-x64:https://httpbin.org/status/200' --skip-download"
assert_contains "cat update-test" 'url = "https://httpbin.org/status/200"'
assert_succeed "mise generate tool-stub update-test --platform-url 'linux-x64:http://127.0.0.1:$SERVER_PORT/status/200' --skip-download"
assert_contains "cat update-test" 'url = "http://127.0.0.1:'"$SERVER_PORT"'/status/200"'

# Update the same platform with a new URL
assert_succeed "mise generate tool-stub update-test --platform-url 'linux-x64:https://httpbin.org/status/202' --skip-download"
assert_contains "cat update-test" 'url = "https://httpbin.org/status/202"'
assert_succeed "mise generate tool-stub update-test --platform-url 'linux-x64:http://127.0.0.1:$SERVER_PORT/status/202' --skip-download"
assert_contains "cat update-test" 'url = "http://127.0.0.1:'"$SERVER_PORT"'/status/202"'
# Ensure old URL is not present
assert_not_contains "cat update-test" 'url = "https://httpbin.org/status/200"'
assert_not_contains "cat update-test" 'url = "http://127.0.0.1:'"$SERVER_PORT"'/status/200"'

# Test 13: Test error when trying to change version on existing stub
assert_succeed "mise generate tool-stub version-test --version '1.0.0' --url 'https://httpbin.org/status/200' --skip-download"
assert_succeed "mise generate tool-stub version-test --version '1.0.0' --url 'http://127.0.0.1:$SERVER_PORT/status/200' --skip-download"
assert_contains "cat version-test" 'version = "1.0.0"'

# Try to change version - should fail
assert_fail "mise generate tool-stub version-test --version '2.0.0' --url 'https://httpbin.org/status/200' --skip-download"
assert_fail "mise generate tool-stub version-test --version '2.0.0' --url 'http://127.0.0.1:$SERVER_PORT/status/200' --skip-download"

# Test 14: Test auto-platform detection from URL
# Use a URL that contains platform information
Expand All @@ -110,7 +129,7 @@ assert_contains "cat linux-auto-test" 'url = "https://github.com/BurntSushi/ripg

# Test 16: Test mixed explicit and auto-detected platforms
# First add with explicit platform
assert_succeed "mise generate tool-stub mixed-test --platform-url 'linux-x64:https://httpbin.org/status/200' --skip-download"
assert_succeed "mise generate tool-stub mixed-test --platform-url 'linux-x64:http://127.0.0.1:$SERVER_PORT/status/200' --skip-download"
assert_contains "cat mixed-test" "[platforms.linux-x64]"

# Then add with auto-detected platform
Expand Down Expand Up @@ -146,7 +165,7 @@ cd generate_tool_stub_slow_test

# Test 1: Generate tool stub with checksum and size detection
# Using a small, reliable file for testing
assert_succeed "mise generate tool-stub hello-stub --url 'https://httpbin.org/json'"
assert_succeed "mise generate tool-stub hello-stub --url 'http://127.0.0.1:$SERVER_PORT/json'"
Copy link

@cursor cursor bot Sep 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Test Script Variable Scope Issue

The slow tests section runs as a separate script, so the SERVER_PORT variable defined in the first script is not available. This results in an undefined SERVER_PORT when constructing URLs, causing tests to fail.

Fix in Cursor Fix in Web


# Verify the generated stub exists and is executable
assert_succeed "test -x hello-stub"
Expand Down
65 changes: 65 additions & 0 deletions e2e/helpers/scripts/tool_stub_test_server.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
#!/usr/bin/env python3
"""
HTTP Test Server for tool-stub E2E Testing
Serves mock endpoints to avoid external network dependencies
"""

import http.server
import socketserver
import sys
import json
from pathlib import Path

class ToolStubTestHandler(http.server.SimpleHTTPRequestHandler):
def do_GET(self):
"""Handle GET requests for test endpoints"""
if self.path == '/status/200':
self.send_response(200)
self.send_header('Content-Type', 'text/plain')
self.end_headers()
self.wfile.write(b'OK')
elif self.path == '/status/201':
self.send_response(201)
self.send_header('Content-Type', 'text/plain')
self.end_headers()
self.wfile.write(b'Created')
elif self.path == '/status/202':
self.send_response(202)
self.send_header('Content-Type', 'text/plain')
self.end_headers()
self.wfile.write(b'Accepted')
elif self.path == '/json':
self.send_response(200)
self.send_header('Content-Type', 'application/json')
self.end_headers()
content = json.dumps({
"slideshow": {
"author": "Yours Truly",
"date": "date of publication",
"title": "Sample Slide Show"
}
})
self.wfile.write(content.encode('utf-8'))
else:
# Return 404 for other paths
self.send_error(404, "File not found")

def log_message(self, format, *args):
"""Suppress log messages for cleaner test output"""
pass

def start_server(port):
"""Start the HTTP test server"""
with socketserver.TCPServer(("127.0.0.1", port), ToolStubTestHandler) as httpd:
Copy link

Copilot AI Sep 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TCPServer doesn't handle multiple concurrent connections well by default. Consider using socketserver.ThreadingTCPServer instead to handle concurrent requests during testing.

Suggested change
with socketserver.TCPServer(("127.0.0.1", port), ToolStubTestHandler) as httpd:
with socketserver.ThreadingTCPServer(("127.0.0.1", port), ToolStubTestHandler) as httpd:

Copilot uses AI. Check for mistakes.
print(f"Tool stub test server running on port {port}", flush=True)
try:
httpd.serve_forever()
except KeyboardInterrupt:
print("\nShutting down...", flush=True)

if __name__ == '__main__':
if len(sys.argv) < 2:
print("Usage: tool_stub_test_server.py <port>", file=sys.stderr)
sys.exit(1)
port = int(sys.argv[1])
start_server(port)
Loading