Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@
#
# 2. radixark/miles:dev-cu13-arm64
# build-arg:ENABLE_CUDA_13=1 \
# build-arg:SGLANG_IMAGE_TAG=v0.5.9-cu130-arm64 \
# build-arg:SGLANG_IMAGE_TAG=v0.5.10-cu130 \

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The SGLANG_IMAGE_TAG in the example comment for the arm64 build has lost its -arm64 suffix. Since the header (line 4) and the WHEELS_TAG (line 7) both specify arm64/aarch64, the image tag should likely include the architecture suffix to ensure the correct base image is pulled, consistent with the previous version.

#    build-arg:SGLANG_IMAGE_TAG=v0.5.10-cu130-arm64 \

# build-arg:WHEELS_TAG=cu130-aarch64 \

ARG SGLANG_IMAGE_TAG=v0.5.9
ARG SGLANG_IMAGE_TAG=v0.5.10
FROM lmsysorg/sglang:${SGLANG_IMAGE_TAG} AS sglang

# ======================================== Arguments =============================================
Expand Down Expand Up @@ -88,7 +88,7 @@ RUN pip install megatron-energon --no-deps
RUN pip install multi-storage-client --no-deps

COPY requirements.txt /tmp/requirements.txt
RUN pip install -r /tmp/requirements.txt
RUN rm -rf /usr/lib/python3/dist-packages/jwt /usr/lib/python3/dist-packages/PyJWT* && pip install -r /tmp/requirements.txt

# https://github.com/pytorch/pytorch/issues/168167
RUN if [ "${ENABLE_CUDA_13}" = "1" ]; then \
Expand Down
2 changes: 1 addition & 1 deletion miles/rollout/session/session_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ def __init__(self, args, backend_url: str):
)

# Close the httpx connection pool when uvicorn shuts down to avoid FD leaks.
self.app.add_event_handler("shutdown", self.client.aclose)
self.app.router.on_shutdown.append(self.client.aclose)

setup_session_routes(self.app, self, args)

Expand Down
2 changes: 1 addition & 1 deletion miles/router/router.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ def __init__(self, args, verbose=False):
self.verbose = verbose

self.app = FastAPI()
self.app.add_event_handler("startup", self._start_background_health_check)
self.app.router.on_startup.append(self._start_background_health_check)

# URL -> Active Request Count (load state)
self.worker_request_counts: dict[str, int] = {}
Expand Down
1 change: 1 addition & 0 deletions tests/fast/utils/chat_template_utils/test_template.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ def _make_serving(tokenizer) -> OpenAIServingChat:
serving.use_dpsk_v32_encoding = False
serving.is_gpt_oss = False
serving.tool_call_parser = None
serving.reasoning_parser = None
return serving


Expand Down
Loading