Skip to content

[Debug] Enable curl retry aligned with openai #1539

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Gaohan123 merged 2 commits into from
Mar 3, 2026
+14 −10
Merged

[Debug] Enable curl retry aligned with openai #1539

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0186eff
enable curl retry aligned with openai
tzhouam Feb 27, 2026
5fbe4b8
Refactor curl request handling in multimodal generation scripts to ut…
tzhouam Mar 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions examples/online_serving/qwen2_5_omni/run_curl_multimodal_generation.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -160,10 +160,7 @@ esac
echo "Running query type: $QUERY_TYPE"
echo ""


output=$(curl -sS -X POST http://localhost:8091/v1/chat/completions \
-H "Content-Type: application/json" \
Copy link
Copy Markdown
Collaborator

@hsliuustc0106 hsliuustc0106 Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good: Configurable retry parameters

Defining MAX_RETRIES and RETRY_DELAY as variables makes it easy to adjust the retry behavior if needed.

Suggestion: Consider making these configurable via environment variables:

MAX_RETRIES=${MAX_RETRIES:-3}
RETRY_DELAY=${RETRY_DELAY:-3}

This would allow CI or users to override the defaults without modifying the script.

-d @- <<EOF
request_body=$(cat <<EOF
{
"model": "Qwen/Qwen2.5-Omni-7B",
"sampling_params_list": $sampling_params_list,
Expand All @@ -186,7 +183,12 @@ output=$(curl -sS -X POST http://localhost:8091/v1/chat/completions \
]
}
EOF
)
)

Copy link
Copy Markdown
Collaborator

@hsliuustc0106 hsliuustc0106 Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good: Proper HTTP status code extraction

Using curl -w "\n%{http_code}" and then parsing with tail -1 and sed '$d' is a clean way to separate the response body from the status code.

The logic correctly checks for 2xx status codes (200-299) as success.

output=$(curl -sS --retry 3 --retry-delay 3 --retry-connrefused \
-X POST http://localhost:8091/v1/chat/completions \
-H "Content-Type: application/json" \
-d "$request_body")

# Here it only shows the text content of the first choice. Audio content has many binaries, so it's not displayed here.
echo "Output of request: $(echo "$output" | jq '.choices[0].message.content')"
12 changes: 7 additions & 5 deletions examples/online_serving/qwen3_omni/run_curl_multimodal_generation.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,10 +138,7 @@ esac
echo "Running query type: $QUERY_TYPE"
echo ""


output=$(curl -sS -X POST http://localhost:8091/v1/chat/completions \
-H "Content-Type: application/json" \
Copy link
Copy Markdown
Collaborator

@hsliuustc0106 hsliuustc0106 Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good: Consistent implementation

The retry logic is identical between the Qwen2.5-Omni and Qwen3-Omni scripts, which is good for maintainability. Same suggestions apply here as for the Qwen2.5-Omni script.

-d @- <<EOF
request_body=$(cat <<EOF
{
"model": "Qwen/Qwen3-Omni-30B-A3B-Instruct",
"sampling_params_list": $sampling_params_list,
Expand All @@ -164,7 +161,12 @@ output=$(curl -sS -X POST http://localhost:8091/v1/chat/completions \
]
}
EOF
)
)

output=$(curl -sS --retry 3 --retry-delay 3 --retry-connrefused \
-X POST http://localhost:8091/v1/chat/completions \
-H "Content-Type: application/json" \
-d "$request_body")

# Here it only shows the text content of the first choice. Audio content has many binaries, so it's not displayed here.
echo "Output of request: $(echo "$output" | jq '.choices[0].message.content')"