Repository navigation
feat(bfcl): multi-model nightly A/B matrix on per-model runners #1791
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 8 commits
78559c3
77a07f3
de62d32
e5a16fa
0cbc70e
1a95dd6
5c923ee
27dfd69
b861468
ed1a623
a5d6edc
ea0ce36
6df6622
5dc5b4e
bda57e2
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -22,30 +22,34 @@ on: | |
| - ".github/workflows/nightly-bfcl.yml" | ||
| workflow_dispatch: | ||
| inputs: | ||
| only: | ||
| description: "Run only this matrix leg (qwen3.6|gpt-oss|deepseek-v4|minimax-m2.7|kimi-k2.6); empty = all" | ||
| required: false | ||
| default: "" | ||
| model: | ||
| description: "HF model id (weights) served on both arms" | ||
| description: "Override HF model id (weights) for the selected leg; empty = matrix default. Override model and bfcl_model together — mismatched weights/handler pair if only one is overridden." | ||
| required: false | ||
| default: "Qwen/Qwen3.6-27B" | ||
| default: "" | ||
| bfcl_model: | ||
| description: "BFCL FC handler name (must end in -FC; auto-registered if new)" | ||
| description: "Override BFCL FC handler name (must end in -FC); empty = matrix default. Override model and bfcl_model together — mismatched weights/handler pair if only one is overridden." | ||
| required: false | ||
| default: "Qwen/Qwen3.6-27B-FC" | ||
| default: "" | ||
| categories: | ||
| description: "Comma-separated BFCL test categories" | ||
| required: false | ||
| default: "simple_python,simple_java,simple_javascript,multiple,parallel,parallel_multiple,irrelevance,live_simple,live_multiple,live_parallel,live_parallel_multiple,live_irrelevance,live_relevance,multi_turn_base,multi_turn_miss_func,multi_turn_miss_param,multi_turn_long_context" | ||
| vllm_tool_parser: | ||
| description: "pure-vLLM --tool-call-parser" | ||
| description: "Override pure-vLLM --tool-call-parser; empty = matrix default" | ||
| required: false | ||
| default: "qwen3_xml" | ||
| default: "" | ||
| smg_tool_parser: | ||
| description: "SMG --tool-call-parser" | ||
| description: "Override SMG --tool-call-parser; empty = matrix default (empty also = SMG auto-detect)" | ||
| required: false | ||
| default: "qwen_xml" | ||
| default: "" | ||
| reasoning_parser: | ||
| description: "reasoning parser for both arms (empty for non-thinking SKUs)" | ||
| description: "Override reasoning parser for BOTH arms; empty = matrix defaults" | ||
| required: false | ||
| default: "qwen3" | ||
| default: "" | ||
|
|
||
| concurrency: | ||
| group: nightly-bfcl-${{ github.event_name }}-${{ github.ref }} | ||
|
|
@@ -91,20 +95,100 @@ jobs: | |
| path: bindings/python/dist/*.whl | ||
| retention-days: 7 | ||
|
|
||
| # Compute the model matrix as JSON so the dispatch `only` leg-filter can depend | ||
| # on inputs — the `matrix` context is not available in a job-level `if`. | ||
| setup: | ||
| # No needs: build-wheel — this job is pure CPU JSON generation with no wheel | ||
| # dependency. bfcl-ab still gates on build-wheel.result == 'success'. | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| contents: read | ||
| outputs: | ||
| matrix: ${{ steps.build.outputs.matrix }} | ||
| steps: | ||
| - id: build | ||
| env: | ||
| ONLY: ${{ github.event.inputs.only }} | ||
| run: | | ||
| python3 - <<'PY' >> "$GITHUB_OUTPUT" | ||
| import json, os | ||
| legs = [ | ||
| # H100 legs (TP=2 per arm, both arms concurrent: GPUs 0,1 + 2,3). | ||
| {"name": "qwen3.6", "runner": "4-gpu-h100", "tp": 2, | ||
| "gpu_a": "0,1", "gpu_b": "2,3", | ||
| "model": "Qwen/Qwen3.6-27B", "bfcl_model": "Qwen/Qwen3.6-27B-FC", | ||
| "vllm_tool": "qwen3_xml", "vllm_reason": "qwen3", | ||
| "smg_tool": "qwen_xml", "smg_reason": "qwen3", | ||
| "vllm_extra": "", "gpu_mem": "0.85", "startup_timeout": "600"}, | ||
| # gpt-oss SMG arm passes NO tool parser (smg_tool empty) -> harmony auto-detect. | ||
| {"name": "gpt-oss", "runner": "4-gpu-h100", "tp": 2, | ||
| "gpu_a": "0,1", "gpu_b": "2,3", | ||
| "model": "openai/gpt-oss-120b", "bfcl_model": "openai/gpt-oss-120b-FC", | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
For new legs such as gpt-oss/DeepSeek/MiniMax/Kimi that Useful? React with 👍 / 👎. |
||
| "vllm_tool": "openai", "vllm_reason": "", | ||
| "smg_tool": "", "smg_reason": "", | ||
| "vllm_extra": "", "gpu_mem": "0.85", "startup_timeout": "600"}, | ||
| # Blackwell legs (8xB300, TP=4 per arm, both arms concurrent: 0-3 + 4-7). | ||
| {"name": "deepseek-v4", "runner": "blackwell", "tp": 4, | ||
| "gpu_a": "0,1,2,3", "gpu_b": "4,5,6,7", | ||
| "model": "deepseek-ai/DeepSeek-V4-Flash", "bfcl_model": "deepseek-ai/DeepSeek-V4-Flash-FC", | ||
|
Comment on lines
+145
to
+147
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
For these new Blackwell matrix legs, the selected checkpoints are TP=4 models, but the later Useful? React with 👍 / 👎. |
||
| "vllm_tool": "deepseek_v4", "vllm_reason": "deepseek_v4", | ||
| "smg_tool": "deepseek_v4", | ||
| "smg_reason": "deepseek_v31", # TODO(confirm): no deepseek_v4 SMG reasoning parser yet | ||
| # --kv-cache-dtype fp8 + --block-size 256 are REQUIRED: V4 auto-quantizes to | ||
| # deepseek_v4_fp8 and its FlashMLA kernel asserts fp8 kv-cache (else EngineCore dies). | ||
| "vllm_extra": "--tokenizer-mode deepseek_v4 --trust-remote-code --kv-cache-dtype fp8 --block-size 256", | ||
| "gpu_mem": "0.90", "startup_timeout": "2400"}, | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
# Verify evidence for remote-code execution + token presence in this workflow.
# Expected: Lines showing both `HF_TOKEN` and `--trust-remote-code` in the same job.
nl -ba .github/workflows/nightly-bfcl.yml | sed -n '130,160p;184,194p;218,230p'Repository: lightseekorg/smg Length of output: 101 🏁 Script executed: # Read the workflow file to examine the specific lines and context
cat -n .github/workflows/nightly-bfcl.yml | sed -n '130,165p'Repository: lightseekorg/smg Length of output: 2889 🏁 Script executed: # Check for HF_TOKEN environment setup
cat -n .github/workflows/nightly-bfcl.yml | sed -n '180,200p'Repository: lightseekorg/smg Length of output: 1427 🏁 Script executed: # Check if there are model revision pins anywhere (e.g., using `@revision` or similar)
grep -n "revision\|@\|deepseek\|minimax\|kimi" .github/workflows/nightly-bfcl.yml | head -20Repository: lightseekorg/smg Length of output: 1522 🏁 Script executed: # Find scripts that handle model loading/launching
fd -t f "launch" . | head -10Repository: lightseekorg/smg Length of output: 336 🏁 Script executed: # Check if any revision pins exist elsewhere (e.g., in Python setup or launch scripts)
grep -r "revision\|@[a-f0-9]\|--revision" . --include="*.sh" --include="*.py" | grep -i "deepseek\|minimax\|kimi" | head -20Repository: lightseekorg/smg Length of output: 42 🏁 Script executed: # Check how vLLM is invoked with the model
grep -n "python.*vllm\|vLLM\|serve\|MODEL" .github/workflows/nightly-bfcl.yml | grep -A2 -B2 "vllm"Repository: lightseekorg/smg Length of output: 483 🏁 Script executed: # Check launch_arm.sh for how models are specified/loaded
cat scripts/bfcl/launch_arm.sh | head -100Repository: lightseekorg/smg Length of output: 4598 🏁 Script executed: # Also check register_bfcl_model.py to see if it accepts or enforces revision specs
cat bindings/python/src/smg/launch_router.py | grep -A10 -B10 "model\|revision" | head -50Repository: lightseekorg/smg Length of output: 42 Pin immutable model revisions when enabling These legs execute remote model repo Python during startup while 🤖 Prompt for AI Agents |
||
| {"name": "minimax-m2.7", "runner": "blackwell", "tp": 4, | ||
| "gpu_a": "0,1,2,3", "gpu_b": "4,5,6,7", | ||
| "model": "MiniMaxAI/MiniMax-M2.7", "bfcl_model": "MiniMaxAI/MiniMax-M2.7-FC", | ||
| "vllm_tool": "minimax_m2", "vllm_reason": "minimax_m2", | ||
| "smg_tool": "minimax_m2", "smg_reason": "minimax", | ||
| "vllm_extra": "--trust-remote-code", "gpu_mem": "0.90", "startup_timeout": "2400"}, | ||
| # Kimi-K2.6 loads native int4 weights (~500GB) -> fits TP=4 half-node. | ||
| {"name": "kimi-k2.6", "runner": "blackwell", "tp": 4, | ||
| "gpu_a": "0,1,2,3", "gpu_b": "4,5,6,7", | ||
| "model": "moonshotai/Kimi-K2.6", "bfcl_model": "moonshotai/Kimi-K2.6-FC", | ||
| "vllm_tool": "kimi_k2", "vllm_reason": "kimi_k2", | ||
| "smg_tool": "kimik2", | ||
| "smg_reason": "kimi_k25", # TODO(confirm): no kimi_k2 SMG reasoning parser yet | ||
| # Generous startup: 1T MoE at TP=4 spends minutes on load + torch.compile + | ||
| # weight quant; the default 420s timed out mid-compile. | ||
| "vllm_extra": "--trust-remote-code", "gpu_mem": "0.90", "startup_timeout": "2400"}, | ||
| ] | ||
| only = os.environ.get("ONLY", "") | ||
| valid_names = [l["name"] for l in legs] | ||
| if only: | ||
| legs = [l for l in legs if l["name"] == only] | ||
| # All legs (H100 + Blackwell) run on PRs as a sanity check; the run stays | ||
| # cheap because the PR CATEGORIES below are a tiny non-live subset | ||
| # (simple_python,irrelevance) for every leg regardless of runner. | ||
| if only and not legs: | ||
| raise SystemExit(f"unknown leg {only!r}; valid: {valid_names}") | ||
|
coderabbitai[bot] marked this conversation as resolved.
Resolved
|
||
| print("matrix=" + json.dumps({"include": legs})) | ||
| PY | ||
|
|
||
| bfcl-ab: | ||
| needs: build-wheel | ||
| if: ${{ !cancelled() && needs.build-wheel.result == 'success' }} | ||
| runs-on: 4-gpu-h100 # TP=2 per arm, both arms concurrent (GPUs 0,1 + 2,3) | ||
| # Generous: the nightly set adds multi_turn (multi-turn state simulation), | ||
| # which is much slower than the single-turn AST/live categories. | ||
| # Concise leg label (default would list every matrix field): model + runner + TP. | ||
| name: bfcl-ab ${{ matrix.model }} (${{ matrix.runner }} TP${{ matrix.tp }}) | ||
| needs: [build-wheel, setup] | ||
| if: ${{ !cancelled() && needs.build-wheel.result == 'success' && needs.setup.result == 'success' }} | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: ${{ fromJSON(needs.setup.outputs.matrix) }} | ||
| runs-on: ${{ matrix.runner }} | ||
| # Generous: the nightly set adds multi_turn (state simulation), much slower | ||
| # than the single-turn AST/live categories. | ||
| timeout-minutes: 240 | ||
| permissions: | ||
| contents: read | ||
| env: | ||
| ROUTER_LOCAL_MODEL_PATH: /models | ||
| HF_HOME: /models | ||
| MODEL: ${{ github.event.inputs.model || 'Qwen/Qwen3.6-27B' }} | ||
| BFCL_MODEL_FC: ${{ github.event.inputs.bfcl_model || 'Qwen/Qwen3.6-27B-FC' }} | ||
| MODEL: ${{ github.event.inputs.model || matrix.model }} | ||
| BFCL_MODEL_FC: ${{ github.event.inputs.bfcl_model || matrix.bfcl_model }} | ||
|
Comment on lines
+204
to
+205
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
For a Useful? React with 👍 / 👎. |
||
| # Auth for gated/private HF repos + higher download rate limits (used by the | ||
| # weight-staging step and by vLLM/SMG at serve time). | ||
| HF_TOKEN: ${{ secrets.HF_TOKEN }} | ||
| # PR = quick non-live sanity subset; schedule/dispatch = non_live + live + multi_turn. | ||
| CATEGORIES: ${{ github.event.inputs.categories || (github.event_name == 'pull_request' && 'simple_python,irrelevance' || 'simple_python,simple_java,simple_javascript,multiple,parallel,parallel_multiple,irrelevance,live_simple,live_multiple,live_parallel,live_parallel_multiple,live_irrelevance,live_relevance,multi_turn_base,multi_turn_miss_func,multi_turn_miss_param,multi_turn_long_context') }} | ||
| steps: | ||
|
|
@@ -123,27 +207,32 @@ jobs: | |
| - name: Register model in bfcl (no-op if bfcl already ships a handler) | ||
| run: python scripts/bfcl/register_bfcl_model.py --model-id "$MODEL" || true | ||
| - name: Stage model weights | ||
| run: bash scripts/ci_download_model.sh --gpu-tier 2 | ||
| # Only this leg's model — not the whole GPU tier (which pulls unrelated, | ||
| # sometimes-gated repos this A/B never serves). | ||
| run: bash scripts/ci_download_model.sh "$MODEL" | ||
| - name: Launch arms + run official BFCL A/B | ||
| env: | ||
| BFCL_MODEL: ${{ env.MODEL }} | ||
| VLLM_BIN: vllm | ||
| VLLM_PYTHON: python | ||
| SMG_LAUNCH: "python -m smg.launch_router" | ||
| BFCL_VLLM_TOOL_PARSER: ${{ github.event.inputs.vllm_tool_parser || 'qwen3_xml' }} | ||
| BFCL_SMG_TOOL_PARSER: ${{ github.event.inputs.smg_tool_parser || 'qwen_xml' }} | ||
| BFCL_VLLM_REASONING_PARSER: ${{ github.event.inputs.reasoning_parser || 'qwen3' }} | ||
| BFCL_SMG_REASONING_PARSER: ${{ github.event.inputs.reasoning_parser || 'qwen3' }} | ||
| BFCL_TP: "2" | ||
| BFCL_VLLM_TOOL_PARSER: ${{ github.event.inputs.vllm_tool_parser || matrix.vllm_tool }} | ||
| BFCL_SMG_TOOL_PARSER: ${{ github.event.inputs.smg_tool_parser || matrix.smg_tool }} | ||
|
coderabbitai[bot] marked this conversation as resolved.
Resolved
|
||
| BFCL_VLLM_REASONING_PARSER: ${{ github.event.inputs.reasoning_parser || matrix.vllm_reason }} | ||
| BFCL_SMG_REASONING_PARSER: ${{ github.event.inputs.reasoning_parser || matrix.smg_reason }} | ||
| BFCL_VLLM_EXTRA: ${{ matrix.vllm_extra }} | ||
| BFCL_TP: ${{ matrix.tp }} | ||
| BFCL_MAX_MODEL_LEN: "16384" | ||
| BFCL_GPU_MEM_UTIL: "0.85" | ||
| BFCL_GPU_MEM_UTIL: ${{ matrix.gpu_mem }} | ||
| # Per-leg health-wait budget: big MoEs need minutes for load + compile + quant. | ||
| BFCL_STARTUP_TIMEOUT: ${{ matrix.startup_timeout }} | ||
| run: | | ||
| set -euo pipefail | ||
| # TP=2 per arm: Arm A (pure vLLM) on GPUs 0,1; Arm B (SMG -> vLLM gRPC) on GPUs 2,3. | ||
| # Ports are left unset so launch_arm.sh picks free ones — lets two bfcl | ||
| # runners share a node (bin-packed, hostNetwork) without port collisions. | ||
| A_URL=$(BFCL_GPU=0,1 bash scripts/bfcl/launch_arm.sh a) | ||
| B_URL=$(BFCL_GPU=2,3 bash scripts/bfcl/launch_arm.sh b) | ||
| # Both arms concurrent: arm A (pure vLLM) on the first GPU half, arm B | ||
| # (SMG -> vLLM gRPC) on the second. Ports left unset so launch_arm.sh | ||
| # picks free ones (lets bin-packed runners share a node). | ||
| A_URL=$(BFCL_GPU=${{ matrix.gpu_a }} bash scripts/bfcl/launch_arm.sh a) | ||
| B_URL=$(BFCL_GPU=${{ matrix.gpu_b }} bash scripts/bfcl/launch_arm.sh b) | ||
| echo "arm A: $A_URL arm B: $B_URL" | ||
| python scripts/bfcl/run_ab.py \ | ||
| --baseline "vllm=$A_URL" \ | ||
|
|
@@ -157,12 +246,16 @@ jobs: | |
| --tolerance 0.02 || echo "BFCL_AB_REGRESSION=1" >> "$GITHUB_ENV" | ||
| - name: Render summary | ||
| if: always() | ||
| run: cat "$RUNNER_TEMP/bfcl_ab.md" >> "$GITHUB_STEP_SUMMARY" || echo "no report produced" >> "$GITHUB_STEP_SUMMARY" | ||
| run: | | ||
| { | ||
| echo "## BFCL A/B — ${{ matrix.name }} (${{ matrix.model }})" | ||
| cat "$RUNNER_TEMP/bfcl_ab.md" | ||
| } >> "$GITHUB_STEP_SUMMARY" || echo "no report produced for ${{ matrix.name }}" >> "$GITHUB_STEP_SUMMARY" | ||
| - name: Upload comparison | ||
| if: always() | ||
| uses: actions/upload-artifact@v7 | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
# Read-only check for mutable action refs in this workflow.
nl -ba .github/workflows/nightly-bfcl.yml | rg -n 'uses:\s*[^#]+@v[0-9]+'Repository: lightseekorg/smg Length of output: 101 🏁 Script executed: # First, check if the file exists and read around line 247
if [ -f ".github/workflows/nightly-bfcl.yml" ]; then
echo "=== File exists, reading around line 247 ==="
sed -n '240,255p' .github/workflows/nightly-bfcl.yml | cat -n
else
echo "File not found"
git ls-files | grep -i nightly
fiRepository: lightseekorg/smg Length of output: 830 🏁 Script executed: # Count lines and find all 'uses' directives with version tags
echo "=== Total lines and all 'uses' with version tags ==="
wc -l .github/workflows/nightly-bfcl.yml
echo ""
echo "=== All 'uses' directives with `@v` or `@tags` ==="
grep -n "uses:" .github/workflows/nightly-bfcl.yml | head -20Repository: lightseekorg/smg Length of output: 538 🌐 Web query:
💡 Result: In GitHub Actions, pinning to a full-length commit SHA is considered the security gold standard, whereas pinning to a version tag is a matter of convenience that carries inherent risks [1][2]. The fundamental difference lies in immutability: 1. Commit SHA (Immutable): A full-length commit SHA is a unique, fixed identifier for a specific state of a repository [3][4]. Once pinned to a SHA, your workflow is guaranteed to execute the exact same code every time, making it immune to "tag retargeting" or "force-push" attacks where a malicious actor alters the code behind a seemingly legitimate tag [1][2][5]. 2. Version Tag (Mutable): Tags (e.g., v4, v4.1.0) are references that can be moved or deleted by repository maintainers [1][5]. Even if you trust an action's author, a compromised account or repository could allow an attacker to update a tag to point to malicious code, which your workflows would then automatically adopt [1][2][4]. Best Practices: - Always use the full-length commit SHA (e.g., actions/checkout@3df4ab1) instead of an abbreviated one [3][4]. - Maintain readability and auditability by including the version tag as a comment in your workflow file (e.g., uses: actions/checkout@3df4ab1 # v4.0.0) [4][6]. - Automate the maintenance of these pins. Tools like Renovate (using presets such as helpers:pinGitHubActionDigests) or Dependabot can automatically open pull requests to update your SHAs when new releases are published, allowing you to review changes before merging [7][4][5]. - If you are an administrator, consider enabling organization or repository-level policies that enforce commit SHA pinning, which will cause any non-pinned actions to fail [1][4][8]. While pinning to a tag is permissible if you strictly trust the creator, it remains a secondary choice compared to the immutability and security provided by commit SHAs [1][2]. Citations:
🌐 Web query:
💡 Result: Pinning actions/upload-artifact to a commit SHA is a recommended security best practice for GitHub Actions workflows [1][2][3]. As of June 2026, the latest version is v7.0.1 [4][5]. Security Context Using a tag like Citations:
Pin Line 247 uses a mutable version tag ( Note: This same pattern exists elsewhere in the workflow (e.g., lines 72, 75, 92, 190–191). Consider fixing all mutable action references. 🧰 Tools🪛 zizmor (1.25.2)[error] 247-247: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy) (unpinned-uses) 🤖 Prompt for AI AgentsSource: Linters/SAST tools |
||
| with: | ||
| name: bfcl-ab-${{ github.run_id }} | ||
| name: bfcl-ab-${{ matrix.name }}-${{ github.run_id }} | ||
| path: | | ||
| ${{ runner.temp }}/bfcl_ab.md | ||
| ${{ runner.temp }}/bfcl_ab.json | ||
|
|
||
| Original file line number | Diff line number | Diff line change | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -64,6 +64,12 @@ start() { | |||||||||
| echo "[launch_arm] started $name (pid $(cat "$RUN_DIR/$name.pid")) -> $log" >&2 | ||||||||||
| } | ||||||||||
|
|
||||||||||
| # Tail a logfile to stderr for real-time startup visibility in CI. Stdout is | ||||||||||
| # reserved for the base_url (captured by the caller's $(...)), so logs must go to | ||||||||||
| # stderr — which still streams live to the CI console. Prints the tail pid so the | ||||||||||
| # caller can stop it once the server is healthy. | ||||||||||
| stream_log() { tail -n +1 -F "$1" >&2 & echo $!; } | ||||||||||
|
|
||||||||||
| wait_http() { # wait_http <url> <timeout_s> | ||||||||||
| local url="$1" timeout="${2:-300}" waited=0 | ||||||||||
| until curl -sf -m 3 "$url" >/dev/null 2>&1; do | ||||||||||
|
|
@@ -100,7 +106,9 @@ case "$ARM" in | |||||||||
| # shellcheck disable=SC2206 # intentional word-split of optional extra flags | ||||||||||
| [ -n "$VLLM_EXTRA" ] && cmd+=($VLLM_EXTRA) | ||||||||||
| start arm_a "$RUN_DIR/arm_a_vllm.log" "${cmd[@]}" | ||||||||||
| log_tail=$(stream_log "$RUN_DIR/arm_a_vllm.log") | ||||||||||
| wait_http "http://127.0.0.1:$ARM_A_PORT/health" "${BFCL_STARTUP_TIMEOUT:-420}" | ||||||||||
| kill "$log_tail" 2>/dev/null || true | ||||||||||
|
Comment on lines
+118
to
+120
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧹 Nitpick | 🔵 Trivial | ⚡ Quick win Tail processes may be orphaned if health checks fail. If Consider adding a trap handler to ensure cleanup on early exit. 🧹 Proposed trap-based cleanupAdd near the top of the script (after variable declarations): +TAIL_PIDS=()
+cleanup_tails() {
+ for pid in "${TAIL_PIDS[@]}"; do
+ kill "$pid" 2>/dev/null || true
+ done
+}
+trap cleanup_tails EXITThen modify the pattern to track PIDs: - log_tail=$(stream_log "$RUN_DIR/arm_a_vllm.log")
+ log_tail=$(stream_log "$RUN_DIR/arm_a_vllm.log")
+ TAIL_PIDS+=("$log_tail")
wait_http "http://127.0.0.1:$ARM_A_PORT/health" "${BFCL_STARTUP_TIMEOUT:-420}"
- kill "$log_tail" 2>/dev/null || true
+ kill "$log_tail" 2>/dev/null || trueAlso applies to: 128-130, 144-146 🤖 Prompt for AI Agents |
||||||||||
| echo "http://127.0.0.1:$ARM_A_PORT" | ||||||||||
| ;; | ||||||||||
|
|
||||||||||
|
|
@@ -117,18 +125,25 @@ case "$ARM" in | |||||||||
| # shellcheck disable=SC2206 # intentional word-split of optional extra flags | ||||||||||
| [ -n "$VLLM_EXTRA" ] && wcmd+=($VLLM_EXTRA) | ||||||||||
| start arm_b_worker "$RUN_DIR/arm_b_worker.log" "${wcmd[@]}" | ||||||||||
| log_tail=$(stream_log "$RUN_DIR/arm_b_worker.log") | ||||||||||
| wait_grpc "$ARM_B_GRPC_PORT" "${BFCL_STARTUP_TIMEOUT:-420}" | ||||||||||
| kill "$log_tail" 2>/dev/null || true | ||||||||||
| # 2) SMG gateway in front, exposing the OpenAI API. | ||||||||||
| # shellcheck disable=SC2206 # intentional word-split of SMG_LAUNCH | ||||||||||
| declare -a smg_cmd=( | ||||||||||
| $SMG_LAUNCH | ||||||||||
| --model-path "$MODEL" | ||||||||||
| --worker-urls "grpc://127.0.0.1:$ARM_B_GRPC_PORT" | ||||||||||
| --tool-call-parser "$SMG_TOOL_PARSER" | ||||||||||
| --host 0.0.0.0 --port "$ARM_B_GW_PORT" | ||||||||||
| ) | ||||||||||
| # Empty tool-call-parser => let SMG auto-detect (e.g. gpt-oss routes through | ||||||||||
| # the harmony pipeline via HarmonyDetector; passing a parser there is wrong). | ||||||||||
| [ -n "$SMG_TOOL_PARSER" ] && smg_cmd+=(--tool-call-parser "$SMG_TOOL_PARSER") | ||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There is a bug that prevents On line 36, SMG_TOOL_PARSER="${BFCL_SMG_TOOL_PARSER:-qwen}"Because of To fix this, change line 36 to use Additionally, for better compatibility with
Suggested change
References
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When the gpt-oss matrix leg sets Useful? React with 👍 / 👎. |
||||||||||
| [ -n "$SMG_REASONING_PARSER" ] && smg_cmd+=(--reasoning-parser "$SMG_REASONING_PARSER") | ||||||||||
| start arm_b_gateway "$RUN_DIR/arm_b_gateway.log" "${smg_cmd[@]}" | ||||||||||
| log_tail=$(stream_log "$RUN_DIR/arm_b_gateway.log") | ||||||||||
| wait_http "http://127.0.0.1:$ARM_B_GW_PORT/health" "${BFCL_STARTUP_TIMEOUT:-420}" | ||||||||||
| kill "$log_tail" 2>/dev/null || true | ||||||||||
| echo "http://127.0.0.1:$ARM_B_GW_PORT" | ||||||||||
| ;; | ||||||||||
|
|
||||||||||
|
|
||||||||||
Uh oh!
There was an error while loading. Please reload this page.