[Feat] Implement VSR CLI tool for better user experience by srini-abhiram · Pull Request #824 · vllm-project/semantic-router

srini-abhiram · 2025-12-14T09:32:40Z

Close #234

Make sure the code changes pass the pre-commit checks.
Sign-off your commit by using -s when doing git commit
Try to classify PRs for easy understanding of the type of changes, such as [Bugfix], [Feat], and [CI].

Detailed Checklist (Click to Expand)

Thank you for your contribution to semantic-router! Before submitting the pull request, please ensure the PR meets the following criteria. This helps us maintain the code quality and improve the efficiency of the review process.

PR Title and Classification

Please try to classify PRs for easy understanding of the type of changes. The PR title is prefixed appropriately to indicate the type of change. Please use one of the following:

[Bugfix] for bug fixes.
[CI/Build] for build or continuous integration improvements.
[Doc] for documentation fixes and improvements.
[Feat] for new features in the cluster (e.g., autoscaling, disaggregated prefill, etc.).
[Router] for changes to the vllm_router (e.g., routing algorithm, router observability, etc.).
[Misc] for PRs that do not fit the above categories. Please use this sparingly.

Note: If the PR spans more than one category, please include all relevant prefixes.

Code Quality

The PR need to meet the following code quality standards:

Pass all linter checks. Please use pre-commit to format your code. See README.md for installation.
The code need to be well-documented to ensure future contributors can easily understand the code.
Please include sufficient tests to ensure the change is stay correct and robust. This includes both unit tests and integration tests.

DCO and Signed-off-by

When contributing changes to this project, you must agree to the DCO. Commits must include a Signed-off-by: header which certifies agreement with the terms of the DCO.

Using -s with git commit will automatically add this header.

What to Expect for the Reviews

netlify · 2025-12-14T09:33:09Z

✅ Deploy Preview for vllm-semantic-router ready!

Name	Link
🔨 Latest commit	`4719518`
🔍 Latest deploy log	https://app.netlify.com/projects/vllm-semantic-router/deploys/69441a5f1afe540008e4141d
😎 Deploy Preview	https://deploy-preview-824--vllm-semantic-router.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

github-actions · 2025-12-14T09:36:05Z

👥 vLLM Semantic Team Notification

The following members have been identified for the changed files in this PR and have been automatically assigned:

📁 `Root Directory`

Owners: @rootfs, @Xunzhuo
Files changed:

README.md

📁 `src`

Owners: @rootfs, @Xunzhuo, @wangchen615
Files changed:

src/semantic-router/cmd/vsr/QUICKSTART.md
src/semantic-router/cmd/vsr/README.md
src/semantic-router/cmd/vsr/commands/completion.go
src/semantic-router/cmd/vsr/commands/completion_test.go
src/semantic-router/cmd/vsr/commands/config.go
src/semantic-router/cmd/vsr/commands/config_test.go
src/semantic-router/cmd/vsr/commands/dashboard.go
src/semantic-router/cmd/vsr/commands/dashboard_test.go
src/semantic-router/cmd/vsr/commands/debug.go
src/semantic-router/cmd/vsr/commands/debug_test.go
src/semantic-router/cmd/vsr/commands/deploy.go
src/semantic-router/cmd/vsr/commands/deploy_test.go
src/semantic-router/cmd/vsr/commands/get.go
src/semantic-router/cmd/vsr/commands/get_test.go
src/semantic-router/cmd/vsr/commands/http_timeout_test.go
src/semantic-router/cmd/vsr/commands/install.go
src/semantic-router/cmd/vsr/commands/install_test.go
src/semantic-router/cmd/vsr/commands/model.go
src/semantic-router/cmd/vsr/commands/model_test.go
src/semantic-router/cmd/vsr/commands/signal_handling_test.go
src/semantic-router/cmd/vsr/commands/status.go
src/semantic-router/cmd/vsr/commands/status_test.go
src/semantic-router/cmd/vsr/commands/test.go
src/semantic-router/cmd/vsr/commands/test_test.go
src/semantic-router/cmd/vsr/commands/upgrade.go
src/semantic-router/cmd/vsr/commands/upgrade_test.go
src/semantic-router/cmd/vsr/config/config.yaml
src/semantic-router/cmd/vsr/main.go
src/semantic-router/go.mod
src/semantic-router/go.sum
src/semantic-router/pkg/cli/debug/checker.go
src/semantic-router/pkg/cli/debug/checker_test.go
src/semantic-router/pkg/cli/deployment/deployment.go
src/semantic-router/pkg/cli/deployment/deployment_test.go
src/semantic-router/pkg/cli/deployment/helm.go
src/semantic-router/pkg/cli/deployment/paths_test.go
src/semantic-router/pkg/cli/deployment/pid_management_test.go
src/semantic-router/pkg/cli/deployment/upgrade.go
src/semantic-router/pkg/cli/model/manager.go
src/semantic-router/pkg/cli/model/manager_test.go
src/semantic-router/pkg/cli/output.go
src/semantic-router/pkg/cli/validator.go
src/semantic-router/pkg/cli/validator_test.go
src/semantic-router/pkg/config/config.go
src/semantic-router/pkg/k8s/reconciler.go

📁 `tools`

Owners: @yuluo-yx, @rootfs, @Xunzhuo
Files changed:

tools/make/build-run-test.mk

📁 `website`

Owners: @Xunzhuo, @rootfs, @yuluo-yx
Files changed:

website/docs/cli/commands-reference.md
website/docs/cli/overview.md
website/docs/cli/troubleshooting.md

🎉 Thanks for your contributions!

This comment was automatically generated based on the OWNER files in the repository.

rootfs · 2025-12-15T14:32:37Z

@srini-abhiram Thanks for the work! Can you record a demo so we can use it for the code review? You can use https://asciinema.org/ or youtube.

srini-abhiram · 2025-12-16T06:48:45Z

Test-and-build failing due to cuda errors I believe.

@srini-abhiram Thanks for the work! Can you record a demo so we can use it for the code review? You can use https://asciinema.org/ or youtube.

Yes, I will create a walkthrough. Here's the link to it: https://asciinema.org/a/0Ve0r8qqsJssUuLmUuESL0PsK .
Few things to mention: in the walkthrough the test-prompt uses port 8080 (not opened in compose), the main cmds all follow the happypath. PTAL, thanks! @rootfs

rootfs · 2025-12-16T15:05:10Z

@srini-abhiram this looks so cool! @Xunzhuo @yuluo-yx @samzong @JaredforReal this could be a nice CLI option in addition to dashboard, especially for automation. WDYT?

JaredforReal · 2025-12-16T15:12:07Z

@srini-abhiram @rootfs a solid and great work! Thanks!

The CI error should disappear after "Update branch".

JaredforReal · 2025-12-17T09:50:05Z

@srini-abhiram current CI errors come from new test file in this PR, can u fix it~?

srini-abhiram · 2025-12-17T09:55:32Z

@JaredforReal for sure, I will fix those CI errors. And also update the docs in upcoming PR.

srini-abhiram · 2025-12-18T08:03:50Z

@JaredforReal the cli tests have a dependency on ibcandle_semantic_router.so , should we split the tests of cli and exclude it from test-semantic-router section?
CGO_ENABLED should be 0, because the cli tests dont need it.

JaredforReal · 2025-12-18T08:16:21Z

@srini-abhiram yes, maybe we can do something like:

# cli tests
CGO_ENABLED=0 go test ./cmd/vsr/...

# semantic-router core test
CGO_ENABLED=1 go test ./pkg/extproc/...

this may get things more clear~

This commit fixes CI test-and-build failures for PR vllm-project#824 by addressing two root causes: 1. **CGO Dependency Chain**: CLI tests have transitive CGO dependencies through pkg/cli/model → pkg/classification → candle-binding, requiring the Rust shared library (libcandle_semantic_router.so) at compile time. 2. **Test Execution Strategy**: Split test targets to avoid duplication and ensure proper build dependencies are met. Changes: - tools/make/build-run-test.mk: * Added test-cli target that depends on build-router and sets LD_LIBRARY_PATH * Modified test-semantic-router to exclude CLI tests using grep -v '/cmd/vsr' * Updated main test target to run both test-cli and test-semantic-router * Enhanced build-cli to inject version, commit hash, and build date - Fixed 6 CLI test bugs found during validation: * model.go: Fixed Short description text mismatch * model_test.go: Added missing output flag initialization * completion.go: Added OnlyValidArgs validator for shell argument * config_test.go: Use Name() instead of Use for subcommand matching * debug.go: Fixed Short description text * signal_handling_test.go: Added cmd.Wait() after Process.Kill() All CLI tests now pass successfully with proper CGO setup. Local CI validation confirms critical tests pass. Resolves vllm-project#234 Signed-off-by: Srinivas A <56465971+srini-abhiram@users.noreply.github.com>

This PR introduces the VSR (vLLM Semantic Router) CLI tool - a comprehensive command-line interface that reduces setup time from hours to minutes and provides a unified interface for deployment, monitoring, and troubleshooting across multiple environments. ## Key Features ### Core Commands - **deploy**: Multi-environment deployment (Local, Docker Compose, Kubernetes, Helm) - **config**: Configuration management with validation and templates - **model**: Model lifecycle management (download, list, validate, remove, inspect) - **status**: Health monitoring and status checks - **debug**: Interactive debugging and diagnostic tools - **dashboard**: Dashboard management and access - **test**: Prompt testing and validation - **upgrade**: Seamless version upgrades - **get**: Resource inspection (logs, pods, services) ### Implementation Details - Built with Go and Cobra framework for robust CLI experience - Comprehensive test coverage with 3,400+ lines of test code - Support for multiple deployment targets with environment detection - Process lifecycle management with orphan process prevention - Integrated health checking and diagnostics - Shell completion support (bash, zsh, fish, powershell) ### Documentation - Complete CLI documentation with quickstart guide - Command reference with examples - Troubleshooting guide - Integration with existing semantic-router documentation ### Files Changed - 49 files changed, 11,531 insertions(+), 16 deletions(-) - New CLI implementation in src/semantic-router/cmd/vsr/ - New CLI packages in src/semantic-router/pkg/cli/ - Documentation in website/docs/cli/ - Build system integration in tools/make/build-run-test.mk Resolves vllm-project#234 Signed-off-by: Srinivas A <56465971+srini-abhiram@users.noreply.github.com>

This commit fixes CI test-and-build failures for PR vllm-project#824 by addressing two root causes: 1. **CGO Dependency Chain**: CLI tests have transitive CGO dependencies through pkg/cli/model → pkg/classification → candle-binding, requiring the Rust shared library (libcandle_semantic_router.so) at compile time. 2. **Test Execution Strategy**: Split test targets to avoid duplication and ensure proper build dependencies are met. Changes: - tools/make/build-run-test.mk: * Added test-cli target that depends on build-router and sets LD_LIBRARY_PATH * Modified test-semantic-router to exclude CLI tests using grep -v '/cmd/vsr' * Updated main test target to run both test-cli and test-semantic-router * Enhanced build-cli to inject version, commit hash, and build date - Fixed 6 CLI test bugs found during validation: * model.go: Fixed Short description text mismatch * model_test.go: Added missing output flag initialization * completion.go: Added OnlyValidArgs validator for shell argument * config_test.go: Use Name() instead of Use for subcommand matching * debug.go: Fixed Short description text * signal_handling_test.go: Added cmd.Wait() after Process.Kill() All CLI tests now pass successfully with proper CGO setup. Local CI validation confirms critical tests pass. Resolves vllm-project#234 Signed-off-by: Srinivas A <56465971+srini-abhiram@users.noreply.github.com>

srini-abhiram · 2025-12-18T16:22:17Z

@JaredforReal ,The current CI errors are unrelated to the PR, can we re-run them?

JaredforReal · 2025-12-18T16:58:53Z

@srini-abhiram trying!

JaredforReal · 2025-12-18T17:36:23Z

@srini-abhiram all green！ ready to merge？

srini-abhiram · 2025-12-19T02:41:44Z

@srini-abhiram all green！ ready to merge？

@JaredforReal Thanks for re running, please proceed i'm done with the changes.

samzong · 2025-12-19T06:21:33Z

@srini-abhiram This is great. This is a big progress. Can you contact me on slack?

or join slack: https://join.slack.com/t/vllm-dev/shared_invite/zt-3lrh7gw29-MPJdxpIdGnWsFji5qu24ow

…t#824) Signed-off-by: Wilson Wu <iwilsonwu@gmail.com>

srini-abhiram marked this pull request as ready for review December 14, 2025 09:35

srini-abhiram requested review from Xunzhuo, rootfs and wangchen615 as code owners December 14, 2025 09:35

github-actions Bot assigned rootfs, wangchen615, Xunzhuo and yuluo-yx Dec 14, 2025

srini-abhiram force-pushed the issue-234 branch 3 times, most recently from 263f97f to c0b0f07 Compare December 14, 2025 15:03

rootfs previously approved these changes Dec 16, 2025

View reviewed changes

JaredforReal previously approved these changes Dec 16, 2025

View reviewed changes

srini-abhiram dismissed stale reviews from JaredforReal and rootfs via b2196dc December 18, 2025 13:03

srini-abhiram added 2 commits December 18, 2025 15:09

srini-abhiram force-pushed the issue-234 branch from cd6f33d to 4719518 Compare December 18, 2025 15:14

JaredforReal approved these changes Dec 18, 2025

View reviewed changes

JaredforReal merged commit a500d46 into vllm-project:main Dec 19, 2025
42 of 44 checks passed

wilsonwu pushed a commit to wilsonwu/semantic-router that referenced this pull request Dec 23, 2025

[Feat] Implement VSR CLI tool for better user experience (vllm-projec…

5248f17

…t#824) Signed-off-by: Wilson Wu <iwilsonwu@gmail.com>

Conversation

srini-abhiram commented Dec 14, 2025

PR Title and Classification

Code Quality

DCO and Signed-off-by

What to Expect for the Reviews

Uh oh!

netlify Bot commented Dec 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for vllm-semantic-router ready!

Uh oh!

github-actions Bot commented Dec 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

👥 vLLM Semantic Team Notification

📁 Root Directory

📁 src

📁 tools

📁 website

🎉 Thanks for your contributions!

Uh oh!

rootfs commented Dec 15, 2025

Uh oh!

srini-abhiram commented Dec 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rootfs commented Dec 16, 2025

Uh oh!

JaredforReal commented Dec 16, 2025

Uh oh!

JaredforReal commented Dec 17, 2025

Uh oh!

srini-abhiram commented Dec 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

srini-abhiram commented Dec 18, 2025

Uh oh!

JaredforReal commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

srini-abhiram commented Dec 18, 2025

Uh oh!

JaredforReal commented Dec 18, 2025

Uh oh!

JaredforReal commented Dec 18, 2025

Uh oh!

srini-abhiram commented Dec 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

samzong commented Dec 19, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

netlify Bot commented Dec 14, 2025 •

edited

Loading

github-actions Bot commented Dec 14, 2025 •

edited

Loading

📁 `Root Directory`

📁 `src`

📁 `tools`

📁 `website`

srini-abhiram commented Dec 16, 2025 •

edited

Loading

srini-abhiram commented Dec 17, 2025 •

edited

Loading

JaredforReal commented Dec 18, 2025 •

edited

Loading

srini-abhiram commented Dec 19, 2025 •

edited

Loading