docs: add KServe and HAMi DRA inference lab - #755
Conversation
Signed-off-by: lixd <xueduan.li@gmail.com>
Signed-off-by: lixd <xueduan.li@gmail.com>
hami-robot
Bot
added
kind/documentation
✅ Deploy Preview for project-hami ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
Warning Review limit reached
Next review available in: 29 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthrough新增 Lab 11，介绍使用 HAMi-DRA 和 KServe Standard 部署双副本 vLLM 服务，并通过 ResourceClaim 共享一张 NVIDIA Tesla T4。新增 Kubernetes 示例资源及中英文文档导航配置。 ChangesKServe HAMi-DRA 实验
Estimated code review effort: 2 (Simple) | ~10 minutes Sequence Diagram(s)sequenceDiagram
participant Learner
participant Kubernetes
participant HAMiDRA
participant KServe
participant vLLM
Learner->>Kubernetes: 安装 HAMi-DRA、Envoy Gateway 和 KServe
Learner->>Kubernetes: 部署 Gateway、ResourceClaimTemplate 和 InferenceService
KServe->>HAMiDRA: 创建两个独立 ResourceClaim
HAMiDRA->>vLLM: 为两个 Predictor 副本分配共享 GPU 资源
Learner->>vLLM: 发送 OpenAI 兼容推理请求
vLLM-->>Learner: 返回推理结果
Possibly related issues
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (2)
tutorials/labs/kserve-hami-dra.md (1)
50-59: 🩺 Stability & Availability | 🔵 TrivialAdd storage and registry preflight checks.
The prerequisites check GPU and Hugging Face access but not node ephemeral storage or access to the OCI and image registries used later. Based on the supplied PR objectives, a later rerun hit ephemeral-storage pressure and controller image-pull failures. Add a free-space check and list the required registry access before the long deployment wait.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tutorials/labs/kserve-hami-dra.md` around lines 50 - 59, Expand the Prerequisites section to include a node ephemeral-storage free-space preflight check and explicitly list access to the OCI and container image registries used by the lab. Place these requirements before the deployment steps or long waits, preserving the existing GPU, Hugging Face, and runtime prerequisites.i18n/zh/docusaurus-plugin-content-docs-tutorials/current/labs/kserve-hami-dra.md (1)
50-59: 🩺 Stability & Availability | 🔵 Trivial补充存储和镜像仓库的前置检查。
当前前提条件检查了 GPU 和 Hugging Face 访问，但没有检查节点临时存储空间，也没有说明后续 OCI 和镜像仓库的访问要求。根据 PR 目标中的验证记录，后续重跑曾因临时存储压力和 Controller 镜像拉取失败而中断。请在长时间部署前增加磁盘空间检查，并列出所需的镜像仓库访问要求。
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@i18n/zh/docusaurus-plugin-content-docs-tutorials/current/labs/kserve-hami-dra.md` around lines 50 - 59, 在“前提条件”部分补充节点临时存储空间检查，确保长时间部署前确认有足够磁盘容量；同时明确集群需要访问后续使用的 OCI 和 Controller 镜像仓库，并保留现有 GPU、模型下载及运行时要求。
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tutorials/labs/examples/11-kserve-hami-dra/01-gateway.yaml`:
- Around line 19-23: Update the Kubernetes envoyService configuration under
provider to set externalTrafficPolicy to Cluster for the NodePort service,
preserving the existing service type and other settings.
In `@tutorials/labs/kserve-hami-dra.md`:
- Around line 90-93: Add an explicit non-empty GPU_NODE validation and failure
branch before kubectl label node in both tutorials/labs/kserve-hami-dra.md
(lines 90-93) and
i18n/zh/docusaurus-plugin-content-docs-tutorials/current/labs/kserve-hami-dra.md
(lines 90-93), so execution stops before labeling or Helm installation when no
Tesla-T4 GPU node is found.
- Around line 64-66: Quote the GPU custom-column argument in both tutorials:
update tutorials/labs/kserve-hami-dra.md lines 64-66 and
i18n/zh/docusaurus-plugin-content-docs-tutorials/current/labs/kserve-hami-dra.md
lines 64-66 so the complete -o custom-columns=... argument preserves the escaped
dot before kubectl parses it; leave the following node-label command unchanged.
- Around line 245-247: In both tutorials/labs/kserve-hami-dra.md lines 245-247
and
i18n/zh/docusaurus-plugin-content-docs-tutorials/current/labs/kserve-hami-dra.md
lines 245-247, extract the hostname from the KServe InferenceService .status.url
after the readiness check and reuse it as the curl Host header, replacing the
hard-coded hostname in both language versions.
---
Nitpick comments:
In
`@i18n/zh/docusaurus-plugin-content-docs-tutorials/current/labs/kserve-hami-dra.md`:
- Around line 50-59: 在“前提条件”部分补充节点临时存储空间检查，确保长时间部署前确认有足够磁盘容量；同时明确集群需要访问后续使用的 OCI
和 Controller 镜像仓库，并保留现有 GPU、模型下载及运行时要求。
In `@tutorials/labs/kserve-hami-dra.md`:
- Around line 50-59: Expand the Prerequisites section to include a node
ephemeral-storage free-space preflight check and explicitly list access to the
OCI and container image registries used by the lab. Place these requirements
before the deployment steps or long waits, preserving the existing GPU, Hugging
Face, and runtime prerequisites.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 37801aa6-8c67-4a80-bbff-b075b9989223
📒 Files selected for processing (8)
i18n/zh/docusaurus-plugin-content-docs-tutorials/current/labs/kserve-hami-dra.mdi18n/zh/docusaurus-plugin-content-docs-tutorials/current/overview.mdi18n/zh/docusaurus-plugin-content-docs-tutorials/current/tags.ymlsidebars-tutorials.jstutorials/labs/examples/11-kserve-hami-dra/01-gateway.yamltutorials/labs/examples/11-kserve-hami-dra/02-inference-service.yamltutorials/labs/kserve-hami-dra.mdtutorials/overview.md
Signed-off-by: lixd <xueduan.li@gmail.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: lixd, rootsongjc The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
2 participants
What type of PR is this?
/kind documentation
What this PR does / why we need it:
hf://Qwen/Qwen2.5-0.5B-Instructdirectly through KServe Storage Initializer, removing hostPath/PV model preparationWhich issue(s) this PR fixes:
Fixes #
Checklist:
npm run lintandnpm run format:checkpassnpm run buildsucceeds for bothenandzhgit commit -s)Validation notes:
hf://Qwen/Qwen2.5-0.5B-Instructin the test clusterTesla T4, 3072 MiBand an OpenAI-compatible request returned a responseSummary by CodeRabbit