Skip to content

Commit f8b95fd

Browse files
atchernychtzulingk
authored andcommitted
docs: update deploy readme (#2306)
1 parent add5fa8 commit f8b95fd

File tree

1 file changed

+0
-76
lines changed

1 file changed

+0
-76
lines changed

docs/guides/dynamo_deploy/README.md

Lines changed: 0 additions & 76 deletions
Original file line numberDiff line numberDiff line change
@@ -53,8 +53,6 @@ kubectl apply -f components/backends/vllm/deploy/agg.yaml -n ${NAMESPACE}
5353
You can use `kubectl get dynamoGraphDeployment -n ${NAMESPACE}` to view your deployment.
5454
You can use `kubectl delete dynamoGraphDeployment <your-dep-name> -n ${NAMESPACE}` to delete the deployment.
5555

56-
We provide a Custom Resource YAML file for many examples under the `deploy/` folder.
57-
Use [VLLM YAML](../../components/backends/vllm/deploy/agg.yaml) for an example.
5856

5957
**Note 1** Example Image
6058

@@ -99,77 +97,3 @@ Additional Resources:
9997
- [Port Forward Documentation](https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/)
10098
- [Examples Deployment Guide](../../examples/README.md#deploying-a-particular-example)
10199

102-
[View vLLM K8s](../../../components/backends/vllm/deploy/README.md)
103-
104-
[View TRT-LLM K8s](../../../components/backends/trtllm/deploy/README.md)
105-
106-
### Deploying a particular example
107-
108-
```bash
109-
# Set your dynamo root directory
110-
cd <root-dynamo-folder>
111-
export PROJECT_ROOT=$(pwd)
112-
export NAMESPACE=<your-namespace> # the namespace you used to deploy Dynamo cloud to.
113-
```
114-
115-
Deploying an example consists of the simple `kubectl apply -f ... -n ${NAMESPACE}` command. For example:
116-
117-
```bash
118-
kubectl apply -f components/backends/vllm/deploy/agg.yaml -n ${NAMESPACE}
119-
```
120-
121-
You can use `kubectl get dynamoGraphDeployment -n ${NAMESPACE}` to view your deployment.
122-
You can use `kubectl delete dynamoGraphDeployment <your-dep-name> -n ${NAMESPACE}` to delete the deployment.
123-
124-
We provide a Custom Resource YAML file for many examples under the `deploy/` folder.
125-
Use [VLLM YAML](../../components/backends/vllm/deploy/agg.yaml) for an example.
126-
127-
**Note 1** Example Image
128-
129-
The examples use a prebuilt image from the `nvcr.io` registry.
130-
You can utilize public images from [Dynamo NGC](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/collections/ai-dynamo) or build your own image and update the image location in your CR file prior to applying. Either way, you will need to overwrite the image in the example YAML.
131-
132-
To build your own image:
133-
134-
```bash
135-
./container/build.sh --framework <your-inference-framework>
136-
```
137-
138-
For example for the `sglang` run
139-
```bash
140-
./container/build.sh --framework sglang
141-
```
142-
143-
To overwrite the image in the example:
144-
145-
```bash
146-
extraPodSpec:
147-
mainContainer:
148-
image: <image-in-your-$DYNAMO_IMAGE>
149-
```
150-
151-
**Note 2**
152-
Setup port forward if needed when deploying to Kubernetes.
153-
154-
List the services in your namespace:
155-
156-
```bash
157-
kubectl get svc -n ${NAMESPACE}
158-
```
159-
Look for one that ends in `-frontend` and use it for port forward.
160-
161-
```bash
162-
SERVICE_NAME=$(kubectl get svc -n ${NAMESPACE} -o name | grep frontend | sed 's|.*/||' | sed 's|-frontend||' | head -n1)
163-
kubectl port-forward svc/${SERVICE_NAME}-frontend 8080:8080 -n ${NAMESPACE}
164-
```
165-
166-
Additional Resources:
167-
- [Port Forward Documentation](https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/)
168-
- [Examples Deployment Guide](../../examples/README.md#deploying-a-particular-example)
169-
170-
- Used for manually deploying inference graphs to Kubernetes
171-
- Contains Helm charts and configurations for deploying individual inference pipelines
172-
- Provides full control over deployment parameters
173-
- Requires manual management of infrastructure components
174-
- Documentation:
175-
- [Helm Deployment Guide](../../../deploy/helm/README.md): detailed instructions for manual deployment

0 commit comments

Comments
 (0)