docs: Backport: Dyn 591 (#2247) to 0.4.0

components/backends/trtllm/deploy/README.md

@@ -0,0 +1 @@
+This folder contains deployment examples for the TRTLLM inference backend.

components/backends/vllm/deploy/README.md

@@ -0,0 +1 @@
+This folder contains examples for the VLLM inference backend.

docs/examples/README.md

@@ -40,9 +40,14 @@ kubectl apply -f components/backends/vllm/deploy/agg.yaml -n ${NAMESPACE}
 You can use `kubectl get dynamoGraphDeployment -n ${NAMESPACE}` to view your deployment.
 You can use `kubectl delete dynamoGraphDeployment <your-dep-name> -n ${NAMESPACE}` to delete the deployment.
 
-We provide a Custom Resource yaml file for many examples under the `deploy/` folder.
-Use [VLLM YAML](../../components/backends/vllm/deploy/agg.yaml) for an example.
+We provide a Custom Resource yaml file for many examples under the `components/backends/<backend-name>/deploy/`folder.
+Consult the examples below for the CRs for your specific inference backend.
 
+[View SGLang k8s](/components/backends/sglang/deploy/README.md)
+
+[View vLLM K8s](/components/backends/vllm/deploy/README.md)
+
+[View TRTLLM k8s](/components/backends/trtllm/deploy/README.md)
 
 **Note 1** Example Image
 

docs/guides/dynamo_deploy/README.md

@@ -17,16 +17,87 @@ limitations under the License.
 
 # Deploying Inference Graphs to Kubernetes
 
-We expect users to deploy their inference graphs using CRDs or helm charts.
+ We expect users to deploy their inference graphs using CRDs or helm charts.
+
+# 1. Install Dynamo Cloud.
+
+Prior to deploying an inference graph the user should deploy the Dynamo Cloud Platform. Reference the [Quickstart Guide](quickstart.md) for steps to install Dynamo Cloud with Helm.
 
-Prior to deploying an inference graph the user should deploy the Dynamo Cloud Platform.
 Dynamo Cloud acts as an orchestration layer between the end user and Kubernetes, handling the complexity of deploying your graphs for you. This is a one-time action, only necessary the first time you deploy a DynamoGraph.
 
+# 2. Deploy your inference graph.
+
+We provide a Custom Resource YAML file for many examples under the components/backends/{engine}/deploy folders. Consult the examples below for the CRs for a specific inference backend.
+
+[View SGLang K8s](../../components/backends/sglang/deploy/README.md)
+
+[View vLLM K8s](../../components/backends/vllm/deploy/README.md)
+
+[View TRT-LLM K8s](../../components/backends/trtllm/deploy/README.md)
+
+### Deploying a particular example
+
+```bash
+# Set your dynamo root directory
+cd <root-dynamo-folder>
+export PROJECT_ROOT=$(pwd)
+export NAMESPACE=<your-namespace> # the namespace you used to deploy Dynamo cloud to.
+```
+
+Deploying an example consists of the simple `kubectl apply -f ... -n ${NAMESPACE}` command. For example:
+
+```bash
+kubectl apply -f components/backends/vllm/deploy/agg.yaml -n ${NAMESPACE}
+```
+
+You can use `kubectl get dynamoGraphDeployment -n ${NAMESPACE}` to view your deployment.
+You can use `kubectl delete dynamoGraphDeployment <your-dep-name> -n ${NAMESPACE}` to delete the deployment.
+
+We provide a Custom Resource YAML file for many examples under the `deploy/` folder.
+Use [VLLM YAML](../../components/backends/vllm/deploy/agg.yaml) for an example.
+
+**Note 1** Example Image
+
+The examples use a prebuilt image from the `nvcr.io` registry.
+You can utilize public images from [Dynamo NGC](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/collections/ai-dynamo) or build your own image and update the image location in your CR file prior to applying. Either way, you will need to overwrite the image in the example YAML.
+
+To build your own image:
+
+```bash
+./container/build.sh --framework <your-inference-framework>
+```
+
+For example for the `sglang` run
+```bash
+./container/build.sh --framework sglang
+```
+
+To overwrite the image in the example:
+
+```bash
+extraPodSpec:
+        mainContainer:
+          image: <image-in-your-$DYNAMO_IMAGE>
+```
+
+**Note 2**
+Setup port forward if needed when deploying to Kubernetes.
+
+List the services in your namespace:
+
+```bash
+kubectl get svc -n ${NAMESPACE}
+```
+Look for one that ends in `-frontend` and use it for port forward.
 
-# 1. Please follow [Installing Dynamo Cloud](./dynamo_cloud.md) for steps to install.
-For details about the Dynamo Cloud Platform, see the [Dynamo Operator Guide](dynamo_operator.md)
+```bash
+SERVICE_NAME=$(kubectl get svc -n ${NAMESPACE} -o name | grep frontend | sed 's|.*/||' | sed 's|-frontend||' | head -n1)
+kubectl port-forward svc/${SERVICE_NAME}-frontend 8080:8080 -n ${NAMESPACE}
+```
 
-# 2. Follow [Examples](../../examples/README.md) to see how you can deploy your Inference Graphs.
+Additional Resources:
+- [Port Forward Documentation](https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/)
+- [Examples Deployment Guide](../../examples/README.md#deploying-a-particular-example)
 
 
 ## Manual Deployment with Helm Charts