feat: add OpenShift deployment infrastructure with GPU support

This commit adds comprehensive OpenShift deployment support with GPU-enabled
specialist model containers, providing a complete automation solution for
deploying the semantic router to OpenShift clusters.

## New Files (deploy/openshift/):

**Core Deployment:**
- deployment.yaml: Kubernetes deployment manifest with GPU support
  * 4-container pod: semantic-router, model-a, model-b, envoy-proxy
  * CDI annotations for GPU device injection (gpu=0, gpu=1)
  * GPU node selection and tolerations
  * PVC mounts for models and cache
  * Production log levels (INFO for containers, info for Envoy)

- deploy-to-openshift.sh: Main deployment automation script (826 lines)
  * Auto-detection of OpenShift server and existing login
  * Enhanced deployment method with llm-katan specialists
  * Alternative methods: kustomize, template
  * Configurable resources, storage, logging
  * Automatic namespace creation
  * Inline Dockerfile build for llm-katan image
  * Service and route creation
  * Optional port forwarding (disabled by default)
  * Displays OpenWebUI endpoint at completion

- cleanup-openshift.sh: Cleanup automation script (494 lines)
  * Auto-detection of cluster and namespace
  * Graceful cleanup with confirmation
  * Port forwarding cleanup
  * Comprehensive resource deletion

**Configuration:**
- config-openshift.yaml: Semantic router config for OpenShift
  * Math-specialist and coding-specialist endpoints
  * Category-to-specialist routing
  * PII and jailbreak detection configuration

- envoy-openshift.yaml: Envoy proxy configuration
  * HTTP listener on port 8801
  * External processing filter
  * Specialist model routing
  * /v1/models aggregation

**Container Image:**
- Dockerfile.llm-katan: GPU-enabled specialist container image
  * Python 3.10-slim base
  * PyTorch with CUDA 12.1 support
  * llm-katan, transformers, accelerate packages
  * HuggingFace caching configuration
  * Health check endpoint

**Alternative Deployment Methods:**
- kustomization.yaml: Kustomize deployment option
- template.yaml: OpenShift template with parameters

**Documentation & Validation:**
- README.md: Comprehensive deployment documentation
- validate-deployment.sh: 12-test validation script
  * Namespace, deployment, container readiness
  * GPU detection in both specialist containers
  * Model loading verification
  * PVC, service, route checks
  * GPU node scheduling confirmation

## Integration Files:

- Makefile: Add include for tools/make/openshift.mk
- tools/make/openshift.mk: Optional make targets for OpenShift operations
  * openshift-deploy, openshift-cleanup, openshift-status
  * openshift-logs, openshift-routes, openshift-test
  * Port forwarding helpers

## Key Features:

1. **GPU Support**: Full NVIDIA GPU support via CDI device injection
2. **Specialist Models**: Real llm-katan containers for math/coding tasks
3. **Zero-Touch Deployment**: Auto-detection of cluster, automatic builds
4. **Production Ready**: Production log levels, proper health checks
5. **Validation**: Comprehensive 12-test validation suite
6. **UX Enhancements**: OpenWebUI endpoint display, optional port forwarding
7. **Clean Separation**: Only touches deploy/openshift/ (plus minimal Makefile)

## Architecture:

```
Pod: semantic-router
├── semantic-router (main ExtProc service, port 50051)
├── model-a (llm-katan math specialist, port 8000, GPU 0)
├── model-b (llm-katan coding specialist, port 8001, GPU 1)
└── envoy-proxy (gateway, port 8801)
```

## Testing:

Validated on OpenShift with NVIDIA L4 GPUs:
- All 4 containers running
- GPUs detected in both specialist containers
- Models loaded on CUDA
- PVCs bound
- Services and routes accessible
- Streaming functionality working

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
Signed-off-by: Yossi Ovadia <[email protected]>

Author: yossiovadia

Makefile

@@ -16,6 +16,7 @@ _run:
 		-f tools/make/pre-commit.mk \
 		-f tools/make/docker.mk \
 		-f tools/make/kube.mk \
+		-f tools/make/openshift.mk \
 		$(MAKECMDGOALS)
 
 .PHONY: _run

deploy/openshift/Dockerfile.llm-katan

@@ -0,0 +1,45 @@
+# Optimized Dockerfile for llm-katan - OpenShift compatible
+FROM python:3.10-slim
+
+# Install minimal system dependencies
+RUN apt-get update && \
+    apt-get install -y --no-install-recommends \
+        git \
+        curl \
+        && rm -rf /var/lib/apt/lists/*
+
+# Set working directory
+WORKDIR /app
+
+# Install PyTorch with CUDA 12.1 support (compatible with CUDA 12.x drivers)
+RUN pip install --no-cache-dir \
+    torch torchvision --index-url https://download.pytorch.org/whl/cu121
+
+# Install llm-katan and its dependencies
+RUN pip install --no-cache-dir \
+    llm-katan \
+    transformers \
+    accelerate \
+    fastapi \
+    uvicorn \
+    click \
+    pydantic \
+    numpy
+
+# Set environment variables for caching
+ENV HF_HUB_CACHE=/tmp/hf_cache
+ENV TRANSFORMERS_CACHE=/tmp/transformers_cache
+ENV HF_HOME=/tmp/hf_cache
+
+# Create cache directories
+RUN mkdir -p /tmp/hf_cache /tmp/transformers_cache
+
+# Expose ports
+EXPOSE 8000 8001
+
+# Health check
+HEALTHCHECK --interval=30s --timeout=10s --start-period=60s --retries=3 \
+    CMD curl -f http://localhost:8000/health || exit 1
+
+# Default command - this will be overridden by deployment args
+CMD ["llm-katan", "--help"]

deploy/openshift/README.md

@@ -0,0 +1,180 @@
+# OpenShift Deployment for Semantic Router
+
+This directory contains OpenShift-specific deployment manifests for the vLLM Semantic Router.
+
+## Quick Deployment
+
+### Prerequisites
+
+- OpenShift cluster access
+- `oc` CLI tool configured and logged in
+- Cluster admin privileges (or permissions to create namespaces and routes)
+
+### One-Command Deployment
+
+```bash
+oc apply -k deploy/openshift/
+```
+
+### Step-by-Step Deployment
+
+1. **Create namespace:**
+
+   ```bash
+   oc apply -f deploy/openshift/namespace.yaml
+   ```
+
+2. **Deploy core resources:**
+
+   ```bash
+   oc apply -f deploy/openshift/pvc.yaml
+   oc apply -f deploy/openshift/deployment.yaml
+   oc apply -f deploy/openshift/service.yaml
+   ```
+
+3. **Create external routes:**
+
+   ```bash
+   oc apply -f deploy/openshift/routes.yaml
+   ```
+
+## Accessing Services
+
+After deployment, the services will be accessible via OpenShift Routes:
+
+### Get Route URLs
+
+```bash
+# Classification API (HTTP REST)
+oc get route semantic-router-api -n vllm-semantic-router-system -o jsonpath='{.spec.host}'
+
+# gRPC API
+oc get route semantic-router-grpc -n vllm-semantic-router-system -o jsonpath='{.spec.host}'
+
+# Metrics
+oc get route semantic-router-metrics -n vllm-semantic-router-system -o jsonpath='{.spec.host}'
+```
+
+### Example Usage
+
+```bash
+# Get the API route
+API_ROUTE=$(oc get route semantic-router-api -n vllm-semantic-router-system -o jsonpath='{.spec.host}')
+
+# Test health endpoint
+curl https://$API_ROUTE/health
+
+# Test classification
+curl -X POST https://$API_ROUTE/api/v1/classify/intent \
+  -H "Content-Type: application/json" \
+  -d '{"text": "What is machine learning?"}'
+```
+
+## Architecture Differences from Kubernetes
+
+### Security Context
+
+- Removed `runAsNonRoot: false` for OpenShift compatibility
+- Enhanced security context with `capabilities.drop: ALL` and `seccompProfile`
+- OpenShift automatically enforces non-root containers
+
+### Networking
+
+- Uses OpenShift Routes instead of port-forwarding for external access
+- TLS termination handled by OpenShift router
+- Automatic HTTPS certificates via OpenShift
+
+### Storage
+
+- Uses OpenShift's default storage class
+- PVC automatically bound to available storage
+
+## Monitoring
+
+### Check Deployment Status
+
+```bash
+# Check pods
+oc get pods -n vllm-semantic-router-system
+
+# Check services
+oc get services -n vllm-semantic-router-system
+
+# Check routes
+oc get routes -n vllm-semantic-router-system
+
+# Check logs
+oc logs -f deployment/semantic-router -n vllm-semantic-router-system
+```
+
+### Metrics
+
+Access Prometheus metrics via the metrics route:
+
+```bash
+METRICS_ROUTE=$(oc get route semantic-router-metrics -n vllm-semantic-router-system -o jsonpath='{.spec.host}')
+curl https://$METRICS_ROUTE/metrics
+```
+
+## Cleanup
+
+Remove all resources:
+
+```bash
+oc delete -k deploy/openshift/
+```
+
+Or remove individual components:
+
+```bash
+oc delete -f deploy/openshift/routes.yaml
+oc delete -f deploy/openshift/service.yaml
+oc delete -f deploy/openshift/deployment.yaml
+oc delete -f deploy/openshift/pvc.yaml
+oc delete -f deploy/openshift/namespace.yaml
+```
+
+## Troubleshooting
+
+### Common Issues
+
+**1. Pod fails to start due to security context:**
+
+```bash
+oc describe pod -l app=semantic-router -n vllm-semantic-router-system
+```
+
+**2. Storage issues:**
+
+```bash
+oc get pvc -n vllm-semantic-router-system
+oc describe pvc semantic-router-models -n vllm-semantic-router-system
+```
+
+**3. Route not accessible:**
+
+```bash
+oc get routes -n vllm-semantic-router-system
+oc describe route semantic-router-api -n vllm-semantic-router-system
+```
+
+### Resource Requirements
+
+The deployment requires:
+
+- **Memory**: 3Gi request, 6Gi limit
+- **CPU**: 1 core request, 2 cores limit
+- **Storage**: 10Gi for model storage
+
+Adjust resource limits in `deployment.yaml` if needed for your cluster capacity.
+
+## Files Overview
+
+- `namespace.yaml` - Namespace with OpenShift-specific annotations
+- `pvc.yaml` - Persistent volume claim for model storage
+- `deployment.yaml` - Main application deployment with OpenShift security contexts
+- `service.yaml` - Services for gRPC, HTTP API, and metrics
+- `routes.yaml` - OpenShift routes for external access
+- `config.yaml` - Application configuration
+- `tools_db.json` - Tools database for semantic routing
+- `kustomization.yaml` - Kustomize configuration for easy deployment