Serving R with Seldon-Core.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Serve R with Seldon-Core and Kubernetes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Install s2i to ~/tmp**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "mkdir tmp\n",
    "cd tmp\n",
    "wget https://github.com/openshift/source-to-image/releases/download/v1.1.13/source-to-image-v1.1.13-b54d75d3-linux-amd64.tar.gz\n",
    "tar -xvf source-to-image-v1.1.13-b54d75d3-linux-amd64.tar.gz"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Create environment file for s2i**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!mkdir ./serving/.s2i"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile ./serving/.s2i/environment\n",
    "MODEL_NAME=babyweight.R\n",
    "API_TYPE=REST\n",
    "SERVICE_TYPE=MODEL\n",
    "PERSISTENCE=0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Create install.R file for dependencies**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile ./serving/install.R\n",
    "install.packages(\"glmnet\")\n",
    "install.packages(\"caret\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Copy Model File to Deploy for Cloud Storage**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "export STORAGE_BUCKET=r-demo-data\n",
    "export MODEL_VERSION=20190408_121335\n",
    "\n",
    "gsutil cp gs://$STORAGE_BUCKET/$MODEL_VERSION/model.rds ./serving\n",
    "gsutil cp gs://$STORAGE_BUCKET/$MODEL_VERSION/preproc.rds ./serving"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Create R Runtime Model File**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile ./serving/babyweight.R\n",
    "library(methods)\n",
    "library(glmnet)\n",
    "\n",
    "predict.babyweight <- function(babyweight,newdata=list()) {\n",
    "    # center and scale data\n",
    "    newdata[,1:5] <- sapply(newdata[,1:5], as.numeric)\n",
    "    newdata <- predict(babyweight$preproc, newdata)\n",
    "\n",
    "    # set factor data types\n",
    "    newdata[, c(\"is_male\", \"child_race\")] <- sapply(newdata[, c(\"is_male\", \"child_race\")] , as.factor)\n",
    "\n",
    "    # inverse of log transform\n",
    "    exp(predict(babyweight$model, newdata))\n",
    "}\n",
    "\n",
    "new_babyweight <- function(modelfile, preprocfile) {\n",
    "    model <- readRDS(modelfile)\n",
    "    preproc <- readRDS(preprocfile)\n",
    "    structure(list(model=model,preproc=preproc), class = \"babyweight\")\n",
    "}\n",
    "\n",
    "initialise_seldon <- function(params) {\n",
    "    new_babyweight(\"model.rds\", \"preproc.rds\")\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Build R Image**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "export PATH=$PATH:$PWD/tmp\n",
    "export PROJECT_ID=$(gcloud config get-value project)\n",
    "cd ./serving\n",
    "s2i build . seldonio/seldon-core-s2i-r:0.1 gcr.io/$PROJECT_ID/r-babyweight:latest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Test Serving Container Locally**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "export PROJECT_ID=$(gcloud config get-value project)\n",
    "docker run --name \"babyweight_predictor\" -d --rm -p 5000:5000 gcr.io/$PROJECT_ID/r-babyweight:latest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "curl -g http://localhost:5000/predict --data-urlencode \\\n",
    "        'json={\"data\": \n",
    "                {\"names\": \n",
    "                    [\"is_male\", \n",
    "                     \"child_race\",  \n",
    "                     \"mother_age\", \n",
    "                     \"father_age\", \n",
    "                     \"gestation_weeks\"],\n",
    "                 \"ndarray\": \n",
    "                     [[1,1,40,44,39]]}}'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# view logs for debugging\n",
    "!docker logs babyweight_predictor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# stop container\n",
    "!docker rm babyweight_predictor --force"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!gcloud auth configure-docker"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "export PROJECT_ID=$(gcloud config get-value project)\n",
    "docker push gcr.io/$PROJECT_ID/r-babyweight:latest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Create a 2 node kubernetes cluster on Google Kubernetes Engine**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!gcloud container clusters create babyweight-cluster --num-nodes=2 --region us-central1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Verify Deployment\n",
    "!gcloud compute instances list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Deploy serving application to GKE**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "export PROJECT_ID=$(gcloud config get-value project)\n",
    "kubectl run babyweight --image=gcr.io/$PROJECT_ID/r-babyweight:latest --port 5000"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# verify deployment\n",
    "!kubectl get pods"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Expose the application on port 80, get the external IP, and send a request**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!kubectl expose deployment babyweight --type=LoadBalancer --port 80 --target-port 5000"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!kubectl get service"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "export EXTERNAL_IP=35.238.0.217\n",
    "curl -g http://$EXTERNAL_IP:80/predict --data-urlencode \\\n",
    "        'json={\"data\": \n",
    "                {\"names\": \n",
    "                    [\"is_male\", \n",
    "                     \"child_race\",  \n",
    "                     \"mother_age\", \n",
    "                     \"father_age\", \n",
    "                     \"gestation_weeks\"],\n",
    "                 \"ndarray\": \n",
    "                     [[1,1,40,44,39]]}}'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}