diff --git a/docs/examples/intro/local.md b/docs/examples/intro/local.md index 8c5349fc..9530e1ee 100644 --- a/docs/examples/intro/local.md +++ b/docs/examples/intro/local.md @@ -4,8 +4,11 @@ This notebook will walk you through an end-to-end example deploying a Tempo pipe ## Prerequisites - * rclone and conda installed. - * Run this notebook within the `seldon-examples` conda environment. Details to create this can be found [here](). +This notebooks needs to be run in the `tempo-examples` conda environment defined below. Create from project root folder: + +```bash +conda env create --name tempo-examples --file conda/tempo-examples.yaml +``` ## Architecture diff --git a/docs/index.rst b/docs/index.rst index f0a66cf4..ee21feab 100755 --- a/docs/index.rst +++ b/docs/index.rst @@ -8,6 +8,7 @@ Tempo :caption: Documentation: Quickstart + Architecture .. toctree:: :titlesonly: diff --git a/docs/overview/architecture.md b/docs/overview/architecture.md new file mode 100644 index 00000000..e87cf3c0 --- /dev/null +++ b/docs/overview/architecture.md @@ -0,0 +1,47 @@ +# Architecture + +Overview of Tempo architecture. + +![internals](internals.png) + + +## Model and Pipeline + +A Model is the core deployment artifact in tempo and describes the link to a saved machine learning component. A Pipeline is a custom python orchestrator that references other Tempo models or pipelines specied in its definition. + +![model-pipeline](model-pipeline.png) + + +A Model can be: + + * An optimized prepackaged server running a model artifact created by the data scientist, e.g. a Tnesorflow model. + * Custom python code (defined via a class or function). + + +A Pipeline is custom python code (defined via a class or function) which references other Tempo components as callabale functions. + +An example is shown below for the outlier demo. There we have a Pipeline which calls the outlier model and if the request is determined to be an outlier then the processing is halted otherwise the model is called. + +![outlier demo](outlier-demo.png) + +## Runtimes + +Runtimes provide the core functionality to deploy a Tempo model to an infrastructure platform. + +Read more about [Tempo Runtimes](./runtimes.html) + + +## Protocols + +Models when deployed will expose endpoints that respect a particular protocol. The available protocols in Tempo are shown below. + +| Protocol | Description | +|--------|---------| +| KFServingV2Protocol | [V2 definition](https://github.com/kubeflow/kfserving/tree/master/docs/predict-api/v2) | +| KFServingV1Protocol | [Tensorflow protocol](https://github.com/tensorflow/serving/blob/master/tensorflow_serving/g3doc/api_rest.md) | +| SeldonProtocol | [Seldon protcol definition](https://docs.seldon.io/projects/seldon-core/en/latest/graph/protocols.html#rest-and-grpc-seldon-protocol) | + + +The default protocol is KFServingV2 protocol. + +If calling the model with tempo you will not need to deal with the protocol explicitly as translation from the defined python types to runtime payloads and vice versa will be automatic. diff --git a/docs/overview/internals.png b/docs/overview/internals.png new file mode 100644 index 00000000..a7665e99 Binary files /dev/null and b/docs/overview/internals.png differ diff --git a/docs/overview/model-pipeline.png b/docs/overview/model-pipeline.png new file mode 100644 index 00000000..544414f4 Binary files /dev/null and b/docs/overview/model-pipeline.png differ diff --git a/docs/overview/outlier-demo.png b/docs/overview/outlier-demo.png new file mode 100644 index 00000000..d1415588 Binary files /dev/null and b/docs/overview/outlier-demo.png differ diff --git a/docs/overview/runtimes.md b/docs/overview/runtimes.md new file mode 100644 index 00000000..eb6aa2ab --- /dev/null +++ b/docs/overview/runtimes.md @@ -0,0 +1,20 @@ +# Runtimes + +Tempo runtimes provide the core functionality to deploy a tempo Model. They must provide concrete implementations for the following functionality: + +| Method | Action | +|--------|---------| +| deploy | deploy a model | +| undeploy | undeploy a model | +| wait_ready | wait for a deployed model to be ready | +| get_endpoint | get the URL for the deployed model so it can be called | +| to_k8s_yaml | optionally get the Kubernetes declarative yaml for the model | + +The Runtimes defined within Tempo are: + +| Runtime | Infrastructure Target | Example | +| ------- | --------------------- | -------- | +| SeldonDockerRuntime | deploy Tempo models to Docker | [intro](../examples/intro/local.html) | +| SeldonKubernetesRuntime | deploy Tempo models to a Kubernetes cluster with Seldon Core installed | [intro_k8s](../examples/intro/k8s.html) | +| KFServingKubernetesRuntime | deploy Tempo models to a Kubernetes cluster with KFServing installed | [intro](../examples/kfserving/README.html) | +| SeldonDeployRuntime | deploy Tempo models to a Kubernetes cluster with Seldon Deploy installed | | \ No newline at end of file