feat(cli,model,llm-runtime): unify LLM runtimes (#285)

Because

- LLM runtimes are rapidly evolving, and mainstream tools like
`Transformers`, `vLLM`, and `MLC LLM` now support advanced LLM features.

This commit

- refactored CLI codebase
- unified Dockerfile for multi-platform build current supporting CPU for
both `amd64` and `arm64` and GPU for only `amd66`
- added unit testing for CLI commands

Author: pinglin

.github/CODEOWNERS

@@ -1 +1 @@
-* @GeorgeWilliamStrong @pinglin
+* @pinglin

.github/CONTRIBUTING.md

@@ -27,34 +27,34 @@ Before delving into the details to come up with your first PR, please familiaris
 
 To confirm these system dependencies are configured correctly:
 
-```bash
-$ make doctor
+```shell
+make doctor
 ```
 
 #### Installation
 
 Fetch git submodules:
 
-```bash
-$ make get-proto
+```shell
+make get-proto
 ```
 
 Fetch and generate json schema files and dataclass:
 
-```bash
-$ make update-specs
+```shell
+make update-specs
 # `instill/resources/schema/airbyte/OAuth2.py` will be generated with an error, need to manually remove the `OAuth2.` in `OAuth2.AuthType.OAuth2_0`
 ```
 
 > [!IMPORTANT]  
-> **Create a `.venv` folder if you are rusing `conda`**  
+> **Create a `.venv` folder if you are using `conda`**  
 > This will be fixed soon, but currently if you are installing the development dependencies inside a conda env,
-> you will need to create a `.venv` folder in your proejct root to prevent errors
+> you will need to create a `.venv` folder in your project root to prevent errors
 
 Install project dependencies into a virtual environment:
 
-```text
-$ make install
+```shell
+make install
 ```
 
 ### Development Tasks
@@ -63,28 +63,28 @@ $ make install
 
 Run the tests:
 
-```text
-$ make test
+```shell
+make test
 ```
 
 Run static analysis:
 
-```text
-$ make check
+```shell
+make check
 ```
 
 Build the documentation:
 
-```text
-$ make docs
+```shell
+make docs
 ```
 
 #### Automatic
 
 Keep all of the above tasks running on change:
 
-```text
-$ make dev
+```shell
+make dev
 ```
 
 > In order to have OS X notifications, `brew install terminal-notifier`.
@@ -93,16 +93,16 @@ $ make dev
 
 The CI server will report overall build status:
 
-```text
-$ make all
+```shell
+make all
 ```
 
 ### Release Tasks
 
 Release to PyPI:
 
-```text
-$ make upload
+```shell
+make upload
 ```
 
 ### Sending PRs

.github/workflows/build.yml

@@ -7,7 +7,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: ["3.9", "3.10", "3.11"]
+        python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
 
     steps:
       - uses: actions/checkout@v4
@@ -19,7 +19,9 @@ jobs:
         with:
           python-version: ${{ matrix.python-version }}
 
-      - uses: Gr1N/setup-poetry@v9
+      - uses: abatilo/actions-poetry@v4
+        with:
+          poetry-version: 2.1.2
 
       - name: Check dependencies
         run: make doctor

.pylint.ini

@@ -69,6 +69,7 @@ disable=
     too-many-instance-attributes,
     too-many-statements,
     too-many-lines,
+    too-many-positional-arguments,
     attribute-defined-outside-init,
     unsupported-assignment-operation,
     unsupported-delete-operation,
@@ -408,5 +409,5 @@ known-third-party=enchant
 [EXCEPTIONS]
 
 # Exceptions that will emit a warning when being caught. Defaults to
-# "Exception"
-overgeneral-exceptions=Exception
+# "builtins.Exception"
+overgeneral-exceptions=builtins.Exception

.verchew.ini

@@ -11,4 +11,4 @@ version = 3
 [Poetry]
 
 cli = poetry
-version = 1
+version = 2

README.md

@@ -1,10 +1,12 @@
-[![Unix Build Status](https://img.shields.io/github/actions/workflow/status/instill-ai/python-sdk/build.yml?branch=main&label=linux)](https://github.com/instill-ai/python-sdk/actions) [![Coverage Status](https://img.shields.io/codecov/c/gh/instill-ai/python-sdk)](https://codecov.io/gh/instill-ai/python-sdk) [![PyPI License](https://img.shields.io/pypi/l/instill-sdk.svg)](https://pypi.org/project/instill-sdk) [![PyPI Version](https://img.shields.io/pypi/v/instill-sdk.svg)](https://pypi.org/project/instill-sdk) [![PyPI Downloads](https://img.shields.io/pypi/dm/instill-sdk.svg?color=orange)](https://pypistats.org/packages/instill-sdk)
+# Instill Core Python SDK
 
-> [!IMPORTANT]<br>
-> **This SDK tool is under active development**<br>
-> For any bug found or featur request, feel free to open any issue regarding this SDK in our [instill-core](https://github.com/instill-ai/instill-core/issues) repo.
+[![Unix Build Status](https://img.shields.io/github/actions/workflow/status/instill-ai/python-sdk/build.yml?branch=main&label=linux)](https://github.com/instill-ai/python-sdk/actions) [![Coverage Status](https://img.shields.io/codecov/c/gh/instill-ai/python-sdk)](https://codecov.io/gh/instill-ai/python-sdk) [![PyPI License](https://img.shields.io/pypi/l/instill-sdk.svg?color=lightgreen)](https://pypi.org/project/instill-sdk) [![PyPI Version](https://img.shields.io/pypi/v/instill-sdk.svg?color=lightgreen)](https://pypi.org/project/instill-sdk) [![PyPI Downloads](https://img.shields.io/pypi/dm/instill-sdk.svg?color=lightgreen)](https://pypistats.org/packages/instill-sdk)
 
-# Overview
+> [!IMPORTANT]
+> **This SDK tool is under active development**
+> For any bug found or feature request, feel free to open any issue regarding this SDK in our [instill-core](https://github.com/instill-ai/instill-core/issues) repo.
+
+## Overview
 
 Welcome to Instill Python SDK, where the world of AI-first application comes alive in the form of Python.
 
@@ -15,53 +17,54 @@ Before you jump into creating your first application with this SDK tool, we reco
 
 ## Setup
 
-> [!NOTE]<br>
+> [!NOTE]
 > For setting up development environment, please refer to [Contributing](#contributing)
 
 ### Requirements
 
-- Python 3.8 - 3.11
+- Python 3.9 - 3.12
+- CUDA 12.2 - 12.8 (if you want to build a model with GPU support)
 
 ### Installation
 
-> [!WARNING]<br>
-> If your host machine is on arm64 architecture(including Apple silicon machines, equipped with m1/m2 processors), there are some issues when installing `grpcio` within `conda` environment. You will have to manually build and install it like below. Read more about this issue [here](https://github.com/grpc/grpc/issues/33714).
+> [!WARNING]
+> If your host machine is on arm64 architecture(including Apple silicon machines, equipped with m1/m2 processors), there are some issues when installing `grpcio` within `conda` environment. You will have to manually build and install it like below. Read more about this issue in the [gRPC GitHub issue](https://github.com/grpc/grpc/issues/33714).
 
-```bash
-$ GRPC_PYTHON_LDFLAGS=" -framework CoreFoundation" pip install grpcio --no-binary :all:
+```shell
+GRPC_PYTHON_LDFLAGS=" -framework CoreFoundation" pip install grpcio --no-binary :all:
 ```
 
 Install it directly into an activated virtual environment:
 
-```text
-$ pip install instill-sdk
+```shell
+pip install instill-sdk
 ```
 
 or add it to your [Poetry](https://poetry.eustace.io/) project:
 
-```text
-$ poetry add instill-sdk
+```shell
+poetry add instill-sdk
 ```
 
 ### Check import
 
 After installation, you can check if it has been installed correctly:
 
-```text
-$ python
+```shell
+python
 >>> import instill
 >>> instill.__version__
 ```
 
-### Config `Instill Core` or `Instill Cloud` instance
+### Config `Instill Core` or `Managed Instill Core` instance
 
 Before we can start using this SDK, you will need to properly config your target instance. We support two ways to setup the configs, which are
 
 #### Config file
 
 create a config file under this path `${HOME}/.config/instill/sdk/python/config.yml`, and within that path you will need to fill in some basic parameters for your desired host.[^1]
 
-[^1]: You can obtain an `api_token` by simply going to Settings > API Tokens page from the console, no matter it is `Instill Core` or `Instill Cloud`.
+[^1]: You can obtain an `api_token` by simply going to Settings > API Tokens page from the console, no matter it is `Instill Core` or `Managed Instill Core`.
 
 Within the config file, you can define multiple instances with the `alias` of your liking, later in the SDK you can refer to this `alias` to switch between instances.[^2]
 
@@ -143,9 +146,9 @@ user = client.mgmt_service.get_user()
 # ...
 ```
 
-Please find more usages for this sdk at [here](https://www.instill-ai.dev/docs/sdk/python#usage)
+Please find more usages for this SDK in our [documentation](https://www.instill-ai.dev/docs/sdk/python#usage)
 
-**You can also find some notebook examples [here](https://github.com/instill-ai/python-sdk/tree/main/notebooks)**
+**You can also find some notebook examples in our [notebooks directory](https://github.com/instill-ai/python-sdk/tree/main/notebooks)**
 
 ### Create a model
 
@@ -168,7 +171,7 @@ client.model_service.create_model(
 
 `Instill Model` is an advanced MLOps/LLMOps platform that was specifically crafted to facilitate the efficient management and orchestration of model deployments for unstructured data ETL. With `Instill Model`, you can easily create, manage, and deploy your own custom models with ease in `Instill Core` or on the cloud with `Instill Cloud`.
 
-Follow the instructions [here](https://www.instill-ai.dev/docs/model/create) to build and deploy your model.
+Follow the [model creation guide](https://www.instill-ai.dev/docs/model/create) to build and deploy your model.
 
 ### Create pipeline
 

instill/clients/client.py

@@ -5,7 +5,7 @@
 from instill.clients.model import ModelClient
 from instill.clients.pipeline import PipelineClient
 from instill.helpers.const import HOST_URL_PROD
-from instill.utils.error_handler import NotServingException
+from instill.utils.error_handler import NotServingException, NamespaceException
 from instill.utils.logger import Logger
 
 
@@ -76,7 +76,7 @@ def _lookup_namespace_uid(self, namespace_id: str):
         ):
             namespace_uid = self.mgmt.get_organization(namespace_id).organization.uid
         else:
-            raise Exception("namespace ID not available")
+            raise NamespaceException("namespace ID not available")
 
         return namespace_uid
 

instill/clients/mgmt.py

@@ -14,7 +14,7 @@
 from instill.clients.base import Client, RequestFactory
 from instill.clients.instance import InstillInstance
 from instill.helpers.const import HOST_URL_PROD
-from instill.utils.error_handler import grpc_handler
+from instill.utils.error_handler import grpc_handler, NamespaceException
 
 
 class MgmtClient(Client):
@@ -75,7 +75,7 @@ def _lookup_namespace_uid(self, namespace_id: str):
         ):
             namespace_uid = self.get_organization(namespace_id).organization.uid
         else:
-            raise Exception("namespace ID not available")
+            raise NamespaceException("namespace ID not available")
 
         return namespace_uid