Skip to content

Add support for modelscope and update doc #1381

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
May 12, 2025
Merged

Add support for modelscope and update doc #1381

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
15da984
Add support for modelscope
yeahdongcn May 10, 2025
9985b9e
Format changes for passing CI
yeahdongcn May 10, 2025
1aac29e
Address review comments
yeahdongcn May 11, 2025
9e41094
Add unit tests
yeahdongcn May 11, 2025
590199c
Address review comments
yeahdongcn May 11, 2025
34af059
Address issues found by CI
yeahdongcn May 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,7 @@ RamaLama supports multiple AI model registries types called transports.
| Transports | Web Site |
| :------------- | :--------------------------------------------------- |
| HuggingFace | [`huggingface.co`](https://www.huggingface.co) |
| ModelScope | [`modelscope.cn`](https://www.modelscope.cn) |
| Ollama | [`ollama.com`](https://www.ollama.com) |
| OCI Container Registries | [`opencontainers.org`](https://opencontainers.org) |
| |Examples: [`quay.io`](https://quay.io), [`Docker Hub`](https://docker.io), [`Pulp`](https://pulpproject.org), and [`Artifactory`](https://jfrog.com/artifactory/)|
Expand All @@ -145,7 +146,7 @@ How to change transports.

Use the RAMALAMA_TRANSPORT environment variable to modify the default. `export RAMALAMA_TRANSPORT=huggingface` Changes RamaLama to use huggingface transport.

Individual model transports can be modified when specifying a model via the `huggingface://`, `oci://`, or `ollama://` prefix.
Individual model transports can be modified when specifying a model via the `huggingface://`, `oci://`, `modelscope://`, or `ollama://` prefix.

Example:
```
Expand Down
7 changes: 4 additions & 3 deletions docs/ramalama-bench.1.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,10 @@ ramalama\-bench - benchmark specified AI Model

| Transports | Prefix | Web Site |
| ------------- | ------ | --------------------------------------------------- |
| URL based | https://, http://, file:// | `https://web.site/ai.model`, `file://tmp/ai.model`|
| HuggingFace | huggingface://, hf://, hf.co/ | [`huggingface.co`](https://www.huggingface.co) |
| Ollama | ollama:// | [`ollama.com`](https://www.ollama.com) |
| URL based | https://, http://, file:// | `https://web.site/ai.model`, `file://tmp/ai.model`|
| HuggingFace | huggingface://, hf://, hf.co/ | [`huggingface.co`](https://www.huggingface.co)|
| ModelScope | modelscope://, ms:// | [`modelscope.cn`](https://modelscope.cn/)|
| Ollama | ollama:// | [`ollama.com`](https://www.ollama.com)|
| OCI Container Registries | oci:// | [`opencontainers.org`](https://opencontainers.org)|
|||Examples: [`quay.io`](https://quay.io), [`Docker Hub`](https://docker.io),[`Artifactory`](https://artifactory.com)|

Expand Down
2 changes: 1 addition & 1 deletion docs/ramalama-list.1.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ List all Models downloaded to users homedir
```
$ ramalama list
NAME MODIFIED SIZE
ollama://smollm:135m 16 hours ago 5.5M
ollama://smollm:135m 16 hours ago 5.5M
huggingface://afrideva/Tiny-Vicuna-1B-GGUF/tiny-vicuna-1b.q2_k.gguf 14 hours ago 460M
ollama://granite-code:3b 5 days ago 1.9G
ollama://granite-code:latest 1 day ago 1.9G
Expand Down
8 changes: 8 additions & 0 deletions docs/ramalama-login.1.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,14 @@ $ ramalama login --token=XYZ
```
Logging in to Hugging Face requires the `huggingface-cli` tool. For installation and usage instructions, see the documentation of the Hugging Face command line interface: [*https://huggingface.co/docs/huggingface_hub/en/guides/cli*](https://huggingface.co/docs/huggingface_hub/en/guides/cli).

Login to ModelScope registry
```
$ export RAMALAMA_TRANSPORT=modelscope
$ ramalama login --token=XYZ
```

Logging in to ModelScope requires the `modelscope` tool. For installation and usage instructions, see the documentation of the ModelScope command line interface: [*https://www.modelscope.cn/docs/Beginner-s-Guide/Environment-Setup*](https://www.modelscope.cn/docs/Beginner-s-Guide/Environment-Setup).

## SEE ALSO
**[ramalama(1)](ramalama.1.md)**

Expand Down
7 changes: 4 additions & 3 deletions docs/ramalama-perplexity.1.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,10 @@ ramalama\-perplexity - calculate the perplexity value of an AI Model

| Transports | Prefix | Web Site |
| ------------- | ------ | --------------------------------------------------- |
| URL based | https://, http://, file:// | `https://web.site/ai.model`, `file://tmp/ai.model`|
| HuggingFace | huggingface://, hf://, hf.co/ | [`huggingface.co`](https://www.huggingface.co) |
| Ollama | ollama:// | [`ollama.com`](https://www.ollama.com) |
| URL based | https://, http://, file:// | `https://web.site/ai.model`, `file://tmp/ai.model`|
| HuggingFace | huggingface://, hf://, hf.co/ | [`huggingface.co`](https://www.huggingface.co)|
| ModelScope | modelscope://, ms:// | [`modelscope.cn`](https://modelscope.cn/)|
| Ollama | ollama:// | [`ollama.com`](https://www.ollama.com)|
| OCI Container Registries | oci:// | [`opencontainers.org`](https://opencontainers.org)|
|||Examples: [`quay.io`](https://quay.io), [`Docker Hub`](https://docker.io),[`Artifactory`](https://artifactory.com)|

Expand Down
7 changes: 4 additions & 3 deletions docs/ramalama-run.1.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,10 @@ ramalama\-run - run specified AI Model as a chatbot

| Transports | Prefix | Web Site |
| ------------- | ------ | --------------------------------------------------- |
| URL based | https://, http://, file:// | `https://web.site/ai.model`, `file://tmp/ai.model`|
| HuggingFace | huggingface://, hf://, hf.co/ | [`huggingface.co`](https://www.huggingface.co) |
| Ollama | ollama:// | [`ollama.com`](https://www.ollama.com) |
| URL based | https://, http://, file:// | `https://web.site/ai.model`, `file://tmp/ai.model`|
| HuggingFace | huggingface://, hf://, hf.co/ | [`huggingface.co`](https://www.huggingface.co)|
| ModelScope | modelscope://, ms:// | [`modelscope.cn`](https://modelscope.cn/)|
| Ollama | ollama:// | [`ollama.com`](https://www.ollama.com)|
| OCI Container Registries | oci:// | [`opencontainers.org`](https://opencontainers.org)|
|||Examples: [`quay.io`](https://quay.io), [`Docker Hub`](https://docker.io),[`Artifactory`](https://artifactory.com)|

Expand Down
9 changes: 5 additions & 4 deletions docs/ramalama-serve.1.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,10 @@ registry if it does not exist in local storage.

| Transports | Prefix | Web Site |
| ------------- | ------ | --------------------------------------------------- |
| URL based | https://, http://, file:// | `https://web.site/ai.model`, `file://tmp/ai.model`|
| HuggingFace | huggingface://, hf://, hf.co/ | [`huggingface.co`](https://www.huggingface.co) |
| Ollama | ollama:// | [`ollama.com`](https://www.ollama.com) |
| URL based | https://, http://, file:// | `https://web.site/ai.model`, `file://tmp/ai.model`|
| HuggingFace | huggingface://, hf://, hf.co/ | [`huggingface.co`](https://www.huggingface.co)|
| ModelScope | modelscope://, ms:// | [`modelscope.cn`](https://modelscope.cn/)|
| Ollama | ollama:// | [`ollama.com`](https://www.ollama.com)|
| OCI Container Registries | oci:// | [`opencontainers.org`](https://opencontainers.org)|
|||Examples: [`quay.io`](https://quay.io), [`Docker Hub`](https://docker.io),[`Artifactory`](https://artifactory.com)|

Expand Down Expand Up @@ -112,7 +113,7 @@ On Nvidia based GPU systems, RamaLama defaults to using the
`nvidia-container-runtime`. Use this option to override this selection.

#### **--port**, **-p**
port for AI Model server to listen on. It must be available. If not specified,
port for AI Model server to listen on. It must be available. If not specified,
the serving port will be 8080 if available, otherwise a free port in 8081-8090 range.

#### **--privileged**
Expand Down
7 changes: 4 additions & 3 deletions docs/ramalama.1.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,9 +72,10 @@ RamaLama supports multiple AI model registries types called transports. Supporte

| Transports | Prefix | Web Site |
| ------------- | ------ | --------------------------------------------------- |
| URL based | https://, http://, file:// | `https://web.site/ai.model`, `file://tmp/ai.model`|
| HuggingFace | huggingface://, hf://, hf.co/ | [`huggingface.co`](https://www.huggingface.co) |
| Ollama | ollama:// | [`ollama.com`](https://www.ollama.com) |
| URL based | https://, http://, file:// | `https://web.site/ai.model`, `file://tmp/ai.model`|
| HuggingFace | huggingface://, hf://, hf.co/ | [`huggingface.co`](https://www.huggingface.co)|
| ModelScope | modelscope://, ms:// | [`modelscope.cn`](https://modelscope.cn/)|
| Ollama | ollama:// | [`ollama.com`](https://www.ollama.com)|
| OCI Container Registries | oci:// | [`opencontainers.org`](https://opencontainers.org)|
|||Examples: [`quay.io`](https://quay.io), [`Docker Hub`](https://docker.io),[`Artifactory`](https://artifactory.com)|

Expand Down
2 changes: 1 addition & 1 deletion ramalama/cli.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -261,7 +261,7 @@ def normalize_registry(registry):
if not registry or registry == "" or registry.startswith("oci://"):
return "oci://"

if registry in ["ollama", "hf", "huggingface"]:
if registry in ["ollama", "hf", "huggingface", "ms", "modelscope"]:
return registry

return "oci://" + registry
Expand Down
21 changes: 17 additions & 4 deletions ramalama/model_factory.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@

from ramalama.common import rm_until_substring
from ramalama.huggingface import Huggingface
from ramalama.modelscope import ModelScope
from ramalama.model import MODEL_TYPES
from ramalama.model_store import GlobalModelStore, ModelStore
from ramalama.oci import OCI
Expand All @@ -28,8 +29,8 @@ def __init__(
self.ignore_stderr = ignore_stderr
self.container = args.container

self.model_cls: type[Union[Huggingface, Ollama, OCI, URL]]
self.create: Callable[[], Union[Huggingface, Ollama, OCI, URL]]
self.model_cls: type[Union[Huggingface, ModelScope, Ollama, OCI, URL]]
self.create: Callable[[], Union[Huggingface, ModelScope, Ollama, OCI, URL]]
self.model_cls, self.create = self.detect_model_model_type()

self.pruned_model = self.prune_model_input()
Expand All @@ -44,6 +45,8 @@ def detect_model_model_type(
) -> Tuple[type[Union[Huggingface, Ollama, OCI, URL]], Callable[[], Union[Huggingface, Ollama, OCI, URL]]]:
if self.model.startswith("huggingface://") or self.model.startswith("hf://") or self.model.startswith("hf.co/"):
return Huggingface, self.create_huggingface
if self.model.startswith("modelscope://") or self.model.startswith("ms://"):
return ModelScope, self.create_modelscope
if self.model.startswith("ollama://") or "ollama.com/library/" in self.model:
return Ollama, self.create_ollama
if self.model.startswith("oci://") or self.model.startswith("docker://"):
Expand All @@ -53,19 +56,23 @@ def detect_model_model_type(

if self.transport == "huggingface":
return Huggingface, self.create_huggingface
if self.transport == "modelscope":
return ModelScope, self.create_modelscope
if self.transport == "ollama":
return Ollama, self.create_ollama
if self.transport == "oci":
return OCI, self.create_oci

raise KeyError(f'transport "{self.transport}" not supported. Must be oci, huggingface, or ollama.')
raise KeyError(f'transport "{self.transport}" not supported. Must be oci, huggingface, modelscope, or ollama.')

def prune_model_input(self) -> str:
# remove protocol from model input
pruned_model_input = rm_until_substring(self.model, "://")

if self.model_cls == Huggingface:
pruned_model_input = rm_until_substring(pruned_model_input, "hf.co/")
elif self.model_cls == ModelScope:
pruned_model_input = rm_until_substring(pruned_model_input, "modelscope.cn/")
elif self.model_cls == Ollama:
pruned_model_input = rm_until_substring(pruned_model_input, "ollama.com/library/")

Expand All @@ -79,7 +86,7 @@ def validate_oci_model_input(self):
if self.model.startswith(t + "://"):
raise ValueError(f"{self.model} invalid: Only OCI Model types supported")

def set_optional_model_store(self, model: Union[Huggingface, Ollama, OCI, URL]):
def set_optional_model_store(self, model: Union[Huggingface, ModelScope, Ollama, OCI, URL]):
if self.use_model_store:
name, _, orga = model.extract_model_identifiers()
model.store = ModelStore(GlobalModelStore(self.store_path), name, model.model_type, orga)
Expand All @@ -90,6 +97,12 @@ def create_huggingface(self) -> Huggingface:
model.draft_model = self.draft_model
return model

def create_modelscope(self) -> ModelScope:
model = ModelScope(self.pruned_model)
self.set_optional_model_store(model)
model.draft_model = self.draft_model
return model

def create_ollama(self) -> Ollama:
model = Ollama(self.pruned_model)
self.set_optional_model_store(model)
Expand Down
Loading
Loading