Skip to content
Huaizheng edited this page Apr 22, 2021 · 4 revisions

MLModelCI is an easy-to-use toolkit to build high-performing ML products from newly trained research models.

APIs described in this are classified by different levels


modelci service

  • modelci service init

  • modelci service stop

  • modelci service clean

  • modelci service monitor

modelci modelhub

  • modelci modelhub publish
Publish a model to the ModelCI. You can choose either use a YAML file (using the -f option), or input all parameters using other options/arguments.

Arguments:                                                                                                                                                                                                                             
  [FILE_OR_DIR]  Model weight files 

Options:                                                                                                                                                                                                                               
  -n, --name TEXT                 Architecture                                                                                                                                                                                         
  -fw, --framework [TensorFlow|PyTorch]                                                                                                                                                                                                
                                  Framework                                                                                                                                                                                            
  -e, --engine [NONE|TFS|TORCHSCRIPT|ONNX|TRT|TVM|CUSTOMIZED|PYTORCH]                                                                                                                                                                  
                                  Engine                                                                                                                                                                                               
  -v, --version INTEGER RANGE     Version number                                                                                                                                                                                       
  -t, --task [Image_Classification|Object_Detection|Segmentation]                                                                                                                                                                      
                                  Task                                                                                                                                                                                                 
  -d, --dataset TEXT              Dataset name                                                                                                                                                                                         
  --metric DICT                   Metrics in the form of mapping JSON string.                                                                                                                                                          
                                  The map type is                                                                                                                                                                                      
                                  `Dict[types.models.mlmodel.Metric, float]`.                                                                                                                                                          
                                  An example is '{"acc": 0.76}.'  [default:                                                                                                                                                            
                                  {}]                                                                                                                                                                                                  
                                                                                                                                                                                                                                       
  -i, --input PYDANTIC MODEL JSON                                                                                                                                                                                                      
                                  List of shape definitions for input tensors.                                                                                                                                                         
                                  An example of one shape definition is                                                                                                                                                                
                                  '{"name": "input", "shape": [-1, 3, 224,                                                                                                                                                             
                                  224], "dtype": "TYPE_FP32", "format":                                                                                                                                                                
                                  "FORMAT_NCHW"}'  [default: ]                                                                                                                                                                         
                                                                                                                                                                                                                                       
  -o, --output PYDANTIC MODEL JSON                                                                                                                                                                                                     
                                  List of shape definitions for output                                                                                                                                                                 
                                  tensors. An example of one shape definition                                                                                                                                                          
                                  is '{"name": "output", "shape": [-1, 1000],                                                                                                                                                          
                                  "dtype": "TYPE_FP32"}'  [default: ]                                                                                                                                                                  
                                                                                                                                                                                                                                       
  -c, --convert                   Convert the model to other possible format.                                                                                                                                                          
                                  [default: True]

  -p, --profile                   Profile the published model(s).  [default:                                                                                                                                                           
                                  False]                                                                                                                                                                                               
                                                                                                                                                                                                                                       
  -f, --yaml-file PATH            Path to configuration YAML file. You should                                                                                                                                                          
                                  either set the `yaml_file` field or fields                                                                                                                                                           
                                  (`FILE_OR_DIR`, `--name`, `--framework`,                                                                                                                                                             
                                  `--engine`, `--version`, `--task`,                                                                                                                                                                   
                                  `--dataset`,`--metric`, `--input`,                                                                                                                                                                   
                                  `--output`).                                                                                                                                                                                         
                                                                                                                                                                                                                                       
  --help                          Show this message and exit.
  • modelci modelhub ls
Show a table that lists all models published in MLModelCI.

Options:                                                                                                                                                                                                                               
  -n, --name TEXT                 Model architecture name                                                                                                                                                                              
  -fw, --framework [TensorFlow|PyTorch]                                                                                                                                                                                                
                                  Framework                                                                                                                                                                                            
  -e, --engine [NONE|TFS|TORCHSCRIPT|ONNX|TRT|TVM|CUSTOMIZED|PYTORCH]                                                                                                                                                                  
                                  Serving engine                                                                                                                                                                                       
  -v, --version INTEGER           Version                                                                                                                                                                                              
  -a, --all                       Display queried models. otherwise, only                                                                                                                                                              
                                  partial result will be shown.  [default:                                                                                                                                                             
                                  False]                                                                                                                                                                                               
                                                                                                                                                                                                                                       
  --help                          Show this message and exit.
  • modelci modelhub get

download a model weight file from an url

  • modelci modelhub export

For quickly download a model from Pytorch/Tensorflow Hub to our system. Not officially supported API

  • modelci modelhub detail
Show a single model in detail.

Arguments:
  MODEL_ID  Model ID  [required]

Options:
  --help  Show this message and exit.
  • modelci modelhub update
Update a single model info

Arguments:
  MODEL_ID  Model ID  [required]

Options:
  -n, --name TEXT                 Architecture
  -fw, --framework [TensorFlow|PyTorch]
                                  Framework
  -e, --engine [NONE|TFS|TORCHSCRIPT|ONNX|TRT|TVM|CUSTOMIZED|PYTORCH]
                                  Engine
  -v, --version INTEGER RANGE     Version number
  -t, --task [Image_Classification|Object_Detection|Segmentation]
                                  Task
  -d, --dataset TEXT              Dataset name
  --metric DICT                   Metrics in the form of mapping JSON string.
                                  The map type is
                                  `Dict[types.models.mlmodel.Metric, float]`.
                                  An example is '{"acc": 0.76}.'

  -i, --input PYDANTIC MODEL JSON
                                  List of shape definitions for input tensors.
                                  An example of one shape definition is
                                  '{"name": "input", "shape": [-1, 3, 224,
                                  224], "dtype": "TYPE_FP32", "format":
                                  "FORMAT_NCHW"}'  [default: ]

  -o, --output PYDANTIC MODEL JSON
                                  List of shape definitions for output
                                  tensors. An example of one shape definition
                                  is '{"name": "output", "shape": [-1, 1000],
                                  "dtype": "TYPE_FP32"}'  [default: ]

  --help                          Show this message and exit.
  • modelci modelhub delete
Delete a single model by its id

Arguments:
  MODEL_ID  Model ID  [required]

Options:
  --help  Show this message and exit.

  • modelci modelhub convert
convert a single model to all possible alternatives by its id or yaml file.

Arguments:
  You should use one and only option from -i and -f as an argument to 
  assign the model to be converted.
  
Options:
  -i, --id          model id in the modelhub
  -f, --yaml-file   Path to configuration YAML file. You should                                                                                                                                                          
                    either set the `yaml_file` field or fields                                                                                                                                                           
                    (`FILE_OR_DIR`, `--name`, `--framework`,                                                                                                                                                             
                    `--engine`, `--version`, `--task`,                                                                                                                                                                   
                    `--dataset`,`--metric`, `--input`,                                                                                                                                                                   
                    `--output`).  
  -r, --register    whether to register the converted models to modelhub,
                    default false
  --help  Show this message and exit.
  • modelci modelhub profile
  • modelci modelhub dispatch

modelci modelps

  • modelci modelps visulize

pop up a webpage to display a model's structure

  • modelci modelps edit

pop up a webpage to guide users edit a model step by step

  • modelci modelps tune

fine-tune a model on a specific dataset

Clone this wiki locally