The contents in this repository is for demonstrating Riva SDK workflow. Please execute the notebook files described below in sequence to avoid some dependencies.
It shows how to use basic riva api with simple examples.
It shows how to finetune the existing pre-trained model to your target task using NeMo toolkit based on TTS example(fastptich-HifiGan).
It contains how to convert NeMo to Riva format, how to build Riva to get RMIR, and finally, how to deploy it to the Riva Service.
Some additional information about NeMo is included.
Korean ASR models(Conformer-CTC, Citrinet) demonstration with Riva.
Note that the RIVA Server and Client are using the same docker image. I'd like to make sure that the docker network riva-speech
might not be actually needed according to the environment(anyone who tires this demo can adjust it according to your own enviornment). In this demo case, the remote machine is running within the VPN, which requires the private docker network defined to connect with each other, and to the browser for the Juptyerlab.
-
Clone the repository.
git clone https://github.com/woojinsoh/riva_demo.git cd riva_demo
-
Get access to NGC and install NGC CLI tools.
-
Download Riva resources from NGC.
mkdir –p ./resources && cd resources ngc registry resource download-version "nvidia/riva/riva_quickstart:2.8.1" # updated(2023.01.02)
-
Create docker network.
docker network create riva-speech
-
Initialize and start Riva Server.
- Move to the resource directory.
cd riva_quickstart:2.8.1
- Modify
config.sh
with your preffered configuration from downloaded resources. Then, run:
bash riva_init.sh # it usually takes over 40 mins by default.
-
Start Riva Server.
- Add
--net=riva-speech
option to docker running command inriva_start.sh
from downloaded resources. Then, run:
bash riva_start.sh
- Add
-
Start Riva Client.
- Add some options to docker running command in
riva_start_client.sh
. For example,- Mount your workspace to the docker container(i.e.,
-v <myworkspace_path>:/workspace
) - Apply the same docker network(
--net=riva-speech
) with that of the Riva server. - Additional port mappings(
-p 1005:8888
for jupyterlab,-p 8009:8009
for additional usage) are also required.
- Mount your workspace to the docker container(i.e.,
- Then, run:
bash riva_start_client.sh
- Add some options to docker running command in
- Binary wheel for format converter from NeMo to Riva.
- Automatically initialize and setup runnable riva environment.
- Pull riva container images(server, client, servicemaker)
- Download default built models(RMIRs) enabled in
config.sh
from NGC.- Download location:
$riva_model_loc/rmir
, which is defined as docker volume.
- Download location:
- Optimize the downloaded models and deploy them to model repository.
- Deploy location:
$riva_model_loc/models
- Deploy location:
- Start riva server port # 8000, 8001, 8002 for triton, port 50051(default) for riva-speech api.
- Preload model repos based on desired services.
- Start the riva client.
- Shut down the riva server container.
- Clean up local riva installation.
- You might need to remove docker volume manually after this shell script is done.