Skip to content

Refactor exporting NeMo models #2362

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
csukuangfj merged 8 commits into from
Jul 9, 2025
Merged

Refactor exporting NeMo models #2362

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
810d1c7
export int8 NeMo models
csukuangfj Jul 8, 2025
2545f92
export int8 NeMo models
csukuangfj Jul 8, 2025
6f5b9f9
export int8 NeMo models
csukuangfj Jul 8, 2025
f062f82
export int8 NeMo models
csukuangfj Jul 8, 2025
1c4fd97
export nvidia/parakeet-tdt_ctc-0.6b-ja
csukuangfj Jul 8, 2025
42a093b
minor fixes
csukuangfj Jul 8, 2025
242e8d3
update apk for vad+asr
csukuangfj Jul 9, 2025
67f2587
fix typos
csukuangfj Jul 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .github/workflows/export-nemo-fast-conformer-hybrid-transducer-ctc-non-streaming.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,11 @@ jobs:
sherpa-onnx-nemo-fast-conformer-ctc-en-de-es-fr-14288
sherpa-onnx-nemo-fast-conformer-ctc-be-de-en-es-fr-hr-it-pl-ru-uk-20k
sherpa-onnx-nemo-parakeet_tdt_ctc_110m-en-36000
sherpa-onnx-nemo-fast-conformer-ctc-en-24500-int8
sherpa-onnx-nemo-fast-conformer-ctc-es-1424-int8
sherpa-onnx-nemo-fast-conformer-ctc-en-de-es-fr-14288-int8
sherpa-onnx-nemo-fast-conformer-ctc-be-de-en-es-fr-hr-it-pl-ru-uk-20k-int8
sherpa-onnx-nemo-parakeet_tdt_ctc_110m-en-36000-int8
)

for m in ${models[@]}; do
Expand Down Expand Up @@ -89,6 +94,11 @@ jobs:
sherpa-onnx-nemo-fast-conformer-ctc-en-de-es-fr-14288
sherpa-onnx-nemo-fast-conformer-ctc-be-de-en-es-fr-hr-it-pl-ru-uk-20k
sherpa-onnx-nemo-parakeet_tdt_ctc_110m-en-36000
sherpa-onnx-nemo-fast-conformer-ctc-en-24500-int8
sherpa-onnx-nemo-fast-conformer-ctc-es-1424-int8
sherpa-onnx-nemo-fast-conformer-ctc-en-de-es-fr-14288-int8
sherpa-onnx-nemo-fast-conformer-ctc-be-de-en-es-fr-hr-it-pl-ru-uk-20k-int8
sherpa-onnx-nemo-parakeet_tdt_ctc_110m-en-36000-int8
)
for d in ${dirs[@]}; do
tar cjvf ${d}.tar.bz2 ./$d
Expand Down
57 changes: 50 additions & 7 deletions .github/workflows/export-nemo-fast-conformer-hybrid-transducer-ctc.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,13 +54,18 @@ jobs:
curl -SL -O https://hf-mirror.com/csukuangfj/sherpa-onnx-nemo-ctc-en-conformer-small/resolve/main/test_wavs/trans.txt
popd

cp -av test_wavs ./sherpa-onnx-nemo-streaming-fast-conformer-ctc-en-80ms
cp -av test_wavs ./sherpa-onnx-nemo-streaming-fast-conformer-ctc-en-480ms
cp -av test_wavs ./sherpa-onnx-nemo-streaming-fast-conformer-ctc-en-1040ms

tar cjvf sherpa-onnx-nemo-streaming-fast-conformer-ctc-en-80ms.tar.bz2 sherpa-onnx-nemo-streaming-fast-conformer-ctc-en-80ms
tar cjvf sherpa-onnx-nemo-streaming-fast-conformer-ctc-en-480ms.tar.bz2 sherpa-onnx-nemo-streaming-fast-conformer-ctc-en-480ms
tar cjvf sherpa-onnx-nemo-streaming-fast-conformer-ctc-en-1040ms.tar.bz2 sherpa-onnx-nemo-streaming-fast-conformer-ctc-en-1040ms
names=(
sherpa-onnx-nemo-streaming-fast-conformer-ctc-en-80ms
sherpa-onnx-nemo-streaming-fast-conformer-ctc-en-480ms
sherpa-onnx-nemo-streaming-fast-conformer-ctc-en-1040ms
sherpa-onnx-nemo-streaming-fast-conformer-ctc-en-80ms-int8
sherpa-onnx-nemo-streaming-fast-conformer-ctc-en-480ms-int8
sherpa-onnx-nemo-streaming-fast-conformer-ctc-en-1040ms-int8
)
for d in ${names[@]}; do
cp -av test_wavs $d/
tar cjvf $d.tar.bz2 $d
done

- name: Release
uses: svenstaro/upload-release-action@v2
Expand All @@ -71,3 +76,41 @@ jobs:
repo_name: k2-fsa/sherpa-onnx
repo_token: ${{ secrets.UPLOAD_GH_SHERPA_ONNX_TOKEN }}
tag: asr-models

- name: Publish to huggingface
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
uses: nick-fields/retry@v3
with:
max_attempts: 20
timeout_seconds: 200
shell: bash
command: |
git config --global user.email "csukuangfj@gmail.com"
git config --global user.name "Fangjun Kuang"

models=(
sherpa-onnx-nemo-streaming-fast-conformer-ctc-en-80ms
sherpa-onnx-nemo-streaming-fast-conformer-ctc-en-480ms
sherpa-onnx-nemo-streaming-fast-conformer-ctc-en-1040ms
sherpa-onnx-nemo-streaming-fast-conformer-ctc-en-80ms-int8
sherpa-onnx-nemo-streaming-fast-conformer-ctc-en-480ms-int8
sherpa-onnx-nemo-streaming-fast-conformer-ctc-en-1040ms-int8
)

for m in ${models[@]}; do
rm -rf huggingface
export GIT_LFS_SKIP_SMUDGE=1
export GIT_CLONE_PROTECTION_ACTIVE=false
git clone https://csukuangfj:$HF_TOKEN@huggingface.co/csukuangfj/$m huggingface
cp -av $m/* huggingface
cd huggingface
git lfs track "*.onnx"
git lfs track "*.wav"
git status
git add .
git status
git commit -m "first commit"
git push https://csukuangfj:$HF_TOKEN@huggingface.co/csukuangfj/$m main
cd ..
done
10 changes: 10 additions & 0 deletions .github/workflows/export-nemo-fast-conformer-hybrid-transducer-transducer-non-streaming.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,11 @@ jobs:
sherpa-onnx-nemo-fast-conformer-transducer-en-de-es-fr-14288
sherpa-onnx-nemo-fast-conformer-transducer-be-de-en-es-fr-hr-it-pl-ru-uk-20k
sherpa-onnx-nemo-parakeet_tdt_transducer_110m-en-36000
sherpa-onnx-nemo-fast-conformer-transducer-en-24500-int8
sherpa-onnx-nemo-fast-conformer-transducer-es-1424-int8
sherpa-onnx-nemo-fast-conformer-transducer-en-de-es-fr-14288-int8
sherpa-onnx-nemo-fast-conformer-transducer-be-de-en-es-fr-hr-it-pl-ru-uk-20k-int8
sherpa-onnx-nemo-parakeet_tdt_transducer_110m-en-36000-int8
)

for m in ${models[@]}; do
Expand Down Expand Up @@ -88,6 +93,11 @@ jobs:
sherpa-onnx-nemo-fast-conformer-transducer-en-de-es-fr-14288
sherpa-onnx-nemo-fast-conformer-transducer-be-de-en-es-fr-hr-it-pl-ru-uk-20k
sherpa-onnx-nemo-parakeet_tdt_transducer_110m-en-36000
sherpa-onnx-nemo-fast-conformer-transducer-en-24500-int8
sherpa-onnx-nemo-fast-conformer-transducer-es-1424-int8
sherpa-onnx-nemo-fast-conformer-transducer-en-de-es-fr-14288-int8
sherpa-onnx-nemo-fast-conformer-transducer-be-de-en-es-fr-hr-it-pl-ru-uk-20k-int8
sherpa-onnx-nemo-parakeet_tdt_transducer_110m-en-36000-int8
)
for d in ${dirs[@]}; do
tar cjvf ${d}.tar.bz2 ./$d
Expand Down
57 changes: 50 additions & 7 deletions .github/workflows/export-nemo-fast-conformer-hybrid-transducer-transducer.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,13 +54,18 @@ jobs:
curl -SL -O https://hf-mirror.com/csukuangfj/sherpa-onnx-nemo-ctc-en-conformer-small/resolve/main/test_wavs/trans.txt
popd

cp -av test_wavs ./sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-80ms
cp -av test_wavs ./sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-480ms
cp -av test_wavs ./sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-1040ms

tar cjvf sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-80ms.tar.bz2 sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-80ms
tar cjvf sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-480ms.tar.bz2 sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-480ms
tar cjvf sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-1040ms.tar.bz2 sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-1040ms
models=(
sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-80ms
sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-480ms
sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-1040ms
sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-80ms-int8
sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-480ms-int8
sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-1040ms-int8
)
for m in ${models[@]}; do
cp -av test_wavs $m
tar cjvf $m.tar.bz2 $m
done

- name: Release
uses: svenstaro/upload-release-action@v2
Expand All @@ -71,3 +76,41 @@ jobs:
repo_name: k2-fsa/sherpa-onnx
repo_token: ${{ secrets.UPLOAD_GH_SHERPA_ONNX_TOKEN }}
tag: asr-models

- name: Publish to huggingface
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
uses: nick-fields/retry@v3
with:
max_attempts: 20
timeout_seconds: 200
shell: bash
command: |
git config --global user.email "csukuangfj@gmail.com"
git config --global user.name "Fangjun Kuang"

models=(
sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-80ms
sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-480ms
sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-1040ms
sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-80ms-int8
sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-480ms-int8
sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-1040ms-int8
)

for m in ${models[@]}; do
rm -rf huggingface
export GIT_LFS_SKIP_SMUDGE=1
export GIT_CLONE_PROTECTION_ACTIVE=false
git clone https://csukuangfj:$HF_TOKEN@huggingface.co/csukuangfj/$m huggingface
cp -av $m/* huggingface
cd huggingface
git lfs track "*.onnx"
git lfs track "*.wav"
git status
git add .
git status
git commit -m "first commit"
git push https://csukuangfj:$HF_TOKEN@huggingface.co/csukuangfj/$m main
cd ..
done
Comment on lines +101 to +116

@coderabbitai coderabbitai Bot Jul 9, 2025

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Make the Hugging Face publish loop idempotent & cheaper

  1. git lfs track "*.onnx" and "*.wav" run on every iteration – do it once outside the loop or inside the repo after the first clone.
  2. The commit message "first commit" becomes inaccurate after the first push – consider a generic message ("update models") or include date +"%F" to avoid duplicate-headline warnings.
  3. Add set -euo pipefail at the top of the script block to fail fast on any publishing error.

No diff provided since it spans multiple script blocks.

🤖 Prompt for AI Agents 
In
.github/workflows/export-nemo-fast-conformer-hybrid-transducer-transducer.yaml
around lines 101 to 116, the git lfs track commands for "*.onnx" and "*.wav" are
redundantly run inside the loop for each model; move these commands outside the
loop or run them only once after the initial clone to improve efficiency. Also,
update the commit message from the static "first commit" to a more generic
message like "update models" or include a timestamp using date +"%F" to prevent
duplicate commit message warnings. Finally, add set -euo pipefail at the top of
the script block to ensure the script fails immediately on any error during
publishing.

105 changes: 105 additions & 0 deletions .github/workflows/export-nemo-parakeet-tdt.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
name: export-nemo-parakeet-tdt

on:
push:
branches:
- refactor-export-nemo
workflow_dispatch:

concurrency:
group: export-nemo-parakeet-tdt-${{ github.ref }}
cancel-in-progress: true

jobs:
export-nemo-parakeet-tdt-0_6b-v2:
if: github.repository_owner == 'k2-fsa' || github.repository_owner == 'csukuangfj'
name: parakeet tdt
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [macos-latest]
python-version: ["3.10"]

steps:
- uses: actions/checkout@v4

- name: Setup Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Install python dependencies
shell: bash
run: |
pip install \
nemo_toolkit['asr'] \
"numpy<2" \
ipython \
kaldi-native-fbank \
librosa \
onnx==1.17.0 \
onnxmltools==1.13.0 \
onnxruntime==1.17.1 \
soundfile

- name: Run
shell: bash
run: |
cd scripts/nemo/parakeet-tdt_ctc-0.6b-ja
./run-ctc.sh

- name: Collect files
shell: bash
run: |
models=(
sherpa-onnx-nemo-parakeet-tdt_ctc-0.6b-ja-35000-int8
)
for m in ${models[@]}; do
mv -v scripts/nemo/parakeet-tdt_ctc-0.6b-ja/$m .
tar cjfv $m.tar.bz2 $m
done
Comment on lines +54 to +61

@coderabbitai coderabbitai Bot Jul 9, 2025

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Fix shell script quoting issues.

Shellcheck identifies several quoting issues that should be addressed for robustness.

          models=(
            sherpa-onnx-nemo-parakeet-tdt_ctc-0.6b-ja-35000-int8
          )
-          for m in ${models[@]}; do
-            mv -v scripts/nemo/parakeet-tdt_ctc-0.6b-ja/$m .
-            tar cjfv $m.tar.bz2 $m
+          for m in "${models[@]}"; do
+            mv -v "scripts/nemo/parakeet-tdt_ctc-0.6b-ja/$m" .
+            tar cjfv "$m.tar.bz2" "$m"
           done
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
run: |
models=(
sherpa-onnx-nemo-parakeet-tdt_ctc-0.6b-ja-35000-int8
)
for m in ${models[@]}; do
mv -v scripts/nemo/parakeet-tdt_ctc-0.6b-ja/$m .
tar cjfv $m.tar.bz2 $m
done
run: |
models=(
sherpa-onnx-nemo-parakeet-tdt_ctc-0.6b-ja-35000-int8
)
for m in "${models[@]}"; do
mv -v "scripts/nemo/parakeet-tdt_ctc-0.6b-ja/$m" .
tar cjfv "$m.tar.bz2" "$m"
done
🧰 Tools
🪛 actionlint (1.7.7)

54-54: shellcheck reported issue in this script: SC2068:error:4:10: Double quote array expansions to avoid re-splitting elements

(shellcheck)

54-54: shellcheck reported issue in this script: SC2086:info:5:47: Double quote to prevent globbing and word splitting

(shellcheck)

54-54: shellcheck reported issue in this script: SC2086:info:6:12: Double quote to prevent globbing and word splitting

(shellcheck)

54-54: shellcheck reported issue in this script: SC2086:info:6:23: Double quote to prevent globbing and word splitting

(shellcheck)

🤖 Prompt for AI Agents 
In .github/workflows/export-nemo-parakeet-tdt-tdt.yaml around lines 54 to 61,
the shell script has quoting issues that can cause word splitting or globbing
problems. Fix this by quoting the array expansion as "${models[@]}" in the for
loop and quoting variable expansions like "$m" in the mv and tar commands to
ensure proper handling of filenames and prevent errors.



- name: Publish to huggingface
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
uses: nick-fields/retry@v3
with:
max_attempts: 20
timeout_seconds: 200
shell: bash
command: |
git config --global user.email "csukuangfj@gmail.com"
git config --global user.name "Fangjun Kuang"

models=(
sherpa-onnx-nemo-parakeet-tdt_ctc-0.6b-ja-35000-int8
)

for m in ${models[@]}; do
rm -rf huggingface
export GIT_LFS_SKIP_SMUDGE=1
export GIT_CLONE_PROTECTION_ACTIVE=false
git clone https://csukuangfj:$HF_TOKEN@huggingface.co/csukuangfj/$m huggingface
cp -av $m/* huggingface
cd huggingface
git lfs track "*.onnx"
git lfs track "*.wav"
git status
git add .
git status
git commit -m "first commit"
git push https://csukuangfj:$HF_TOKEN@huggingface.co/csukuangfj/$m main
cd ..
done

- name: Release
uses: svenstaro/upload-release-action@v2
with:
file_glob: true
file: ./*.tar.bz2
overwrite: true
repo_name: k2-fsa/sherpa-onnx
repo_token: ${{ secrets.UPLOAD_GH_SHERPA_ONNX_TOKEN }}
tag: asr-models
32 changes: 32 additions & 0 deletions scripts/apk/generate-vad-asr-apk-script.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -565,6 +565,38 @@ def get_models():

ls -lh

popd
""",
),
Model(
model_name="sherpa-onnx-nemo-parakeet_tdt_ctc_110m-en-36000-int8",
idx=33,
lang="en",
lang2="English",
short_name="parakeet_tdt_ctc_110m",
cmd="""
pushd $model_name

rm -rfv test_wavs

ls -lh

popd
""",
),
Model(
model_name="sherpa-onnx-nemo-parakeet-tdt_ctc-0.6b-ja-35000-int8",
idx=34,
lang="ja",
lang2="Japanese",
short_name="parakeet-tdt_ctc_0.6b_ja",
cmd="""
pushd $model_name

rm -rfv test_wavs

ls -lh

popd
""",
),
Expand Down
1 change: 1 addition & 0 deletions scripts/nemo/fast-conformer-hybrid-transducer-ctc/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,5 +23,6 @@ This folder contains scripts for exporting models from
- https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nemo/models/stt_multilingual_fastconformer_hybrid_large_pc

- https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nemo/models/parakeet-tdt_ctc-110m
- https://huggingface.co/nvidia/parakeet-tdt_ctc-0.6b-ja

@coderabbitai coderabbitai Bot Jul 9, 2025

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix markdown formatting issues.

The line has incorrect indentation and should format the URL properly according to markdown standards.

Apply this diff to fix the formatting:

-  - https://huggingface.co/nvidia/parakeet-tdt_ctc-0.6b-ja
+- https://huggingface.co/nvidia/parakeet-tdt_ctc-0.6b-ja

Committable suggestion skipped: line range outside the PR's diff.

🧰 Tools
🪛 LanguageTool

[grammar] ~26-~26: Use correct spacing
Context: ...s/nemo/models/parakeet-tdt_ctc-110m - https://huggingface.co/nvidia/parakeet-tdt_ctc-0.6b-ja to sherpa-onnx.

(QB_NEW_EN_OTHER_ERROR_IDS_5)

🪛 markdownlint-cli2 (0.17.2)

26-26: Unordered list indentation
Expected: 0; Actual: 2

(MD007, ul-indent)

26-26: Bare URL used

(MD034, no-bare-urls)

🤖 Prompt for AI Agents 
In scripts/nemo/fast-conformer-hybrid-transducer-ctc/README.md at line 26, the
URL is not properly formatted in markdown due to incorrect indentation. Fix this
by removing any leading spaces before the URL and ensure it is either on its own
line or formatted as a proper markdown link so it renders correctly.


to `sherpa-onnx`.
7 changes: 7 additions & 0 deletions scripts/nemo/fast-conformer-hybrid-transducer-ctc/export-onnx-ctc-non-streaming.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
import nemo.collections.asr as nemo_asr
import onnx
import torch
from onnxruntime.quantization import QuantType, quantize_dynamic


def get_args():
Expand Down Expand Up @@ -86,6 +87,12 @@ def main():
}
add_meta_data(filename, meta_data)

quantize_dynamic(
model_input="./model.onnx",
model_output="./model.int8.onnx",
weight_type=QuantType.QUInt8,
)

print("preprocessor", asr_model.cfg.preprocessor)
print(meta_data)

Expand Down
6 changes: 6 additions & 0 deletions scripts/nemo/fast-conformer-hybrid-transducer-ctc/export-onnx-ctc.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
import nemo.collections.asr as nemo_asr
import onnx
import torch
from onnxruntime.quantization import QuantType, quantize_dynamic


def get_args():
Expand Down Expand Up @@ -114,6 +115,11 @@ def main():
"comment": "Only the CTC branch is exported",
}
add_meta_data(filename, meta_data)
quantize_dynamic(
model_input="./model.onnx",
model_output="./model.int8.onnx",
weight_type=QuantType.QUInt8,
)

print(meta_data)

Expand Down
8 changes: 8 additions & 0 deletions scripts/nemo/fast-conformer-hybrid-transducer-ctc/export-onnx-transducer-non-streaming.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
import nemo.collections.asr as nemo_asr
import onnx
import torch
from onnxruntime.quantization import QuantType, quantize_dynamic


def get_args():
Expand Down Expand Up @@ -90,6 +91,13 @@ def main():
}
add_meta_data("encoder.onnx", meta_data)

for m in ["encoder", "decoder", "joiner"]:
quantize_dynamic(
model_input=f"{m}.onnx",
model_output=f"{m}.int8.onnx",
weight_type=QuantType.QUInt8,
)

print(meta_data)


Expand Down
Loading