Llama 3.2 conversion error - onnx.ModelProto exceeds maximum protobuf size of 2GB: 4943971428

[jonas-elias]

System Info

requirements file:

transformers[torch]==4.44.2
onnxruntime<1.16.0
optimum==1.16.2
tqdm
onnx==1.17.0

Environment/Platform

• Website/web-app
• Browser extension
• Server-side (e.g., Node.js, Deno, Bun)
• Desktop app (e.g., Electron)
• Other (e.g., VSCode extension)

Description

I cannot convert model llama-3.2 by NousResearch/Llama-3.2-1B using the script convert.py of project xenova/transformers.js.

error:

Saving external data to one file...
Post-processing the exported models...
Deduplicating shared (tied) weights...
Found different candidate ONNX initializers (likely duplicate) for the tied weights:
        lm_head.weight: {'onnx::MatMul_5035'}
        model.embed_tokens.weight: {'model.embed_tokens.weight'}
Removing duplicate initializer onnx::MatMul_5035...
Traceback (most recent call last):
  File "C:\Users\jonas\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\optimum\exporters\onnx\__main__.py", line 577, in main_export
    models_and_onnx_configs, onnx_files_subpaths = onnx_config.post_process_exported_models(
  File "C:\Users\jonas\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\optimum\exporters\onnx\config.py", line 126, in post_process_exported_models
    models_and_onnx_configs, onnx_files_subpaths = super().post_process_exported_models(
  File "C:\Users\jonas\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\optimum\exporters\onnx\base.py", line 546, in post_process_exported_models
    remove_duplicate_weights_from_tied_info(
  File "C:\Users\jonas\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\optimum\onnx\graph_transformations.py", line 100, in remove_duplicate_weights_from_tied_info
    check_and_save_model(onnx_model, save_path=save_path)
  File "C:\Users\jonas\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\optimum\onnx\graph_transformations.py", line 146, in check_and_save_model
    raise e
  File "C:\Users\jonas\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\optimum\onnx\graph_transformations.py", line 141, in check_and_save_model
    onnx.checker.check_model(model)
  File "C:\Users\jonas\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\onnx\checker.py", line 171, in check_model
    model if isinstance(model, bytes) else model.SerializeToString()
ValueError: Message onnx.ModelProto exceeds maximum protobuf size of 2GB: 4943971428

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\projects\models\transformers.js\scripts\convert.py", line 546, in <module>
    main()
  File "C:\projects\models\transformers.js\scripts\convert.py", line 449, in main
    main_export(**export_kwargs)
  File "C:\Users\jonas\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\optimum\exporters\onnx\__main__.py", line 581, in main_export
    raise Exception(
Exception: The post-processing of the ONNX export failed. The export can still be performed by passing the option --no-post-process. Detailed error: Message onnx.ModelProto exceeds maximum protobuf size of 2GB: 4943971428

Reproduction

Run this command:

python -m scripts.convert --quantize --model_id NousResearch/Llama-3.2-1B

[xenova]

Hi there 👋 Can you try export with the v3 version of the script?
https://github.com/xenova/transformers.js/blob/v3/scripts/convert.py