macbookpro m1量化baichuan2-13b失败

[mairongchao]

命令：python3 chatglm_cpp/convert.py -i baichuan-inc/Baichuan2-13B-Chat -t q4_0 -o baichuan-ggml.bin
环境：

packages in environment at /opt/homebrew/Caskroom/miniconda/base/envs/pytorch:

Name Version Build Channel

accelerate 0.24.1 pyhd8ed1ab_0 conda-forge
aiofiles 22.1.0 py312hca03da5_0
aiohttp 3.9.0 py312he37b823_0 conda-forge
aiosignal 1.2.0 pyhd3eb1b0_0
altair 5.0.1 py312hca03da5_0
anyio 3.7.1 pyhd8ed1ab_0 conda-forge
appnope 0.1.3 py312hca03da5_1001
argon2-cffi 21.3.0 pyhd3eb1b0_0
argon2-cffi-bindings 21.2.0 py312h80987f9_0
asttokens 2.0.5 pyhd3eb1b0_0
attrs 23.1.0 py312hca03da5_0
aws-c-auth 0.7.7 h886c30d_1 conda-forge
aws-c-cal 0.6.9 hea61927_1 conda-forge
aws-c-common 0.9.8 h93a5062_0 conda-forge
aws-c-compression 0.2.17 hea61927_6 conda-forge
aws-c-event-stream 0.3.2 h0574dc0_7 conda-forge
aws-c-http 0.7.14 h90b1786_2 conda-forge
aws-c-io 0.13.36 he1b4ce3_0 conda-forge
aws-c-mqtt 0.9.10 h8d54690_1 conda-forge
aws-c-s3 0.4.1 ha5b923c_0 conda-forge
aws-c-sdkutils 0.1.12 hea61927_5 conda-forge
aws-checksums 0.1.17 hea61927_5 conda-forge
aws-crt-cpp 0.24.7 hba4ac3b_6 conda-forge
aws-sdk-cpp 1.11.182 h31542fa_7 conda-forge
backcall 0.2.0 pyhd3eb1b0_0
beautifulsoup4 4.12.2 py312hca03da5_0
bitsandbytes 0.41.2.post2 pypi_0 pypi
blas 1.0 openblas
bleach 4.1.0 pyhd3eb1b0_0
blinker 1.6.2 py312hca03da5_0
bottleneck 1.3.5 py312ha86b861_0
brotli 1.0.7 hc377ac9_0
brotli-python 1.0.9 py312h313beb8_7
bzip2 1.0.8 h93a5062_5 conda-forge
c-ares 1.22.1 h93a5062_0 conda-forge
ca-certificates 2023.11.17 hf0a4a13_0 conda-forge
cachetools 4.2.2 pyhd3eb1b0_0
certifi 2023.7.22 py312hca03da5_1
cffi 1.16.0 py312h80987f9_0
charset-normalizer 2.0.4 pyhd3eb1b0_0
chatglm-cpp 0.3.0 pypi_0 pypi
click 8.1.7 py312hca03da5_0
colorama 0.4.6 py312hca03da5_0
comm 0.1.2 py312hca03da5_0
contourpy 1.0.5 py312h48ca7d4_0
cryptography 41.0.3 py312hd4332d6_0
cycler 0.11.0 pyhd3eb1b0_0
dataclasses 0.8 pyh6d0b6a4_7
datasets 2.14.7 pyhd8ed1ab_0 conda-forge
debugpy 1.6.7 py312h313beb8_0
decorator 5.1.1 pyhd3eb1b0_0
defusedxml 0.7.1 pyhd3eb1b0_0
dill 0.3.7 py312hca03da5_0
entrypoints 0.4 py312hca03da5_0
exceptiongroup 1.0.4 py312hca03da5_0
executing 0.8.3 pyhd3eb1b0_0
fastapi 0.104.1 pyhd8ed1ab_0 conda-forge
ffmpeg 4.2.2 h04105a8_0
ffmpy 0.3.0 pyhb6f538c_0 conda-forge
filelock 3.9.0 py312hca03da5_0
fonttools 4.25.0 pyhd3eb1b0_0
freetype 2.12.1 h1192e45_0
frozenlist 1.4.0 py312h80987f9_0
fsspec 2023.9.2 py312hca03da5_0
gettext 0.21.1 h0186832_0 conda-forge
gflags 2.2.2 hc377ac9_0
giflib 5.2.1 h80987f9_3
gitdb 4.0.7 pyhd3eb1b0_0
gitpython 3.1.37 py312hca03da5_0
glog 0.6.0 h6da1cb0_0 conda-forge
gmp 6.2.1 hc377ac9_3
gnutls 3.6.15 h887c41c_0
gradio 3.50.2 pyhd8ed1ab_0 conda-forge
gradio-client 0.6.1 pyhd8ed1ab_0 conda-forge
h11 0.12.0 pyhd3eb1b0_0
h2 4.0.0 py312hca03da5_3
hpack 4.0.0 py_0
httpcore 0.15.0 py312hca03da5_0
httpx 0.23.0 py312hca03da5_0
huggingface_hub 0.17.3 py312hca03da5_0
hyperframe 6.0.1 pyhd3eb1b0_0
icu 73.2 hc8870d7_0 conda-forge
idna 3.4 py312hca03da5_0
importlib-metadata 6.0.0 py312hca03da5_0
importlib_metadata 6.0.0 hd3eb1b0_0
importlib_resources 5.2.0 pyhd3eb1b0_1
ipykernel 6.26.0 pyh3cd1d5f_0 conda-forge
ipython 8.15.0 py312hca03da5_0
ipython_genutils 0.2.0 pyhd3eb1b0_1
ipywidgets 7.6.5 pyhd3eb1b0_1
jedi 0.18.1 py312hca03da5_1
jinja2 3.1.2 py312hca03da5_0
joblib 1.2.0 py312hca03da5_0
jpeg 9e h80987f9_1
jsonschema 4.17.3 py312hca03da5_0
jupyter_client 7.1.2 pyhd3eb1b0_0
jupyter_core 5.5.0 py312h81bd7bf_0 conda-forge
jupyter_server 1.13.5 pyhd3eb1b0_0
jupyterlab_pygments 0.1.2 py_0
jupyterlab_widgets 3.0.5 py312hca03da5_0
kiwisolver 1.4.4 py312h313beb8_0
krb5 1.20.1 hf3e1bf2_1
lame 3.100 h1a28f6b_0
latex2mathml 3.76.0 pypi_0 pypi
lcms2 2.12 hba8e193_0
lerc 3.0 hc377ac9_0
libabseil 20230802.1 cxx17_h13dd4ca_0 conda-forge
libarrow 14.0.1 h63c4d5d_4_cpu conda-forge
libarrow-acero 14.0.1 had9dd58_4_cpu conda-forge
libarrow-dataset 14.0.1 had9dd58_4_cpu conda-forge
libarrow-flight 14.0.1 h1011bfc_4_cpu conda-forge
libarrow-flight-sql 14.0.1 h660fe36_4_cpu conda-forge
libarrow-gandiva 14.0.1 h2b96968_4_cpu conda-forge
libarrow-substrait 14.0.1 h594d712_4_cpu conda-forge
libblas 3.9.0 1_h2ec9a88_netlib conda-forge
libbrotlicommon 1.1.0 hb547adb_1 conda-forge
libbrotlidec 1.1.0 hb547adb_1 conda-forge
libbrotlienc 1.1.0 hb547adb_1 conda-forge
libcblas 3.9.0 1_h9886b1c_netlib
libcrc32c 1.1.2 hc377ac9_0
libcurl 8.4.0 h3e2b118_0
libcxx 16.0.6 h4653b0c_0 conda-forge
libdeflate 1.8 h1a28f6b_5
libedit 3.1.20221030 h80987f9_0
libev 4.33 h1a28f6b_1
libevent 2.1.12 h02f6b3c_1
libexpat 2.5.0 hb7217d7_1 conda-forge
libffi 3.4.2 h3422bc3_5 conda-forge
libgfortran 5.0.0 11_3_0_hca03da5_28
libgfortran5 11.3.0 h009349e_28
libgoogle-cloud 2.12.0 hfb399a7_4 conda-forge
libgrpc 1.59.3 hbcf6334_0 conda-forge
libiconv 1.17 he4db4b2_0 conda-forge
libidn2 2.3.4 h80987f9_0
liblapack 3.9.0 1_h9886b1c_netlib
libllvm15 15.0.7 h504e6bf_3 conda-forge
libnghttp2 1.57.0 h62f6fdd_0
libopenblas 0.3.21 h269037a_0
libopus 1.3 h1a28f6b_1
libparquet 14.0.1 heaab74a_4_cpu conda-forge
libpng 1.6.39 h80987f9_0
libprotobuf 4.24.4 hc9861d8_0 conda-forge
libre2-11 2023.06.02 h1753957_0 conda-forge
libsodium 1.0.18 h1a28f6b_0
libsqlite 3.44.2 h091b4b1_0 conda-forge
libssh2 1.10.0 h02f6b3c_2
libtasn1 4.19.0 h80987f9_0
libthrift 0.19.0 h026a170_1 conda-forge
libtiff 4.4.0 h2fd578a_2
libunistring 0.9.10 h1a28f6b_0
libutf8proc 2.8.0 h1a8c8d9_0 conda-forge
libuv 1.46.0 hb547adb_0 conda-forge
libvpx 1.10.0 hc377ac9_0
libwebp 1.3.2 ha3663a8_0
libwebp-base 1.3.2 h80987f9_0
libxml2 2.11.6 h0d0cfa8_0 conda-forge
libzlib 1.2.13 h53f4e23_5 conda-forge
llvm-openmp 17.0.5 hcd81f8e_0 conda-forge
loguru 0.7.2 py312h81bd7bf_1 conda-forge
lz4-c 1.9.4 h313beb8_0
markdown 3.5.1 pypi_0 pypi
markdown-it-py 2.2.0 py312hca03da5_1
markupsafe 2.1.1 py312h80987f9_0
matplotlib-base 3.8.0 py312hd77ebd4_0
matplotlib-inline 0.1.6 py312hca03da5_0
mdtex2html 1.2.0 pypi_0 pypi
mdurl 0.1.0 py312hca03da5_0
mistune 2.0.4 py312hca03da5_0
mpmath 1.3.0 py312hca03da5_0
multidict 6.0.4 py312h670c8ac_1 conda-forge
multiprocess 0.70.15 py312hca03da5_0
munkres 1.1.4 py_0
nbclassic 1.0.0 pyhb4ecaf3_1 conda-forge
nbclient 0.5.11 pyhd3eb1b0_0
nbconvert 7.11.0 pyhd8ed1ab_0 conda-forge
nbconvert-core 7.11.0 pyhd8ed1ab_0 conda-forge
nbconvert-pandoc 7.11.0 pyhd8ed1ab_0 conda-forge
nbformat 5.1.3 pyhd3eb1b0_0
ncurses 6.4 h463b476_2 conda-forge
nest-asyncio 1.5.6 py312hca03da5_0
nettle 3.7.3 h84b5d62_1
networkx 3.1 py312hca03da5_0
nomkl 3.0 0
notebook 6.5.4 pyha770c72_0 conda-forge
notebook-shim 0.2.3 pyhd8ed1ab_0 conda-forge
numexpr 2.8.7 py312h0f3ea24_0
numpy 1.26.0 py312h7f4fdc5_0
numpy-base 1.26.0 py312he047099_0
openh264 1.8.0 h98b2900_0
openjpeg 2.3.0 h7a6adac_2
openssl 3.2.0 h0d3ecfb_0 conda-forge
orc 1.9.2 h7c018df_0 conda-forge
orjson 3.9.10 py312h37b1423_0 conda-forge
packaging 23.1 py312hca03da5_0
pandas 2.1.1 py312hd77ebd4_0
pandoc 3.1.3 hce30654_0 conda-forge
pandocfilters 1.5.0 pyhd3eb1b0_0
parso 0.8.3 pyhd3eb1b0_0
pexpect 4.8.0 pyhd3eb1b0_3
pickleshare 0.7.5 pyhd3eb1b0_1003
pillow 10.0.1 py312h3b245a6_0
pip 23.3.1 pyhd8ed1ab_0 conda-forge
platformdirs 4.0.0 pyhd8ed1ab_0 conda-forge
prometheus_client 0.14.1 py312hca03da5_0
prompt-toolkit 3.0.36 py312hca03da5_0
protobuf 4.24.4 py312h71bcfcd_0 conda-forge
psutil 5.9.0 py312h80987f9_0
ptyprocess 0.7.0 pyhd3eb1b0_2
pure_eval 0.2.2 pyhd3eb1b0_0
pyarrow 14.0.1 py312h01ffab7_4_cpu conda-forge
pyarrow-hotfix 0.6 pyhd8ed1ab_0 conda-forge
pycparser 2.21 pyhd3eb1b0_0
pydantic 1.10.12 py312h80987f9_1
pydeck 0.8.0 pyhd8ed1ab_0 conda-forge
pydub 0.25.1 pyhd8ed1ab_0 conda-forge
pygments 2.15.1 py312hca03da5_1
pyopenssl 23.2.0 py312hca03da5_0
pyparsing 3.0.9 py312hca03da5_0
pyrsistent 0.18.0 py312h80987f9_0
pysocks 1.7.1 py312hca03da5_0
python 3.12.0 h47c9636_0_cpython conda-forge
python-dateutil 2.8.2 pyhd3eb1b0_0
python-multipart 0.0.6 py312hca03da5_0
python-tzdata 2023.3 pyhd3eb1b0_0
python-xxhash 2.0.2 py312h80987f9_1
python_abi 3.12 4_cp312 conda-forge
pytorch 2.1.0 cpu_generic_py312h28b293b_0 conda-forge
pytz 2023.3.post1 py312hca03da5_0
pyyaml 6.0.1 py312h80987f9_0
pyzmq 25.1.0 py312h313beb8_0
re2 2023.06.02 h6135d0a_0 conda-forge
readline 8.2 h92ec313_1 conda-forge
regex 2023.10.3 py312h80987f9_0
requests 2.31.0 py312hca03da5_0
rfc3986 1.4.0 pyhd3eb1b0_0
rich 13.3.5 py312hca03da5_1
sacremoses 0.0.43 pyhd3eb1b0_0
safetensors 0.3.3 py312h0002256_1 conda-forge
semantic_version 2.8.5 pyhd3eb1b0_0
send2trash 1.8.0 pyhd3eb1b0_1
sentencepiece 0.1.99 py312h48ca7d4_0
setuptools 68.2.2 pyhd8ed1ab_0 conda-forge
six 1.16.0 pyhd3eb1b0_1
sleef 3.5.1 h80987f9_2
smmap 4.0.0 pyhd3eb1b0_0
snappy 1.1.10 h17c5cce_0 conda-forge
sniffio 1.2.0 py312hca03da5_1
soupsieve 2.5 py312hca03da5_0
sse-starlette 1.6.5 pyhd8ed1ab_0 conda-forge
stack_data 0.2.0 pyhd3eb1b0_0
starlette 0.27.0 pyhd8ed1ab_0 conda-forge
streamlit 1.28.2 pyhd8ed1ab_0 conda-forge
sympy 1.12 pyh04b8f61_3 conda-forge
tabulate 0.9.0 pypi_0 pypi
tenacity 8.2.2 py312hca03da5_1
terminado 0.17.1 py312hca03da5_0
tinycss2 1.2.1 py312hca03da5_0
tk 8.6.13 h5083fa2_1 conda-forge
tokenizers 0.15.0 pypi_0 pypi
toml 0.10.2 pyhd3eb1b0_0
toolz 0.12.0 py312hca03da5_0
tornado 6.3.3 py312h80987f9_0
tqdm 4.63.0 pyhd3eb1b0_0
traitlets 5.7.1 py312hca03da5_0
transformers 4.35.2 pypi_0 pypi
typing-extensions 4.8.0 hd8ed1ab_0 conda-forge
typing_extensions 4.8.0 pyha770c72_0 conda-forge
tzdata 2023c h71feb2d_0 conda-forge
tzlocal 2.1 py312hca03da5_1
urllib3 1.26.18 py312hca03da5_0
uvicorn 0.24.0 py312h81bd7bf_0 conda-forge
validators 0.18.2 pyhd3eb1b0_0
watchdog 2.1.6 py312h80987f9_0
wcwidth 0.2.5 pyhd3eb1b0_0
webencodings 0.5.1 py312hca03da5_2
websocket-client 0.58.0 py312hca03da5_4
websockets 10.4 py312h80987f9_1
wheel 0.41.3 pyhd8ed1ab_0 conda-forge
widgetsnbextension 3.5.1 pyh9f0ad1d_3 conda-forge
x264 1!152.20180806 h1a28f6b_0
xxhash 0.8.0 h1a28f6b_3
xz 5.2.6 h57fd34a_0 conda-forge
yaml 0.2.5 h1a28f6b_0
yarl 1.9.2 py312he37b823_1 conda-forge
zeromq 4.3.4 hc377ac9_0
zipp 3.11.0 py312hca03da5_0
zlib 1.2.13 h53f4e23_5 conda-forge
zstd 1.5.5 h4f39d0f_0 conda-forge
报错：
(pytorch) mrc@mrcdeMacBook-Pro chatglm.cpp % python3 chatglm_cpp/convert.py -i baichuan-inc/Baichuan2-13B-Chat -t q4_0 -o baichuan-ggml.bin

Traceback (most recent call last):
File "/Volumes/mydisk/model/chatglm.cpp/chatglm_cpp/convert.py", line 543, in
main()
File "/Volumes/mydisk/model/chatglm.cpp/chatglm_cpp/convert.py", line 537, in main
convert(f, args.model_name_or_path, dtype=args.type)
File "/Volumes/mydisk/model/chatglm.cpp/chatglm_cpp/convert.py", line 469, in convert
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, trust_remote_code=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Caskroom/miniconda/base/envs/pytorch/lib/python3.12/site-packages/transformers/models/auto/tokenization_auto.py", line 755, in from_pretrained
return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Caskroom/miniconda/base/envs/pytorch/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 2024, in from_pretrained
return cls._from_pretrained(
^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Caskroom/miniconda/base/envs/pytorch/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 2256, in _from_pretrained
tokenizer = cls(*init_inputs, **init_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mrc/.cache/huggingface/modules/transformers_modules/baichuan-inc/Baichuan2-13B-Chat/8f6e343d545c503b91429582231d1d354dac2740/tokenization_baichuan.py", line 71, in init
super().init(
File "/opt/homebrew/Caskroom/miniconda/base/envs/pytorch/lib/python3.12/site-packages/transformers/tokenization_utils.py", line 367, in init
self._add_tokens(
File "/opt/homebrew/Caskroom/miniconda/base/envs/pytorch/lib/python3.12/site-packages/transformers/tokenization_utils.py", line 467, in _add_tokens
current_vocab = self.get_vocab().copy()
^^^^^^^^^^^^^^^^
File "/Users/mrc/.cache/huggingface/modules/transformers_modules/baichuan-inc/Baichuan2-13B-Chat/8f6e343d545c503b91429582231d1d354dac2740/tokenization_baichuan.py", line 105, in get_vocab
vocab = {self.convert_ids_to_tokens(i): i for i in range(self.vocab_size)}
^^^^^^^^^^^^^^^
File "/Users/mrc/.cache/huggingface/modules/transformers_modules/baichuan-inc/Baichuan2-13B-Chat/8f6e343d545c503b91429582231d1d354dac2740/tokenization_baichuan.py", line 101, in vocab_size
return self.sp_model.get_piece_size()
^^^^^^^^^^^^^
AttributeError: 'BaichuanTokenizer' object has no attribute 'sp_model'

[li-plus]

最近上游 transformers 更新有不兼容的改动，可以降级到 4.33 试试：

pip install transformers~=4.33.0

[mairongchao]

pip install transformers~=4.33.0

pip install bitsandbytes
pip install scipy
运行上面命令，已经解决问题