common : compute average token length from vocabulary#17632

yifant-code wants to merge 1 commit intoggml-org:masterggml-org/llama.cpp:masterfrom

yifant-code:fix/compute-avg-token-lengthyifant-code/llama.cpp:fix/compute-avg-token-lengthCopy head branch name to clipboard