Code

Fix spm whitespaces#2806

Code

Merged

klosax merged 3 commits intoggerganov:masterggerganov/llama.cpp:masterfrom klosax:fix-spm-whitespacesklosax/llama.cpp:fix-spm-whitespacesCopy head branch name to clipboardAug 26, 2023

+27-41

Commits

Commits on Aug 26, 2023

• llama.cpp : fix spm whitespace escaping + clean up

  klosax

  authored
  d528946

• main.cpp : spm - add whitespace in front of prompt

  klosax

  authored
  43f7c16

• test-tokenizer-0.cpp : spm - add whitespace in front of prompt

  klosax

  authored
  c50b1ae