py : handle byte tokens in get_token_type

[ggerganov]

Continuation of #5084

Fix #5064

@felladrin Could you give this branch another test and see if it works?

[felladrin]

Alright, I'll be free to do the test in a few hours.

ℹ️ Info in case someone wants to test it before me: I'll be using the same model as before (Locutusque/TinyMistral-248M-v2.5). The reason for choosing this is that it's a small Mistral model that uses the Llama2 tokenizer and which repository doesn't have the tokenizer.model & added_tokens.json files.

[felladrin]

Just tested it and it's working great! 🎉

Command:

llama.cpp/main -m model/ggml-model-f32.gguf -p "Building a website can be done in 10 simple steps:\nStep 1:" -n 64 -e --repeat_penalty 1.1 -c 0 --temp 0

Output (without the changes from this PR):

Building a website can be done in 10 simple steps:<0x0A>Step 1: Create a new website<0x0A>- Start by creating a new website.<0x0A>- Add the necessary HTML and CSS to style the website.<0x0A>- Use CSS to style the website, including changing the font, font, and color scheme.<0x0A>- Choose a webpage with the following links for the website you want to create a

Output (with the changes from this PR):

Building a website can be done in 10 simple steps:
Step 1: Create a new website
- Start by creating a new website.
- Add the necessary HTML and CSS to style the website.
- Use CSS to style the website, including changing the font, font, and color scheme.
- Choose a webpage with the following links for the website you want to create a