OSError: Can't load tokenizer for 'Neko-Institute-of-Science/LLaMA-7B-HF'.

[Chip1ne]

Hi!

I tried to train TWOSOME in Tomato Salad environment sh scripts/tomato_salad_ppo_llm.sh and encountered the following error:

Traceback (most recent call last):
File "/home/chip1ne/TWOSOME/twosome/overcooked/ppo_llm_pomdp.py", line 192, in
agent = LLMAgent(normalization_mode=args.normalization_mode, load_8bit=args.load_8bit, task=args.task)
File "/home/chip1ne/TWOSOME/twosome/overcooked/policy_pomdp.py", line 68, in init
self.tokenizer = LlamaTokenizer.from_pretrained(self.base_model)
File "/home/chip1ne/anaconda3/envs/twosome/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 2192, in from_pretrained
raise EnvironmentError(
OSError: Can't load tokenizer for 'Neko-Institute-of-Science/LLaMA-7B-HF'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'Neko-Institute-of-Science/LLaMA-7B-HF' is the correct path to a directory containing all relevant files for a LlamaTokenizer tokenizer.

It seems that there were some issues loading the tokenizer. I'm not sure how to deal with this problem.

[WeihaoTan]

Thanks for reaching out. It seems that you didn't download the tokenizer successfully. Maybe you need to delete the model files and download again. Or you can directly download the model from https://huggingface.co/Neko-Institute-of-Science/LLaMA-7B-HF.