README Llama2 Example Errors Requesting HuggingFace API Key

[collinarnett]

Describe the bug
When initializing the client using llama2 as the big_model argument as specified in the "Running LLM's Locally" section of the README HuggingFace throws:

Repo model meta-llama/Llama-2-7b-hf is gated. You must be authenticated to access it.

To Reproduce
Steps to reproduce the behavior:

1. Install LLM-VM
2. Running

# import our client
from llm_vm.client import Client

# Select the LlaMA 2 model
client = Client(big_model = 'llama2')

Expected behavior
Model is downloaded and client is initialized.

Screenshots

>>> client = Client(big_model = 'llama2')
Traceback (most recent call last):
  File "/nix/store/ryhyfwvxiws6jd444kpm3zxr31yrz75c-python3-3.10.13-env/lib/python3.10/site-packages/huggingface_hub/utils/_errors.py", line 269, in hf_raise_for_status
    response.raise_for_status()
  File "/nix/store/ryhyfwvxiws6jd444kpm3zxr31yrz75c-python3-3.10.13-env/lib/python3.10/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/meta-llama/Llama-2-7b-hf/resolve/main/config.json

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/nix/store/ryhyfwvxiws6jd444kpm3zxr31yrz75c-python3-3.10.13-env/lib/python3.10/site-packages/transformers/utils/hub.py", line 430, in cached_file
    resolved_file = hf_hub_download(
  File "/nix/store/ryhyfwvxiws6jd444kpm3zxr31yrz75c-python3-3.10.13-env/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "/nix/store/ryhyfwvxiws6jd444kpm3zxr31yrz75c-python3-3.10.13-env/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1360, in hf_hub_download
    raise head_call_error
  File "/nix/store/ryhyfwvxiws6jd444kpm3zxr31yrz75c-python3-3.10.13-env/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1233, in hf_hub_download
    metadata = get_hf_file_metadata(
  File "/nix/store/ryhyfwvxiws6jd444kpm3zxr31yrz75c-python3-3.10.13-env/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "/nix/store/ryhyfwvxiws6jd444kpm3zxr31yrz75c-python3-3.10.13-env/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1622, in get_hf_file_metadata
    hf_raise_for_status(r)
  File "/nix/store/ryhyfwvxiws6jd444kpm3zxr31yrz75c-python3-3.10.13-env/lib/python3.10/site-packages/huggingface_hub/utils/_errors.py", line 285, in hf_raise_for_status
    raise GatedRepoError(message, response) from e
huggingface_hub.utils._errors.GatedRepoError: 401 Client Error. (Request ID: Root=1-65846a42-5d524e591835b68578f43fec;8085e1d7-664a-4f33-be03-53c8a5fc43d5)

Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-2-7b-hf/resolve/main/config.json.
Repo model meta-llama/Llama-2-7b-hf is gated. You must be authenticated to access it.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/nix/store/ryhyfwvxiws6jd444kpm3zxr31yrz75c-python3-3.10.13-env/lib/python3.10/site-packages/llm_vm/client.py", line 70, in __init__
    self.teacher = load_model_closure(big_model)(**big_model_config)
  File "/nix/store/ryhyfwvxiws6jd444kpm3zxr31yrz75c-python3-3.10.13-env/lib/python3.10/site-packages/llm_vm/onsite_llm.py", line 99, in __init__
    self.model=self.model_loader(**model_kw_args)
  File "/nix/store/ryhyfwvxiws6jd444kpm3zxr31yrz75c-python3-3.10.13-env/lib/python3.10/site-packages/llm_vm/onsite_llm.py", line 607, in model_loader
    return LlamaForCausalLM.from_pretrained(self.model_uri)
  File "/nix/store/ryhyfwvxiws6jd444kpm3zxr31yrz75c-python3-3.10.13-env/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2600, in from_pretrained
    resolved_config_file = cached_file(
  File "/nix/store/ryhyfwvxiws6jd444kpm3zxr31yrz75c-python3-3.10.13-env/lib/python3.10/site-packages/transformers/utils/hub.py", line 445, in cached_file
    raise EnvironmentError(
OSError: You are trying to access a gated repo.
Make sure to request access at https://huggingface.co/meta-llama/Llama-2-7b-hf and pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=<your_token>`.

Desktop (please complete the following information):

• system: "x86_64-linux"
• host os: Linux 6.1.65, NixOS, 24.05 (Uakari), 24.05.20231204.2c7f3c0
• multi-user?: yes
• sandbox: yes
• version: nix-env (Nix) 2.19.2
• channels(collin): ""
• channels(root): "nixos-21.11.335130.386234e2a61"
• nixpkgs: /nix/store/aiv01710wqn2b7hms2253d1cq89kdzh8-source

Additional context
Add any other context about the problem here.

Suggested Fix
Add a HuggingFace token argument similar to the OpenAI API key.

[wansatya]

Maybe this can help?

#429 (comment)