Add CLI mode with tests

[pseudotensor]

(h2ollm) jon@pseudotensor:~/h2o-llm$ python generate.py --base_model=h2oai/h2ogpt-oig-oasst1-512-6_9b --cli=True

===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes

 and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================
bin /home/jon/miniconda3/envs/h2ollm/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda121.so
/home/jon/miniconda3/envs/h2ollm/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: /home/jon/miniconda3/envs/h2ollm did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...
  warn(msg)
/home/jon/miniconda3/envs/h2ollm/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/opt/clang+llvm-4.0.0-x86_64-linux-gnu-ubuntu-16.04/lib'), PosixPath('/opt/rstudio-1.0.136/bin'), PosixPath('/usr/local/cuda/extras/CUPTI/lib64'), PosixPath('/usr/lib/jvm/default-java/jre/lib/amd64/server'), PosixPath('/home/jon/lib')}
  warn(msg)
CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so
CUDA SETUP: Highest compute capability among GPUs detected: 7.5
CUDA SETUP: Detected CUDA version 121
CUDA SETUP: Loading binary /home/jon/miniconda3/envs/h2ollm/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda121.so...
Using Model h2oai/h2ogpt-oig-oasst1-512-6_9b
device_map: {'': 0}
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:06<00:00,  2.10s/it]

Enter an instruction: Who are you?
Using pad_token, but it is not set yet.
Pre-Generate: 2023-05-26 18:49:36.334195
Generate: 2023-05-26 18:49:36.334220
Hi! I'm h2oGPT, a large language model by H2O.ai, the visionary leader in democratizing AI. How can I help you?Post-Generate: 2023-05-26 18:49:38.308654 decoded_output: 149

Enter an instruction: 

(h2ollm) jon@pseudotensor:~/h2o-llm$ python generate.py --base_model=gptj --cli=True --score_model=None

===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes

 and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================
bin /home/jon/miniconda3/envs/h2ollm/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda121.so
/home/jon/miniconda3/envs/h2ollm/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: /home/jon/miniconda3/envs/h2ollm did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...
  warn(msg)
/home/jon/miniconda3/envs/h2ollm/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/cuda/extras/CUPTI/lib64'), PosixPath('/opt/rstudio-1.0.136/bin'), PosixPath('/usr/lib/jvm/default-java/jre/lib/amd64/server'), PosixPath('/opt/clang+llvm-4.0.0-x86_64-linux-gnu-ubuntu-16.04/lib'), PosixPath('/home/jon/lib')}
  warn(msg)
CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so
CUDA SETUP: Highest compute capability among GPUs detected: 7.5
CUDA SETUP: Detected CUDA version 121
CUDA SETUP: Loading binary /home/jon/miniconda3/envs/h2ollm/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda121.so...
Using Model gptj
Found model file.
gptj_model_load: loading model from '/home/jon/.cache/gpt4all/ggml-gpt4all-j-v1.3-groovy.bin' - please wait ...
gptj_model_load: n_vocab = 50400
gptj_model_load: n_ctx   = 2048
gptj_model_load: n_embd  = 4096
gptj_model_load: n_head  = 16
gptj_model_load: n_layer = 28
gptj_model_load: n_rot   = 64
gptj_model_load: f16     = 2
gptj_model_load: ggml ctx size = 5401.45 MB
gptj_model_load: kv self size  =  896.00 MB
gptj_model_load: ................................... done
gptj_model_load: model size =  3609.38 MB / num tensors = 285

Enter an instruction: Who are you?
I am a computer program designed to assist users with various tasks. I can help them find information, answer questions, and perform calculations.

Enter an instruction: 

================================================================================================================ short test summary info ================================================================================================================
FAILED tests/test_eval.py::test_eval1[False-4] - TypeError: GPTNeoXForCausalLM.__init__() got an unexpected keyword argument 'load_in_4bit'
FAILED tests/test_manual_test.py::test_chat_context - NotImplementedError: MANUAL TEST FOR NOW
FAILED tests/test_manual_test.py::test_upload_one_file - NotImplementedError: MANUAL TEST FOR NOW -- do and ask query of file
FAILED tests/test_manual_test.py::test_upload_multiple_file - NotImplementedError: MANUAL TEST FOR NOW -- do and ask query of files
FAILED tests/test_manual_test.py::test_upload_url - NotImplementedError: MANUAL TEST FOR NOW -- put in URL box https://github.com/h2oai/h2ogpt/ (and ask what is h2ogpt?). Ensure can go to source links
FAILED tests/test_manual_test.py::test_upload_arxiv - NotImplementedError: MANUAL TEST FOR NOW -- paste in arxiv:1706.03762 and ask who wrote attention paper. Ensure can go to source links
FAILED tests/test_manual_test.py::test_upload_pasted_text - NotImplementedError: MANUAL TEST FOR NOW -- do and see test code for what to try
FAILED tests/test_manual_test.py::test_no_db_dirs - NotImplementedError: MANUAL TEST FOR NOW -- Remove db_dirs, ensure can still start up and use in MyData Mode.
FAILED tests/test_manual_test.py::test_upload_unsupported_file - NotImplementedError: MANUAL TEST FOR NOW -- e.g. json, ensure error correct and reasonable, no cascades
FAILED tests/test_manual_test.py::test_upload_to_UserData_and_MyData - NotImplementedError: MANUAL TEST FOR NOW Upload to each when enabled, ensure no failures
FAILED tests/test_manual_test.py::test_chat_control - NotImplementedError: MANUAL TEST FOR NOW save chat, select chats, clear chat, export, import, etc.
FAILED tests/test_manual_test.py::test_subset_only - NotImplementedError: MANUAL TEST FOR NOW UserData, Select Only for subset, then put in whisper.  Ensure get back only chunks of data with url links to data sources.
================================================================================ 12 failed, 48 passed, 25 skipped, 1 xfailed, 1 xpassed, 2 warnings in 906.22s (0:15:06) ================================================================================
(h2ollm) jon@pseudotensor:~/h2o-llm$