Pr139 dev

[risingsunomi]

Merging in PR139 development

• Tested with Qwen2 Instruct, TinyLlama, and Llama3 8B models and successfully generated tokens.
• Currently, the output quality is poor, often producing gibberish. I am investigating top_p, top_k, temperature settings, and the logit sampling method for improvements.
• Successfully tested the setup on an NVIDIA A10 with enough resources for model generation.
• Tested on local NVIDIA-based servers, achieving cross-computer generation on Linux systems. Encountered out-of-memory (OOM) errors, even with smaller models; will explore memory management on low VRAM GPUs.
• Generalized the ShardedHuggingFaceModel forward function compatible with various HuggingFace transformer model classes, though specific handling for different model types is still required.
• Reformatted interface test for pytorch
• Updated generate_deps.py to update tinychat model select dropdown
• Fix for fast check for loading tokenizers as was comparing a string to a PathLib
• Update ShardedHuggingFaceModel to always use caching