-
Notifications
You must be signed in to change notification settings - Fork 121
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@kylehh LGTM!
Please see one clarification question
config/anyscale.yaml
Outdated
|
||
chat_engine: | ||
llm: &llm | ||
type: AnyscaleLLM # Options: [OpenAILLM, AnyscaleLLM] | ||
params: | ||
model_name: meta-llama/Llama-2-7b-chat-hf # The name of the model to use. | ||
model_name: HuggingFaceH4/zephyr-7b-beta # The name of the model to use. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@kylehh why do you want to change the llama2 version?
We have a hacky workaround in CLI to select the batch_size of documents by the expected batch_size of underlying chunks to encode. However this mechanism used a hardcoded number, instead of using the actual Encoder's batch_size
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
I don't understand how is this going to change the initial variable definition, how should it be provided, I already provided anyscale api key and still cannot use the command : canonopy new test This is the error that I get : (canopy-env) kingsize@Goncalos-MacBook-Pro-3 EdGenAI % canopy new test Do you want to continue? [y/N]: y |
Problem
No support for Open Source embedding models
Solution
Added Anyscale Embedding model support for
Type of Change
Test Plan
Describe specific steps for validating this change.
unit test
tests/system/record_encoder/test_anyscale_record_encoder.py
canopy test
config/anyscale.yaml