how to configure two Azure API Keys/Endpoints one for embedding and one for GPT4

[rajendra-t]

Hi Team, I am facing issues in configuring Azure LLM with MemGPT, I have two different Endpoints one for embedding model and one for GPT4, when I configure GPT4 I can converse using Memgpt but not use embedding model and when I configure embedding Model I can create embeddings but I can not converse. I am using the following env variables for configuration. Can you please suggest.

https://memgpt.readme.io/docs/endpoints

[cpacker]

On the latest build (0.3.2) you should be able to set the environment variables split for the LLM and the embedding (see https://github.com/cpacker/MemGPT/blob/main/memgpt/cli/cli_config.py#L33-L44):

export AZURE_OPENAI_KEY="..."

export AZURE_OPENAI_VERSION="2023-07-01-preview"
export AZURE_OPENAI_ENDPOINT="https://....openai.azure.com"
export AZURE_OPENAI_DEPLOYMENT="..."

export AZURE_OPENAI_EMBEDDING_VERSION="2023-07-01-preview"
export AZURE_OPENAI_EMBEDDING_ENDPOINT="https://....openai.azure.com"
export AZURE_OPENAI_EMBEDDING_DEPLOYMENT="...

LMK if this doesn't work for you!

[rajendra-t]

hi Charles, Thanks for the feedback, I have a different AZURE_OPEAI_KEY as well - one for Embedding and one for GPT model - can you please suggest if there is way to pass the two separately?

[cpacker]

Ah I see - we actually don't handle this case at the moment, I just opened an issue for it and will try to implement support for this tomorrow.

See: #1002

[cpacker]

In the meantime if you are comfortable with editing source code, you can manually set the key by changing the line here to be your key value.

[rajendra-t]

hi @cpacker I updated the code in my local and passed embedding API key in above as suggested. Post that I configured to use GPT4-32 model with another API KEY passed into AZURE_OPENAI_KEY. However, with this version of pymemgpt (pymemgpt==0.3.2) I am getting below error, I am not sure why, I have setup all the variables as suggested by you. Can you please suggest?

export AZURE_OPENAI_KEY="..."

export AZURE_OPENAI_VERSION="2023-07-01-preview"
export AZURE_OPENAI_ENDPOINT="https://....openai.azure.com"
export AZURE_OPENAI_DEPLOYMENT="..."

export AZURE_OPENAI_EMBEDDING_VERSION="2023-07-01-preview"
export AZURE_OPENAI_EMBEDDING_ENDPOINT="https://....openai.azure.com"
export AZURE_OPENAI_EMBEDDING_DEPLOYMENT="...

Error:

File "/Users/rajendra.t/Library/CloudStorage/OneDrive-MaerskGroup/GITRepo/MemGPT/.venv/lib/python3.11/site-packages/memgpt/main.py", line 347, in run_agent_loop
new_messages, user_message, skip_next_user_input = process_agent_step(user_message, no_verify)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/rajendra.t/Library/CloudStorage/OneDrive-MaerskGroup/GITRepo/MemGPT/.venv/lib/python3.11/site-packages/memgpt/main.py", line 323, in process_agent_step
new_messages, heartbeat_request, function_failed, token_warning, tokens_accumulated = memgpt_agent.step(
^^^^^^^^^^^^^^^^^^
File "/Users/rajendra.t/Library/CloudStorage/OneDrive-MaerskGroup/GITRepo/MemGPT/.venv/lib/python3.11/site-packages/memgpt/agent.py", line 676, in step
raise e
File "/Users/rajendra.t/Library/CloudStorage/OneDrive-MaerskGroup/GITRepo/MemGPT/.venv/lib/python3.11/site-packages/memgpt/agent.py", line 596, in step
response = self._get_ai_reply(
^^^^^^^^^^^^^^^^^^^
File "/Users/rajendra.t/Library/CloudStorage/OneDrive-MaerskGroup/GITRepo/MemGPT/.venv/lib/python3.11/site-packages/memgpt/agent.py", line 359, in _get_ai_reply
raise e
File "/Users/rajendra.t/Library/CloudStorage/OneDrive-MaerskGroup/GITRepo/MemGPT/.venv/lib/python3.11/site-packages/memgpt/agent.py", line 339, in _get_ai_reply
response = create(
^^^^^^^
File "/Users/rajendra.t/Library/CloudStorage/OneDrive-MaerskGroup/GITRepo/MemGPT/.venv/lib/python3.11/site-packages/memgpt/llm_api_tools.py", line 379, in wrapper
raise e
File "/Users/rajendra.t/Library/CloudStorage/OneDrive-MaerskGroup/GITRepo/MemGPT/.venv/lib/python3.11/site-packages/memgpt/llm_api_tools.py", line 352, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/Users/rajendra.t/Library/CloudStorage/OneDrive-MaerskGroup/GITRepo/MemGPT/.venv/lib/python3.11/site-packages/memgpt/llm_api_tools.py", line 461, in create
return azure_openai_chat_completions_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/rajendra.t/Library/CloudStorage/OneDrive-MaerskGroup/GITRepo/MemGPT/.venv/lib/python3.11/site-packages/memgpt/llm_api_tools.py", line 255, in azure_openai_chat_completions_request
assert resource_name is not None, "Missing required field when calling Azure OpenAI"
^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Missing required field when calling Azure OpenAI

[thefux]

Hi, I'm not sure if I get your problem right, but u can try manually editing memgpt config file ..
I tested it with ollama endpoint, so maybe try updating ur config file as I did below, there you can freely set the endpoints

It's for sure not the best way to do it, but works for me ..