[model client enchancement] Support internally hosted models that can be accessed via requests

[liyin2015]

Is your feature request related to a problem? Please describe.

User case:

In our corporate environment, we use an internally hosted version of the ChatGPT-Omni model due to security restrictions, and we access it via a custom API. We send requests using requests.post with the headers containing an authorization token, and the prompt is passed as params['messages'] in the format {'role': 'system', 'content': sys_prompt}, {'role': 'user', 'content': user_prompt}.

Describe the solution you'd like

There are possibly two solutions: (1) one request based model client (2) as LiteLLM is request native, they are most likely already support this case.

We need to choose the solution with the least amount of work.

Process

We will need contributor to make a short proposal at this issue. And then follow the guideline to add model_client in the contributor guide.

[jimixxperez]

@liyin2015 Having the same issue with a client currently. Custom authentication process for an internally hosted AzureOpenAI endpoint.

I think creating a bridge via the CustomLLM interface of litellm is the just the right amount of abstraction above a low level http client like request or httpx (async), etc. You can easily share it and implement the requirements you have.
You can then simply use this abstraction or you can even inject your custom interface into the litellm proxy that supported tons for configurations.

[liyin2015]

@jimixxperez interesting, can you read the discussion in #269

So basically we want our developers to use provider's sdk (prefer) but tell them to use litellm integration for internally hosted AzureOpenAI endpoint? Will litellm help us extend to GCP or other cloud hosted end point?

It would be great if you can also provide some code examples