Improve prompt exceed truncation logic & Try Raise exception when prompt exceed truncation for openai api

[yhyu13]

Summery

When chat prompt exceed allowed length, try raise excpetion (in openai api) instead of removing any of the prompt messages.

• Fix openai api exception not using http exception (so that openai client can now properly interpret openai exception thrown by webui)
• Improve truncation logic. Instead of removing user role all together, we try keep every other role and remove user prompt by considering both the ratio of total prompt that needs to be truncated and ratio of user prompt among all prompts. We then try this truncation strategy multiple times until re-tries runs out. If re-tries run out, for openai api, we throw an excpetion, and for webUI, we continue processing

Checklist:

• I have read the Contributing guidelines.

[yhyu13]

I sometimes send user prompt exceeding truncation length via the opanai api. Previous method simply drop the user prompt, resulting the model generating random messages in return. Rather, send an excpetion to the client

For example, the excpetion would be openai.InternalServerError

  File "/home/hangyu5/Documents/Git-repoMy/open-ai-doc-sum/gen_file_md.py", line 59, in get_summary
    response = client.chat.completions.create(model=openai_model,
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hangyu5/anaconda3/envs/gtfo/lib/python3.11/site-packages/openai/_utils/_utils.py", line 272, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/hangyu5/anaconda3/envs/gtfo/lib/python3.11/site-packages/openai/resources/chat/completions.py", line 645, in create
    return self._post(
           ^^^^^^^^^^^
  File "/home/hangyu5/anaconda3/envs/gtfo/lib/python3.11/site-packages/openai/_base_client.py", line 1088, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hangyu5/anaconda3/envs/gtfo/lib/python3.11/site-packages/openai/_base_client.py", line 853, in request
    return self._request(
           ^^^^^^^^^^^^^^
  File "/home/hangyu5/anaconda3/envs/gtfo/lib/python3.11/site-packages/openai/_base_client.py", line 916, in _request
    return self._retry_request(
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/hangyu5/anaconda3/envs/gtfo/lib/python3.11/site-packages/openai/_base_client.py", line 958, in _retry_request
    return self._request(
           ^^^^^^^^^^^^^^
  File "/home/hangyu5/anaconda3/envs/gtfo/lib/python3.11/site-packages/openai/_base_client.py", line 916, in _request
    return self._retry_request(
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/hangyu5/anaconda3/envs/gtfo/lib/python3.11/site-packages/openai/_base_client.py", line 958, in _retry_request
    return self._request(
           ^^^^^^^^^^^^^^
  File "/home/hangyu5/anaconda3/envs/gtfo/lib/python3.11/site-packages/openai/_base_client.py", line 930, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.InternalServerError: Internal Server Error

Edit:
Now throwing httperror in the openai ai extension 4a63125
This way, any openai client application could resolve to openai api errors by status code.
E.g.

openai.BadRequestError: Error code: 400 - {'detail': {'message': 'Prompt encoded_length 5537 > max_length 1536', 'code': 400, 'param':

Otherwise, openai client could only resolve to InternalServerError

[yhyu13]

I found the openai extension does not use openai's exception classes, instead it throwing (e.g. OpenAIError, InvalidRequestError) custom ones that our openai client side won't resolve. All of them resolve to openai.InternalServerError. This might be confusin to users. We might need to refactor exception handling in the the openai extension to mimic the response of an openai api server.

Edit:
Fixed both trucating strategy and openai http error in 4a63125

[yhyu13]

Just found out a typical out of context error from openai looks like below, I will adjust the truncation error similar to this later

Invalid response object from API: '{"object":"error","message":"流式输出失败 {"error":{"message":"This model's maximum context length is 4097 tokens. However, you requested 4232 tokens (1232 in the messages, 3000 in the completion). Please reduce the length of the messages or completion.","type":"invalid_request_error","param":"messages","code":"context_length_exceeded"}}","code":50099}' (HTTP response code was 400)

[oobabooga]

That can't be added. Long conversations in the UI would raise exceptions. It would be better to somehow truncate the user input instead of removing it if it's too long. That should be possible by changing a few lines in generate_chat_prompt.

[yhyu13]

That can't be added. Long conversations in the UI would raise exceptions. It would be better to somehow truncate the user input instead of removing it if it's too long. That should be possible by changing a few lines in generate_chat_prompt.

@oobabooga Thanks for your advice, I add an conditional kwarg so that only opani api would raise excpetion on fail to truncate as that is what openai official api would do. Otherwise, the faild truncated prompt would continue for furher processes

[oobabooga]

Closing in favor of #5439