handle dict resp with hidden params (extra hdrs)#3
handle dict resp with hidden params (extra hdrs)#3tanmaykm wants to merge 1 commit intov1.82.1_with_jh_fixesfrom
Conversation
For the pass through route, when the `async_post_call_success_hook` is called, it is provided with a dict containing the response. If it needs to add response headers, it has to pass it through that. In this change we add a check for dict responses and pick up extra headers from it if that is the case.
|
Can you elaborate what the specific issue is that we're fixing here? It seems like you're saying the response is sometimes a dict and sometimes not a dict? |
|
Yeah... so streaming requests do not result in a dict response, while non-streaming requests do. This is for anthropic, and from what I tested. Unfortunately though the streaming requests in anthropic passthrough do not invoke the plugin callback. So there is no way currently for plugin to pass custom headers in streaming responses. But that is probably a separate thing to be fixed. From what I understand (and what claude informs) there are several type of responses for other providers, and some of them may have a way to set the headers some not. So the code checks what it got and gets headers if possible. |
|
BTW Looks like there might be a better way now. I see this was merged recently: BerriAI#20070. If that works we may not need this. Will try it out. |
|
... and that works! Closing this |
2 participants
For the pass through route, when the
async_post_call_success_hookis called, it is provided with a dict containing the response. If it needs to add response headers, it has to pass it through that. In this change we add a check for dict responses and pick up extra headers from it if that is the case.