[Inference API] Adding custom headers for Azure OpenAI#6134
[Inference API] Adding custom headers for Azure OpenAI#6134jonathan-buttner merged 7 commits intomainfrom
Conversation
| description: | ||
| Run `POST _inference/chat_completion/reasoning-chat-completion/_stream` to perform a chat completion task, | ||
| using both `effort` parameter based reasoning configuration and including reasoning generated by the model on previous step. | ||
| Run `POST _inference/chat_completion/reasoning-chat-completion/_stream` to perform a chat completion task, using both |
There was a problem hiding this comment.
I think this was an automated change 🤔
| description: | ||
| Run `POST _inference/chat_completion/reasoning-chat-completion/_stream` to perform a chat completion task, | ||
| using both `enabled` parameter based reasoning configuration and including reasoning generated by the model on previous step. | ||
| Run `POST _inference/chat_completion/reasoning-chat-completion/_stream` to perform a chat completion task, using both |
There was a problem hiding this comment.
I think this was an automated change 🤔
| export class OpenAITaskSettings { | ||
| /** | ||
| * For a `completion`, `chat_completion`, or `text_embedding` task, specify the user issuing the request. | ||
| * Specifies the user issuing the request. |
There was a problem hiding this comment.
Figured I'd clean this one up, the user works for all task types so no reason to specify which ones.
|
Following you can find the validation changes against the target branch for the APIs. No changes detected. You can validate these APIs yourself by using the |
Jan-Kazlouski-elastic
left a comment
Jan-Kazlouski-elastic
left a comment
There was a problem hiding this comment.
Looks good to me. I left a minor suggestion.
| * Specifies custom HTTP header parameters. | ||
| * For example: | ||
| * ``` | ||
| * "headers":{ |
There was a problem hiding this comment.
| * "headers":{ | |
| * "headers": { |
| * } | ||
| * ``` | ||
| */ | ||
| headers?: UserDefinedValue |
There was a problem hiding this comment.
UserDefinedValue is usually the last resort when data has no regular structure. Could this be a Map<String,String> ?
There was a problem hiding this comment.
Yeah good point, I'll switch to use Record<String, String>.
jonathan-buttner
added
the
skip-backport
| * ``` | ||
| */ | ||
| headers?: UserDefinedValue | ||
| headers?: Dictionary<string, string> |
There was a problem hiding this comment.
@l-trotta should I backport this change? This is the other place we were already using UserDefinedValue.
There was a problem hiding this comment.
I think there's even a couple more places in the same file? I'd change them all and not backport, so that the changes only affect 9.4
| * The parameters the functional accepts. This should be formatted as a JSON object. | ||
| */ | ||
| parameters?: UserDefinedValue | ||
| parameters?: Dictionary<string, UserDefinedValue> |
There was a problem hiding this comment.
@l-trotta parameters is expected to be a json object. I'm happy to change this to something else if you have a better idea.
There was a problem hiding this comment.
hmm I see there are multiple of this as well, and since there's no hint of the structure I'd keep this as a UserDefinedValue
There was a problem hiding this comment.
Sounds good, I'll switch it back.
| search | ||
| } | ||
|
|
||
| export type CustomServiceQueryParameter = string[] |
There was a problem hiding this comment.
@l-trotta ideally this would be [string, string] but I got an error that a tuple isn't supported.
There was a problem hiding this comment.
is this a case where the server will return error if there's more or less than 2 items?
There was a problem hiding this comment.
Yeah the server will validate that it's an array of strings of length 2. If it's not, we'll throw an error back to the user.
There was a problem hiding this comment.
yep we don't have anything to enforce this... the description will have to do ^^"
jonathan-buttner
requested review from
DonalEvans,
Jan-Kazlouski-elastic and
l-trotta
4 participants
This PR adds the docs for the custom
headersfield withintask_settingsfor Azure OpenAI. The Elasticsearch functionality was added in this PR: elastic/elasticsearch#142969