New Models and Enhancements

[rawnly]

What

Models

Added new models and deprecated some old ones.

• gpt-4-1106-preview
• gpt-4-vision-preview
• gpt-3.5-turbo-1106
• dall-e-2
• dall-e-3

Options

ChatQuery now supports responseFormat
ImageQuery now has model and style properties
AudioTranscriptionQuery now supports responseFormat
AudioTranslationQuery now supports responseFormat

Why

New features availability

Missing

Looking at OpenAI docs seems like functions and function_call parameters are deprecated and replaced with tools

Affected Areas

Image Generation and Chat Completions

fixes #111
fixes #112
#113 (partially)
fixes #114

[PallavAg]

This won't support passing images to the Vision API right?
content can now be a list according to the OpenAI Documentation.

OpenAI/Sources/OpenAI/Public/Models/ChatQuery.swift

Line 13 in c45f332

public let content: String?

[iteroji]

This won't support passing images to the Vision API right? content can now be a list according to the OpenAI Documentation.

OpenAI/Sources/OpenAI/Public/Models/ChatQuery.swift

Line 13 in c45f332

public let content: String?

The request got quite a bit more complicated than it used to be. Now the content can be a single String, an array of strings, or an image_url object, a text object and also an array of the combination of these. I've been trying to come up with an elegant way to do it, but I don't think there is one without writing custom logic. The other question is if there is a need to support every single possibility or just say the content will always be a an array of content objects (image_url, text and others). The endpoint doesn't care after all if you send a single string or a [{"type": "text" , "text":"Prompt comes here"}].

[rawnly]

I think that there's no need to support all the formats, the array with multiple object s should be fine.

I thought the same and that's why i didn't implement anything in this PR

[whydna]

Can't tell if it's been addressed here, but the new models now support parallel function calling - which means the response can contain an array of ChatFunctionCall's

[rawnly]

Can't tell if it's been addressed here, but the new models now support parallel function calling - which means the response can contain an array of ChatFunctionCall's

Nope. The scope of the PR is to support new models and parameters. Nothing changed on response format. I aim to make a new PR eventually once this gets merged

[rawnly]

But I'd love to solve the content "problem" first in order to be able to use gpt-4-vision-preview asap

[rawnly]

The ChatQuery function sign could be overloaded in order to keep the current one and the new one with the array of objects and keep backward compatibility for those who don't need the new features. At the same time looks like functions object is deprecated and should be replaced with tools which currently supports only functions. Again here an overload is possible but I'm not sure that's a good option.

[ingvarus-bc]

Amazing work! 😍
Thanks a lot for your contribution to the project and that you've included tests as well! 🙏🏼🔥
We'll need to handle a deprecation in functions and function_call. It seems like functions was just moved inside tools and renamed to function with a type parameter.
Do you think we should do it out of scope of this PR?

[rawnly]

Do you think we should do it out of scope of this PR?

Yes, the main idea was to support new models. Then I added the new params too since was a very little effort.
I already am working on a branch to support the new content format

  // content is now required
  // @see https://platform.openai.com/docs/api-reference/chat/create#chat-create-messages
  let content: [Content] 

  public struct Content: Codable, Equatable {
      let type: ChatContentType
      let value: String
      
      public enum ChatContentType: String, Codable {
          case text
          case imageUrl = "image_url"
      }
  }

[marcoboerner]

@rawnly thanks for your work! Was waiting for this functionality! 👍

@ingvarus-bc will there be a release tag with those changes?

[ingvarus-bc]

@ingvarus-bc will there be a release tag with those changes?

Hi, @marcoboerner, sure! Will make a release soon, first would be great to finish up with what we discussed in this PR and gotta merge some previous improvements

[rawnly]

i have a working branch with the new content, it's been tricky but it works (just tested on a local project). (all tests passing, + new test to cover the new entity and some encoding/decoding utilities)

tomorrow i can try implementing the new tools too 🙌

[Arnav-arw]

Any ETA when can this PR be merged?

[ingvarus-bc]

Any ETA when can this PR be merged?

Hi, @Arnav-arw, will do it within this working week for sure as well as a new release tag with some of the new functionality!

[ingvarus-bc]

Hey, @rawnly, need a hand with the content you've been working on?

[rawnly]

Hey, @rawnly, need a hand with the content you've been working on?

sure some help is appreciated! The content is ready but i still have some issues

[ingvarus-bc]

Hey, @rawnly, need a hand with the content you've been working on?

sure some help is appreciated! The content is ready but i still have some issues

Okay, you can make a PR, let's finish it up together, so that we could be all set for the release ✨

[ingvarus-bc]

Hi, @rawnly, it seems like tests are failing, in OpenAITestsDecoder.swift

UPD: There was a quick easy fix to tests. 🙌

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information