DRAFT: CU-865c5pdxy AI as a language primitive via an auto-agent

[raulraja]

AI can generate data that automatically conforms to the serializable JSON schema of any given data type.
This DRAFT shows how this can be accomplished with an auto-ai agent that prioritizes tasks to reach an objective and returns the result in the appropriate format.

For example:

package com.xebia.functional.examples.auto

import com.xebia.functional.auto.ai
import kotlinx.serialization.Serializable

@Serializable
data class Population(val size: Int, val description: String)

suspend fun main() {
    val cadiz: Population = ai("Population of Cádiz, Spain.")
    val seattle: Population = ai("Population of Seattle, WA.")
    println("The population of Cádiz is ${cadiz.size} and the population of Seattle is ${seattle.size}")
}

It still lacks proper retrying and better prompting, but it can already show impressive results even when data types have deeply nested properties:

package com.xebia.functional.examples.auto

import com.xebia.functional.auto.ai
import kotlinx.serialization.Serializable

@Serializable
data class Employee(val firstName: String, val lastName: String, val age: Int, val position: String, val company: Company)

@Serializable
data class Address(val street: String, val city: String, val country: String)

@Serializable
data class Company(val name: String, val address: Address)

suspend fun main() {
    val complexPrompt = "Provide information for an Employee that includes their first name, last name, age, position, and their company's name and address (street, city, and country)."

    val employeeData: Employee = ai(complexPrompt)

    println("Employee Information:\n\n" +
            "Name: ${employeeData.firstName} ${employeeData.lastName}\n" +
            "Age: ${employeeData.age}\n" +
            "Position: ${employeeData.position}\n" +
            "Company: ${employeeData.company.name}\n" +
            "Address: ${employeeData.company.address.street}, ${employeeData.company.address.city}, ${employeeData.company.address.country}")
}

Name: John Doe
Age: 35
Position: Manager
Company: ABC Corporation
Address: 123 Main St, Anytown, USA

[raulraja]

Looks like CI currently fails because it can't find the CIO engine in all of our targets:

* What went wrong:
Could not determine the dependencies of task ':compileKotlinJs'.
> Could not resolve all task dependencies for configuration ':jsCompileClasspath'.
   > Could not resolve io.ktor:ktor-client-cio:2.2.2.
     Required by:
         project :
      > No matching variant of io.ktor:ktor-client-cio:2.2.2 was found. The consumer was configured to find a library for use during 'kotlin-api', preferably optimized for non-jvm, as well as attribute 'org.jetbrains.kotlin.platform.type' with value 'js', attribute 'org.jetbrains.kotlin.js.compiler' with value 'ir' but:
          - Variant 'commonMainMetadataElements' capability io.ktor:ktor-client-cio:2.2.2 declares a library for use during 'kotlin-api':
              - Incompatible because this component declares a component, as well as attribute 'org.jetbrains.kotlin.platform.type' with value 'common' and the consumer needed a component, as well as attribute 'org.jetbrains.kotlin.platform.type' with value 'js'
              - Other compatible attributes:
                  - Doesn't say anything about its target Java environment (preferred optimized for non-jvm)
                  - Doesn't say anything about org.jetbrains.kotlin.js.compiler (required 'ir')
          - Variant 'iosArm32ApiElements-published' capability io.ktor:ktor-client-cio:2.2.2 declares a library for use...

[nomisRev]

These changes don't match the OpenAPI spec IIRC? 🤔 Why did you change these defaults?

[raulraja]

The endpoint failed when sending specific requests one at a time on each one of those fields, saying that None was an invalid value for things like logit_bias. I found several places like this one with similar errors openai/openai-python#288, where it states the open API docs do not match the actual expectations of the endpoints—it similarly happened with all the fields where I had to change the default value to make them work. Having accurate integration tests here that test the spec by calling the endpoints would help but there is the cost attached to calling the Open AI API.

[nomisRev]

open API docs do not match the actual expectations of the endpoints

Alright, I thought it was something like this. The OpenAPI Spec seems community build, and not official. Right?
There seems to be some other bugs, and issues with the OpenAPI spec.. 😞

Is there a way for us to add flaky integration tests, without having to pay for tokens?

[raulraja]

I think the best path is to figure out local model loading from hugging face and the llm model that can select other llm for tasks, all those can run locally and not relly on open ai for many use cases.

[nomisRev]

Idem

[nomisRev]

@raulraja I did a small clean-up, and fixed the Ktor Client issues. CIO is not available for JS, but you can just use CIO for Native/JVM and client-js for JS. Then you can just use HttpClient() from commonMain.

[franciscodr]

Is someone fed up with eating always the same? 😂

[nomisRev]

🤣

[nomisRev]

If #17 is merged in I consider this good to be merged 👍

There are still some things we can improve such as turning the errors into typed errors, but we'd need to think about how we want to design the top-level APIs. I wouldn't keep this PR open much longer, it's already getting quite large.