Support gizmo models (Custom GPTs) in the OpenAI ChatGPT provider, APIs, SDK, and web UI

[defcron]

Please consider supporting gizmo models. You can see how it's done by examining the github project ChatGPT-Mirror, or also the github project chatGPTBox (which is a web browser extension). Both of those projects are supporting gizmo models relatively properly in their respective ChatGPT web model provider implementations, and a popular other project I've tried which is claiming to support them but isn't doing so correctly at all is the github project called chat2api, so you could maybe look at that one for comparison against the other two, to see the differences between what works properly and what doesn't, if you decide to add this feature.

If the feature is already there, it wasn't readily apparent how to use it properly through the gpt4free web UI so then if it's already there for other aspects of this project such as the APIs and client libraries, then consider this to be a feature request to add support for gizmo models to any part which doesn't support them yet.

Thanks for considering. It would be something which makes the difference to me between using this project at all, or else needing to use something else instead such as a combination of ChatGPT-Mirror <--> one-api <--> llm (the github project called "llm"). My use case right now for gpt4free would be only to use with gizmos so this is a feature I would love to see, so I can use gpt4free for my particular porpoises. It's maybe asking a lot though, so I understand if it's not a priority.

[hlohaus]

Hey, @defcron, you can use gizmos models already. Just pass a custom name to OpenaiChat. Can you give me a list of those models? I'll add them to the provider. Got any links to the GitHub projects?

[defcron]

Oh okay, I didn't immediately see a way to do that in the web UI though. And I'm referring to the models named by convention like "gpt-4-gizmo-g-". So there isn't a list really, but I know that using these kinds of models on ChatGPT web requires hitting a few other endpoints to bootstrap the sessions properly, and the backend-api/conversation request also is a little bit different than a normal model request, with an extra field or two and some requirement to specify both the gizmo's ID in one field and also the base model's name that you're requesting to base the gizmo on, which currently has a few restrictions, such as that you can base them on any model available for use in your account in the case where the gizmo doesn't have any Custom Actions / jit_plugins defined in its config, but if it does have any of those, then the gizmo is only able to be based on GPT-4o or GPT-4-1.

This is what I'm currently using to access gizmos mostly correctly at the moment: https://github.com/dairoot/ChatGPT-Mirror

It mostly works well when you put one-api as a middleware in front of it, and then use a client to request one-api such as the "llm" command from https://github.com/simonw/llm

I tried dozens of other solutions, and there was problems with every other combination of tools I tried, the most prominent issue was lack of proper persistent conversation_id tracking and continual chat thread support, but the chat2api project, which is like ChatGPT-Mirror but more popular, has a very messed up attempt at persisting conversation threads which causes gizmo models to think you've just uploaded their entire set of knowledge files during every prompt you send them, and so every response they give involves them mentioning their set of knowledge files and asking you want you want it to do with them, and they tend to disregard the actual prompt you send and instead just talk about their files, so the chat prompts don't really work at all with gizmos when using chat2api to access them. Just thought I'd mention that part of my experience here in case it saves you or some users some trouble and time if trying to have gizmo model chats and trying out various ways of doing so.

Can you confirm though if you've implemented gizmo support specifically though or not? I didn't notice any support code for them when greping through the gpt4free code base, but I didn't look super closely.

[hlohaus]

Hey @defcron, yeah, we do conversations, no problem. We've got a special field for that. And yep, for gizmo, we need to add two more things. How do you wanna do it? You using the GUI, or do you want a CLI thing?

[defcron]

A cli thing would be better.

[hlohaus]

Hey, GPT4Free doesn't have a CLI yet, right? I wanna rewrite GeminiCLI for it. There's a lot to do. @defcron

[defcron]

Yeah I know, I wrote a couple of scripts which use your async client which work well, and it would be nice if the client libraries supported gizmos tho.

[defcron]

Hey, GPT4Free doesn't have a CLI yet, right? I wanna rewrite GeminiCLI for it. There's a lot to do. @defcron

This is what I'm using at the moment, fyi, and it works pretty well for chat with regular models:

#!/usr/bin/env python3

import os
import sys
import asyncio
import json
import g4f
from g4f.client import AsyncClient
from g4f.Provider import OpenaiAccount
from g4f.cookies import set_cookies_dir
from g4f.Provider.needs_auth.OpenaiChat import Conversation

async def main():
    input_text = sys.stdin.read()

    cookies_dir = "/opt/gptenv/g4f/har_and_cookies"
    conversation_json_file_path = "/opt/gptenv/.conversation.g4f.json"
    conversation = None

    # load conversation from file
    if os.path.isfile(conversation_json_file_path):
        try:
            with open(conversation_json_file_path, 'r') as f:
                data = json.load(f)

            conversation = Conversation(
                data["conversation_id"], data["message_id"], data["user_id"],
                data["finish_reason"], data["parent_message_id"], data["is_thinking"]
            )
            conversation.recipient = data["recipient"]
            conversation.p = data["p"]
            conversation.thoughts_summary = data["thoughts_summary"]
            conversation.prompt = data["prompt"]
            conversation.generated_images = data["generated_images"]

        except json.JSONDecodeError as e:
            print(f"Error: JSONDecodeError: {e}\n")
        except Exception as e:
            print(f"Error: Exception: {e}\n")

    set_cookies_dir(cookies_dir)
    client = AsyncClient(provider=OpenaiAccount)
    chunks = []
    current_chunk = None

    async for chunk in client.chat.completions.create(
        messages=[{"role": "user", "content": input_text.strip()}],
        stream=True,
        conversation=conversation
    ):
        current_chunk = chunk
        chunks.append(chunk)
        token = chunk.choices[0].delta.content or ''
        for byte in token.encode('utf-8'):
            sys.stdout.buffer.write(bytes([byte]))
            sys.stdout.buffer.flush()

    last_chunk = current_chunk

    with open(conversation_json_file_path, "w") as f:
        json.dump(last_chunk.conversation.get_dict(), f, indent=4)

    print("")

if __name__ == '__main__':
    asyncio.run(main())

I set it as my bash command_not_found_handle() with a bit of extra stuff and it works pretty nicely.

You could use something from that example script if you want for whatever, or not, but I thought I'd share my solution. There's a couple of hard-coded paths in there because I was lazy, but that can be easily corrected.

@hlohaus ^

[hlohaus]

@defcron

This is the clean example from docs:

import g4f
import asyncio
from g4f.client import AsyncClient
from g4f.Provider import OpenaiAccount

async def main():
    client = AsyncClient(
        provider=OpenaiAccount
    )

    response = await client.chat.completions.create(
        messages="I was born on 01-01-1999.",
    )

    response = await client.chat.completions.create(
        messages="How old i am? I told you before.",
        conversation=response.conversation
    )

    print(response.conversation.__dict__)
    print(response.choices[0].message.content)

asyncio.run(main())

Simply pass the JsonConversation:
dict = conversation.get_dict()
JsonConversation(**dict)

I will add your gizimo model (gpt-4o) with special conversation options and your script to the g4f CLI. But i can't say how many time it takes.

[defcron]

@defcron

This is the clean example from docs:

import g4f
import asyncio
from g4f.client import AsyncClient
from g4f.Provider import OpenaiAccount

async def main():
client = AsyncClient(
provider=OpenaiAccount
)

response = await client.chat.completions.create(
    messages="I was born on 01-01-1999.",
)

response = await client.chat.completions.create(
    messages="How old i am? I told you before.",
    conversation=response.conversation
)

print(response.conversation.__dict__)
print(response.choices[0].message.content)

asyncio.run(main())

Simply pass the JsonConversation: dict = conversation.get_dict() JsonConversation(**dict)

I will add your gizimo model (gpt-4o) with special conversation options and your script to the g4f CLI. But i can't say how many time it takes.

The issue solved by my script is mainly the ability to reuse the same conversation between script invocations, which is why it serializes the JsonConversation as json and saves it to disk, and reloads it from disk if present and then reinstantiates a Conversation from it (I couldn't get it to work like how you show with JsonConversation(**dict) and then using that to instantiate a Conversation object because the initializer list/params/call signature of the Conversation constructor are a bit different than the values returned by conversation.get_dict(), so the **dict thing was failing for me and that's why I'm manually populating some of the private properties manually after instantiation in my script. But if there's a better way that works, I'll prefer a proper way without all those extra lines haha!).

Also, it streams the async response stream output byte-by-byte instead of the more standard token-by-token that most CLI do, which is just a personal preference of mine and matches more closely to the behaviour of chat responses in most of the official websites like on ChatGPT the models' responses appear character-by-character, not token-by-token, so that's what I wanted to do as well and this one does that.

[hlohaus]

You need to use JsonConversation for ** and this lines are not needed too:

        conversation.recipient = data["recipient"]
        conversation.p = data["p"]
        conversation.thoughts_summary = data["thoughts_summary"]
        conversation.prompt = data["prompt"]
        conversation.generated_images = data["generated_images"]

That are only internal fields.

There is already are g4f gui/api comman. Maybe i add a query or create command

[hlohaus]

@defcron I have created the g4f.cli.client:

python -m g4f.cli client -h

usage: __main__.py client [-h] [--debug] [-p PROVIDER] [-m MODEL] [-c COOKIES_DIR] [--conversation-file CONVERSATION_FILE] [--clear-history]
                          [input]

positional arguments:
  input                 Input text (or read from stdin)

options:
  -h, --help            show this help message and exit
  --debug, -d           Enable verbose logging.
  -p PROVIDER, --provider PROVIDER
                        Provider to use. Available: Anthropic, BingCreateImages, BlackForestLabs_Flux1Dev, Blackbox, BlackboxPro, CablyAI,
                        Cerebras, Chatai, Cloudflare, CohereForAI_C4AI_Command, Copilot, CopilotAccount, Custom, DeepInfra, DeepInfraChat,
                        DeepSeek, DeepSeekAPI, DeepseekAI_JanusPro7b, DuckDuckGo, EdgeTTS, Free2GPT, FreeGpt, Gemini, GeminiPro, GigaChat,
                        GithubCopilot, GlhfChat, Grok, Groq, HailuoAI, HarProvider, HuggingChat, HuggingFace, HuggingFaceAPI,
                        HuggingFaceInference, HuggingFaceMedia, HuggingSpace, ImageLabs, LMArenaBeta, LambdaChat, LegacyLMArena, MarkItDown,
                        MetaAI, MetaAIAccount, MicrosoftDesigner, Microsoft_Phi_4_Multimodal, MiniMax, OIVSCodeSer0501, OIVSCodeSer2, Ollama,
                        OpenAIFM, OpenRouter, OpenaiAPI, OpenaiAccount, OpenaiChat, PerplexityApi, PerplexityLabs, Pi, PollinationsAI,
                        PollinationsImage, PuterJS, Qwen_Qwen_2_5, Qwen_Qwen_2_5M, Qwen_Qwen_2_5_Max, Qwen_Qwen_2_72B, Qwen_Qwen_3, Reka,
                        Replicate, StabilityAI_SD35Large, TeachAnything, ThebApi, Together, Video, WeWordle, Websim, WhiteRabbitNeo, You,
                        Yqcloud, gTTS, xAI, AnyProvider.
  -m MODEL, --model MODEL
                        Model to use (provider-specific)
  -c COOKIES_DIR, --cookies-dir COOKIES_DIR
                        Directory containing cookies for authenticated providers
  --conversation-file CONVERSATION_FILE
                        File to store/load conversation state
  --clear-history       Clear conversation history before starting

[github-actions]

Bumping this issue because it has been open for 7 days with no activity. Closing automatically in 7 days unless it becomes active again.

[github-actions]

Closing due to inactivity.