Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
138 changes: 138 additions & 0 deletions apps/docs/content/guides/dify.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
---
title: Dify Integration
description: Connect Dify's open-source LLM app development platform to LLM Gateway for AI-powered agents and workflows
icon: Dify
---

import { Step, Steps } from "fumadocs-ui/components/steps";
import { Callout } from "fumadocs-ui/components/callout";

[Dify](https://dify.ai) is an open-source LLM app development platform that helps developers build and deploy production-grade AI agents, assistants, and workflow automations. By integrating Dify with LLM Gateway, you gain access to 210+ models (including GPT-5.5, Claude, Gemini, DeepSeek, and more) with built-in cost tracking, response caching, and automatic provider fallbacks.

This guide walks you through connecting Dify to LLM Gateway using the custom OpenAI-compatible provider option.

## Prerequisites

- An LLM Gateway API key — [sign up free](https://llmgateway.io/signup) (no credit card required)
- A Dify Cloud account or a self-hosted Dify instance

## Setup

<Steps>
<Step>
### Open Model Provider Settings

Log in to your Dify dashboard. In the bottom-left corner of the sidebar, click on your profile picture or workspace name, and select **Settings**.

Navigate to the **Model Provider** page to view your active and available LLM configurations.

<img
src="/guides/dify/0-add-provider.png"
alt="Opening Model Provider Settings in Dify"
style={{ borderRadius: "8px", margin: "1.5rem 0" }}
/>

</Step>

<Step>
### Add Custom OpenAI-compatible Provider

Scroll down to the **Custom Providers** section. Locate the **OpenAI-API-compatible** card and click **Setup** (or click **Add Custom Provider** and select **OpenAI-API-compatible**).

<img
src="/guides/dify/1-configure-provider.png"
alt="Selecting OpenAI-compatible provider in Dify"
style={{ borderRadius: "8px", margin: "1.5rem 0" }}
/>

</Step>

<Step>
### Configure Provider Connection

In the setup modal, configure the connection settings for LLM Gateway:

- **Provider Name**: Enter `LLM Gateway`.
- **API Endpoint URL**: Enter `https://api.llmgateway.io/v1`.
- **API Key**: Paste your LLM Gateway API key (starts with `llmgtwy_`).

<img
src="/guides/dify/2-add-model.png"
alt="Configuring LLM Gateway connection in Dify"
style={{ borderRadius: "8px", margin: "1.5rem 0" }}
/>

</Step>

<Step>
### Add Models

Inside the same provider configuration, register the models you wish to use from the LLM Gateway catalog:

1. Click **Add Model**.
2. Set **Model Name** to your desired model identifier (e.g., `openai/gpt-4o-mini`, `anthropic/claude-3-5-sonnet`, `deepseek/deepseek-r1`).
3. Set **Model Type** to **LLM**.
4. Configure model capabilities (such as whether it supports system prompts, function calling, vision, etc.).
5. Click **Save** to add the model.

<img
src="/guides/dify/3-provider-added.png"
alt="Registering custom models in Dify"
style={{ borderRadius: "8px", margin: "1.5rem 0" }}
/>

<Callout type="info">
You can add multiple models to the same LLM Gateway provider block by
repeating this step for other model IDs in the catalog.
</Callout>

</Step>

<Step>
### Save and Verify Provider

Once you have added your models and finished configuring the credentials, click **Save** in the main modal. LLM Gateway will now appear as an active provider in your Dify workspace.

<img
src="/guides/dify/4-select-model.png"
alt="LLM Gateway active provider status in Dify"
style={{ borderRadius: "8px", margin: "1.5rem 0" }}
/>

</Step>

<Step>
### Use LLM Gateway Models in Apps

Create a new application (chatbot, agent, or workflow) in Dify or open an existing one. In the top-right corner of the orchestrator/studio view, click the **Model** dropdown.

Select **LLM Gateway** as the provider, and choose one of the custom models you registered (e.g., `gpt-4o-mini`).

<img
src="/guides/dify/5-test-chat.png"
alt="Selecting LLM Gateway models in Dify app builder"
style={{ borderRadius: "8px", margin: "1.5rem 0" }}
/>

</Step>

<Step>
### Run and Monitor

Try sending a message in the debug console to test your connection. Dify will route the prompt through LLM Gateway, allowing you to monitor real-time token usage, latency, and cost details in your LLM Gateway dashboard.

<img
src="/guides/dify/6-chat-details.png"
alt="Verifying chat outputs and latency in Dify"
style={{ borderRadius: "8px", margin: "1.5rem 0" }}
/>

</Step>
</Steps>

## Why Use LLM Gateway with Dify?

- **Access 210+ models** — Switch between OpenAI, Anthropic, Google Vertex, and open-source models with a single provider endpoint.
- **Cost optimization** — Save up to 90% on selected models using LLM Gateway's developer volume pricing.
- **Enterprise fallbacks** — Prevent system downtime by letting LLM Gateway route requests to a backup provider automatically if the primary provider goes offline.
- **Prompt caching** — Automatically cache repeated workflow prompts to reduce execution latencies and save API costs.
162 changes: 93 additions & 69 deletions apps/docs/content/guides/kimi-code.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import { Callout } from "fumadocs-ui/components/callout";

[Kimi Code CLI](https://github.com/MoonshotAI/kimi-code) is an open-source, AI-powered coding agent developed by Moonshot AI designed to automate software development tasks directly within your terminal. It can read and edit code, execute shell commands, search files, and autonomously manage complex coding workflows.

By configuring Kimi Code CLI to use LLM Gateway, you can point it at any model—GPT-5, Gemini, Llama, Claude, or 210+ others—while keeping the same API formats Kimi Code expects, with full cost tracking in your dashboard.
Kimi Code features first-class support for the **models.dev** registry, a community-maintained model catalog. This allows Kimi Code to query and configure LLM Gateway dynamically — fetching all compatible models, capabilities (such as thinking or vision), and pricing without requiring manual TOML editing.

## Prerequisites

Expand Down Expand Up @@ -49,119 +49,143 @@ kimi --version
</Step>

<Step>
### Configure config.toml
### Launch Kimi Code and Open the Provider Manager

Create or edit your Kimi Code configuration file at `~/.kimi-code/config.toml` (on Windows, this is typically under `C:\Users\<YourUsername>\.kimi-code\config.toml`).
Start the interactive terminal in your project directory:

Add the `llmgateway` provider and define the models you want to use. Here is an example configuration that sets up **GPT-5.5**, **Claude Opus 4.6**, **DeepSeek V4 Pro**, **MiniMax M3**, and **Qwen3.7 Max**:
```bash
kimi
```

```toml
default_model = "llmgateway/gpt-5.5"
Once loaded, type the `/provider` command and press Enter. Select **Known third-party provider** to fetch the catalog from the registry:

[providers.llmgateway]
type = "openai"
api_key = "llmgtwy_your_api_key_here"
base_url = "https://api.llmgateway.io/v1"
<img
src="/guides/kimi-code/0-add-provider.png"
alt="Opening Provider Manager in Kimi Code"
style={{ borderRadius: "8px", margin: "1.5rem 0" }}
/>

[models."llmgateway/gpt-5.5"]
provider = "llmgateway"
model = "gpt-5.5"
max_context_size = 1050000
max_output_size = 128000
capabilities = [ "image_in", "thinking", "tool_use" ]
display_name = "GPT-5.5"
</Step>

[models."llmgateway/claude-opus-4-6"]
provider = "llmgateway"
model = "claude-opus-4-6"
max_context_size = 1000000
max_output_size = 128000
capabilities = [ "image_in", "thinking", "tool_use" ]
display_name = "Claude Opus 4.6"
<Step>
### Select LLM Gateway

[models."llmgateway/deepseek-v4-pro"]
provider = "llmgateway"
model = "deepseek-v4-pro"
max_context_size = 1050000
max_output_size = 393216
capabilities = [ "thinking", "tool_use" ]
display_name = "DeepSeek V4 Pro"
Type `llm` to filter the providers and select **LLM Gateway** from the list:

[models."llmgateway/minimax-m3"]
provider = "llmgateway"
model = "minimax-m3"
max_context_size = 1048576
max_output_size = 131072
capabilities = [ "image_in", "thinking", "tool_use" ]
display_name = "MiniMax M3"
<img
src="/guides/kimi-code/1-select-provider.png"
alt="Selecting LLM Gateway in Kimi Code"
style={{ borderRadius: "8px", margin: "1.5rem 0" }}
/>

[models."llmgateway/qwen3.7-max"]
provider = "llmgateway"
model = "qwen3.7-max"
max_context_size = 1000000
max_output_size = 65536
capabilities = [ "thinking", "tool_use" ]
display_name = "Qwen3.7 Max"
```
</Step>

<Step>
### Enter Your API Key

When prompted, paste your LLM Gateway API key and press Enter. Kimi Code will save it securely to your local configuration:

<img
src="/guides/kimi-code/0-config.png"
alt="Configuring config.toml"
src="/guides/kimi-code/2-enter-key.png"
alt="Entering LLM Gateway API Key"
style={{ borderRadius: "8px", margin: "1.5rem 0" }}
/>

<Callout type="info">
Replace `llmgtwy_your_api_key_here` with your actual LLM Gateway API key from
the dashboard.
Your credentials are saved locally to `~/.kimi-code/config.toml`.
</Callout>

</Step>

<Step>
### Run Kimi Code CLI
### Select a Model and Toggle Thinking

Navigate to your project folder and launch the interactive terminal:
The LLM Gateway catalog is now loaded. Use the arrow keys to browse or type to search for your desired model. Select the `llmgateway` tab to view only LLM Gateway models.

```bash
kimi
```
You can also toggle the **Thinking** option (On/Off) at the bottom depending on the model's capabilities:

All requests will now be routed through LLM Gateway, allowing you to use advanced models for local autonomous coding while showing real-time usage and cost statistics on your LLM Gateway dashboard.
<img
src="/guides/kimi-code/3-select-model.png"
alt="Browsing LLM Gateway Models"
style={{ borderRadius: "8px", margin: "1.5rem 0" }}
/>

For example, type `gpt-5.5` to find the latest reasoning model, select it, and press Enter:

<img
src="/guides/kimi-code/1-chat.png"
alt="Running Kimi Code with LLM Gateway"
src="/guides/kimi-code/4-select-gpt-model.png"
alt="Selecting GPT-5.5 Model"
style={{ borderRadius: "8px", margin: "1.5rem 0" }}
/>

</Step>

<Step>
### Start Coding

All set! Kimi Code is now configured. Your requests will be securely routed through LLM Gateway, allowing you to use advanced models for local autonomous coding while showing real-time usage and cost statistics on your LLM Gateway dashboard.

<img
src="/guides/kimi-code/5-chat.png"
alt="Running Kimi Code CLI session"
style={{ borderRadius: "8px", margin: "1.5rem 0" }}
/>

Use `/model` in the terminal session at any time to switch models.

</Step>
</Steps>

## Configuration Details
## Manual Configuration (Advanced)

### The Providers Section
If you prefer to configure your environment manually without using the interactive provider manager, you can write settings directly to your configuration file at `~/.kimi-code/config.toml` (or `C:\Users\<YourUsername>\.kimi-code\config.toml` on Windows).

To connect to LLM Gateway, define a custom provider with `type = "openai"` and specify the base URL pointing to the LLM Gateway endpoint.
Here is an example TOML configuration that registers **GPT-5.5**, **Claude 3.7 Sonnet**, **DeepSeek R1**, and **Qwen3.7 Max** manually:

```toml
default_model = "llmgateway/gpt-5.5"

[providers.llmgateway]
type = "openai"
api_key = "llmgtwy_your_api_key_here"
base_url = "https://api.llmgateway.io/v1"
```

### Defining Custom Models
[models."llmgateway/gpt-5.5"]
provider = "llmgateway"
model = "gpt-5.5"
max_context_size = 1050000
max_output_size = 128000
capabilities = [ "thinking", "tool_use" ]
display_name = "GPT-5.5"

For each model you want to access, add a `[models."<provider_name>/<model_identifier>"]` block:
[models."llmgateway/claude-3.7-sonnet"]
provider = "llmgateway"
model = "claude-3.7-sonnet"
max_context_size = 200000
max_output_size = 8192
capabilities = [ "image_in", "thinking", "tool_use" ]
display_name = "Claude 3.7 Sonnet"

- **provider**: Must match the provider key under `[providers.<key>]` (e.g. `llmgateway`).
- **model**: The exact model ID from the LLM Gateway catalog.
- **capabilities**: An array containing capabilities the model supports, such as `"image_in"`, `"thinking"`, and `"tool_use"`.
- **max_context_size**: The maximum context window of the model.
[models."llmgateway/deepseek-r1"]
provider = "llmgateway"
model = "deepseek-r1"
max_context_size = 131072
max_output_size = 8192
capabilities = [ "thinking", "tool_use" ]
display_name = "DeepSeek R1"

[models."llmgateway/qwen3.7-max"]
provider = "llmgateway"
model = "qwen3.7-max"
max_context_size = 1000000
max_output_size = 65536
capabilities = [ "thinking", "tool_use" ]
display_name = "Qwen3.7 Max"
```

## Why Use LLM Gateway with Kimi Code CLI

- **210+ models** — Access GPT-5, Gemini, Llama, DeepSeek, and more in a single CLI configuration.
- **210+ models** — Access GPT-5.5, Gemini, Llama, DeepSeek, and more in a single CLI configuration.
- **Unified cost tracking** — Get a detailed breakdown of costs per prompt and session in your dashboard.
- **Response caching** — Automatically cache repeated requests (such as parsing or building commands) to save API costs.
- **Automatic fallback** — Keep coding even if a provider encounters temporary downtime.
Expand Down
14 changes: 14 additions & 0 deletions apps/docs/lib/custom-icons.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -341,4 +341,18 @@ export const customIcons: Record<string, IconComponent> = {
/>
</svg>
),
Dify: (props) => (
<svg
style={{ flex: "none", lineHeight: "1" }}
role="img"
viewBox="0 0 24 24"
xmlns="http://www.w3.org/2000/svg"
{...props}
>
<path
fill="currentColor"
d="m22.417 9.334-1.333 4.333-1.334-4.333h-1.583L20.1 14.94c.2.583-.14 1.06-.756 1.06h-.678v1.334h.996c.869 0 1.65-.55 1.945-1.367L24 9.334ZM2.833 6.667H0v8.666h2.833c3.5 0 4.5-2 4.5-4.333s-1-4.334-4.5-4.334zM2.866 14H1.6V8h1.266c2.013 0 2.867.988 2.867 3s-.854 3-2.867 3m11-5.267v.6h-1.532v1.334h1.533V14h-2.534V9.334H8v1.334h1.867V14h-2.2v1.334h10V14h-2.332v-3.333h2.333V9.334h-2.333V8h2.333V6.667h-1.733a2.07 2.07 0 0 0-2.067 2.067Zm-3.266-.2c.681 0 .933-.417.933-.933 0-.515-.252-.933-.933-.933-.68 0-.934.418-.934.933s.253.934.934.934"
/>
</svg>
),
};
Binary file added apps/docs/public/guides/dify/0-add-provider.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added apps/docs/public/guides/dify/2-add-model.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added apps/docs/public/guides/dify/3-provider-added.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added apps/docs/public/guides/dify/4-select-model.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added apps/docs/public/guides/dify/5-test-chat.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added apps/docs/public/guides/dify/6-chat-details.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed apps/docs/public/guides/kimi-code/0-config.png
Binary file not shown.
Binary file removed apps/docs/public/guides/kimi-code/1-chat.png
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added apps/docs/public/guides/kimi-code/2-enter-key.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added apps/docs/public/guides/kimi-code/5-chat.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added apps/ui/public/guides/dify/0-add-provider.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added apps/ui/public/guides/dify/2-add-model.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added apps/ui/public/guides/dify/3-provider-added.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added apps/ui/public/guides/dify/4-select-model.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added apps/ui/public/guides/dify/5-test-chat.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added apps/ui/public/guides/dify/6-chat-details.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added apps/ui/public/guides/kimi-code/2-enter-key.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added apps/ui/public/guides/kimi-code/5-chat.png
Loading
Loading