Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
106 commits
Select commit Hold shift + click to select a range
c51696d
auth_with_role_name add region_name arg for cross-account sts
ta-stripe Feb 20, 2026
5584f2d
update tests to include case with aws_region_name for _auth_with_aws_…
ta-stripe Feb 20, 2026
c0eb566
Only pass region_name to STS client when aws_region_name is set
ta-stripe Feb 20, 2026
6df8dad
Add optional aws_sts_endpoint to _auth_with_aws_role
ta-stripe Feb 20, 2026
782df92
Parametrize ambient-credentials test for no opts, region_name, and aw…
ta-stripe Feb 20, 2026
be885ad
consistently passing region and endpoint args into explicit credentia…
ta-stripe Feb 20, 2026
08a561f
fix env var leakage
ta-stripe Feb 20, 2026
4885b36
fix: bedrock openai-compatible imported-model should also have model …
ta-stripe Feb 20, 2026
107f9dc
feat: show proxy url in ModelHub (#21660)
janfrederickk Feb 21, 2026
4cc58fe
fix(bedrock): correct modelInput format for Converse API batch models…
hztBUAA Feb 21, 2026
c845c33
fix(proxy): add model_ids param to access group endpoints for precise…
hztBUAA Feb 21, 2026
647ea3b
feat(proxy): add custom favicon support\n\nAdd ability to configure a…
hztBUAA Feb 21, 2026
d53ea88
fix(bedrock): prevent double UUID in create_file S3 key (#21650)
hztBUAA Feb 21, 2026
e0aaedc
feat(semantic-cache): support configurable vector dimensions for Qdra…
hztBUAA Feb 21, 2026
cc6ef0e
fix(utils): normalize camelCase thinking param keys to snake_case (#2…
Chesars Feb 21, 2026
ca5c109
feat: add optional digest mode for Slack alert types (#21683)
dkindlund Feb 21, 2026
e37d54e
feat: add blog_posts.json and local backup
yuneng-jiang Feb 22, 2026
be1a543
feat: add GetBlogPosts utility with GitHub fetch and local fallback
yuneng-jiang Feb 22, 2026
7e5c63c
test: add cache reset fixture and LITELLM_LOCAL_BLOG_POSTS test
yuneng-jiang Feb 22, 2026
021a909
feat: add GET /public/litellm_blog_posts endpoint
yuneng-jiang Feb 22, 2026
4b1ce1f
fix: log fallback warning in blog posts endpoint and tighten test
yuneng-jiang Feb 22, 2026
ca9111e
feat: add disable_show_blog to UISettings
yuneng-jiang Feb 22, 2026
70f8e97
feat: add useUISettings and useDisableShowBlog hooks
yuneng-jiang Feb 22, 2026
a82c8c1
fix: rename useUISettings to useUISettingsFlags to avoid naming colli…
yuneng-jiang Feb 22, 2026
98da524
fix: use existing useUISettings hook in useDisableShowBlog to avoid c…
yuneng-jiang Feb 22, 2026
e241e6f
feat: add BlogDropdown component with react-query and error/retry state
yuneng-jiang Feb 22, 2026
1c0cfd7
fix: enforce 5-post limit in BlogDropdown and add cap test
yuneng-jiang Feb 22, 2026
929d592
fix: add retry, stable post key, enabled guard in BlogDropdown
yuneng-jiang Feb 22, 2026
a0965d5
feat: add BlogDropdown to navbar after Docs link
yuneng-jiang Feb 22, 2026
94b76ea
feat: add network_mock transport for benchmarking proxy overhead with…
ryan-crabbe Feb 18, 2026
886f1a3
Litellm dev 02 19 2026 p2 (#21871)
krrishdholakia Feb 22, 2026
9828b99
fix(guardrails/usage_endpoints.py): fix usage dashboard
krrishdholakia Feb 22, 2026
50f36d9
fix(budget): fix timezone config lookup and replace hardcoded timezon…
LeeJuOh Feb 22, 2026
083c899
fix: add missing return type annotations to iterator protocol methods…
WhoisMonesh Feb 22, 2026
66ccbe3
Add gollem Go agent framework cookbook example (#21747)
trevorprater Feb 22, 2026
4fc11f6
fix: avoid mutating caller-owned dicts in SpendUpdateQueue aggregatio…
themavik Feb 22, 2026
bce078a
fix(vertex_ai): enable context-1m-2025-08-07 beta header (#21870)
edwiniac Feb 22, 2026
52585eb
Revert "fix(vertex_ai): enable context-1m-2025-08-07 beta header (#21…
krrishdholakia Feb 22, 2026
0e0abeb
docs(ui): add pre-PR checklist to UI contributing guide
Chesars Feb 22, 2026
22bccc4
Fix entries with fast and us/
Sameerlite Feb 23, 2026
f54fb9a
Add tests for fast and us
Sameerlite Feb 23, 2026
20023bd
Add support for Priority PayGo for vertex ai and gemini
Sameerlite Feb 23, 2026
4d8f509
Add model pricing
Sameerlite Feb 23, 2026
55ee8cd
Merge pull request #21701 from ta-stripe/fix/bedrock-openai-imported-…
Sameerlite Feb 23, 2026
3561bfb
Merge pull request #21640 from ta-stripe/feat/regional-sts-endpoint-f…
Sameerlite Feb 23, 2026
9fc3c77
fix: ensure arrival_time is set before calculating queue time
Harshit28j Feb 23, 2026
4ff1651
Fix: Anthropic model wildcard access issue
Sameerlite Feb 23, 2026
9b27cd8
Add incident report
Sameerlite Feb 23, 2026
262235f
Add ability to see which model cost map is getting used
Sameerlite Feb 23, 2026
eaf3900
Fix name of title
Sameerlite Feb 23, 2026
164cde9
Readd tpm limit
Sameerlite Feb 23, 2026
4c487ac
Merge pull request #21918 from Harshit28j/litellm_arrival_time_fix
Harshit28j Feb 23, 2026
7b5dc3f
State management fixes for CheckBatchCost
ephrimstanley Feb 23, 2026
f23e549
Fix PR review comments
Sameerlite Feb 23, 2026
57af8e6
Merge pull request #21924 from BerriAI/main
Sameerlite Feb 23, 2026
c7aafdf
Merge pull request #21926 from BerriAI/main
Sameerlite Feb 23, 2026
27a3356
State management fixes for CheckBatchCost - Address greptile comments
ephrimstanley Feb 23, 2026
88cd0d3
fix mypy issues:
Sameerlite Feb 23, 2026
99184c4
Add Noma guardrails v2 based on custom guardrails (#21400)
TomAlon Feb 23, 2026
4934d89
Merge pull request #21872 from BerriAI/litellm_dev_02_21_2026_p4
Sameerlite Feb 23, 2026
a3c01b7
Fix code qa issues
Sameerlite Feb 23, 2026
351921d
Fix mypy issues
Sameerlite Feb 23, 2026
16b25bd
Merge pull request #21921 from Point72/ephrimstanley/batch-fixes-feb23
Sameerlite Feb 23, 2026
37d4513
Merge pull request #21917 from BerriAI/litellm_fix_model_cost_map_wil…
Sameerlite Feb 23, 2026
f6f46ae
Fix mypy issues
Sameerlite Feb 23, 2026
8decf04
Merge pull request #21877 from BerriAI/litellm_oss_staging_02_22_2026
Sameerlite Feb 23, 2026
9b5bbee
Merge pull request #21786 from BerriAI/litellm_oss_staging_02_21_2026
Sameerlite Feb 23, 2026
2f8d36b
Fix test_aaamodel_prices_and_context_window_json_is_valid
Sameerlite Feb 23, 2026
61e63b6
Merge pull request #21904 from BerriAI/litellm_fix_model_cost_map
Sameerlite Feb 23, 2026
f97ee62
Merge pull request #21909 from BerriAI/litellm_cost_tracking_gemini
Sameerlite Feb 23, 2026
a26f83f
fix: update calendly on repo
krrishdholakia Feb 23, 2026
fb8b11c
fix(tests): use counter-based mock for time.time in prisma self-heal …
jquinter Feb 23, 2026
ac6f2f3
Merge pull request #21938 from jquinter/fix/flaky-prisma-self-heal-test
jquinter Feb 23, 2026
11a774e
fix(tests): use absolute path for model_prices JSON in validation test
jquinter Feb 23, 2026
a74b6ee
Update tests/test_litellm/test_utils.py
jquinter Feb 23, 2026
bf8c219
fix(tests): use os.path instead of Path to avoid NameError
jquinter Feb 23, 2026
17d97f2
Merge pull request #21939 from jquinter/fix/model-prices-test-relativ…
jquinter Feb 23, 2026
d99d87f
clean up mock transport: remove streaming, add defensive parsing
ryan-crabbe Feb 23, 2026
64d1de0
docs: add Google GenAI SDK tutorial (JS & Python) (#21885)
Chesars Feb 23, 2026
5b41b00
docs: add network mock benchmarking section
ryan-crabbe Feb 23, 2026
8244ad1
docs: tweak benchmarks wording
ryan-crabbe Feb 23, 2026
bba98c2
Merge pull request #21886 from Chesars/docs/ui-contributing-pre-pr-ch…
yuneng-jiang Feb 23, 2026
95d9514
fix: add auth headers and empty latencies guard to benchmark script
ryan-crabbe Feb 23, 2026
fb123ec
refactor: use method-level import for MockOpenAITransport
ryan-crabbe Feb 23, 2026
e0ddb2a
fix: guard print_aggregate against empty latencies
ryan-crabbe Feb 23, 2026
c4c48fe
Merge pull request #21942 from BerriAI/litellm_network_mock
ryan-crabbe Feb 23, 2026
f94d0fe
fix: add INCOMPLETE status to Interactions API enum and test
jquinter Feb 23, 2026
4dfec46
Guardrail Monitor - measure guardrail reliability in prod (#21944)
krrishdholakia Feb 23, 2026
67ceade
docs: add performance & reliability section to v1.81.14 release notes
ryan-crabbe Feb 23, 2026
3681319
Merge pull request #21943 from jquinter/fix/interactions-incomplete-s…
jquinter Feb 23, 2026
bb63de2
fix(tests): make RPM limit test sequential to avoid race condition
jquinter Feb 23, 2026
73fd5a4
feat: Singapore guardrail policies (PDPA + MAS AI Risk Management) (#…
ron-zhong Feb 23, 2026
71b4bd1
Add OpenAI Agents SDK tutorial with LiteLLM Proxy to docs (#21221)
Arindam200 Feb 23, 2026
6574d8d
Merge pull request #21937 from jquinter/fix/flaky-rpm-limit-test
jquinter Feb 23, 2026
0ca9869
Merge pull request #21950 from ryan-crabbe/docs/v1-81-14-perf-section
ryan-crabbe Feb 23, 2026
1ecfbad
adjust blog posts to fetch from github first
yuneng-jiang Feb 23, 2026
4ff2cfd
feat(videos): add variant parameter to video content download (#21955)
nielei3 Feb 23, 2026
94425df
fixing path
yuneng-jiang Feb 23, 2026
08f4a27
Merge remote-tracking branch 'origin' into litellm_blog_dropdown
yuneng-jiang Feb 23, 2026
54b7e1a
adjust blog post path
yuneng-jiang Feb 23, 2026
079ff24
Revert duplicate issue checker to text-based matching, remove duplica…
ryan-crabbe Feb 23, 2026
3faf5e5
Merge pull request #21961 from BerriAI/litellm_revert_duplicate_workf…
ryan-crabbe Feb 23, 2026
09cc3b8
ui changes
yuneng-jiang Feb 23, 2026
9f3fc49
adding tests
yuneng-jiang Feb 24, 2026
a749598
Merge pull request #21859 from BerriAI/litellm_blog_dropdown
yuneng-jiang Feb 24, 2026
02d71bc
fix(anthropic): sanitize tool_use IDs in assistant messages
Canvinus Feb 24, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
blank_issues_enabled: true
contact_links:
- name: Schedule Demo
url: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
url: https://calendly.com/d/cx9p-5yf-2nm/litellm-introductions
about: Speak directly with Krrish and Ishaan, the founders, to discuss issues, share feedback, or explore improvements for LiteLLM
- name: Discord
url: https://discord.com/invite/wuPM9dRgDw
Expand Down
55 changes: 18 additions & 37 deletions .github/workflows/check_duplicate_issues.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,47 +2,28 @@ name: Check Duplicate Issues

on:
issues:
types: [opened]
types: [opened, edited]

jobs:
check-duplicates:
if: github.event.action == 'opened'
check-duplicate:
runs-on: ubuntu-latest
permissions:
contents: read
issues: write
contents: read
steps:
- name: Install Claude Code
run: npm install -g @anthropic-ai/claude-code

- name: Check duplicates
env:
ANTHROPIC_API_KEY: ${{ secrets.LITELLM_VIRTUAL_KEY }}
ANTHROPIC_BASE_URL: ${{ secrets.LITELLM_BASE_URL }}
- name: Check for potential duplicates
uses: wow-actions/potential-duplicates@v1
with:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PROMPT: |
A new issue has been created in the ${{ github.repository }} repository.

Issue number: ${{ github.event.issue.number }}

Lookup this issue with gh issue view ${{ github.event.issue.number }} --repo ${{ github.repository }}.

Search through existing issues (excluding #${{ github.event.issue.number }}) to find potential duplicates.

Use gh issue list --repo ${{ github.repository }} with relevant search terms from the new issue's title and description. Try multiple keyword combinations to search broadly. Check both open and recently closed issues.

Consider:
1. Similar titles or descriptions
2. Same error messages or symptoms
3. Related functionality or components
4. Similar feature requests

If you find potential duplicates, post a SINGLE comment on issue #${{ github.event.issue.number }} using gh issue comment ${{ github.event.issue.number }} --repo ${{ github.repository }} with this format:

_This comment was generated by an LLM and may be inaccurate._

This issue might be a duplicate of existing issues. Please check:
- #[issue_number]: [brief description of similarity]

If you find NO duplicates, do NOT post any comment. Stay silent.
run: claude -p "$PROMPT" --model sonnet --max-turns 10 --allowedTools "Bash(gh issue *)"
label: potential-duplicate
threshold: 0.6
reaction: eyes
comment: |
**⚠️ Potential duplicate detected**

This issue appears similar to existing issue(s):
{{#issues}}
- [#{{number}}]({{html_url}}) - {{title}} ({{accuracy}}% similar)
{{/issues}}

Please review the linked issue(s) to see if they address your concern. If this is not a duplicate, please provide additional context to help us understand the difference.
52 changes: 0 additions & 52 deletions .github/workflows/check_duplicate_prs.yml

This file was deleted.

2 changes: 1 addition & 1 deletion .github/workflows/interpret_load_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ def get_docker_run_command(release_version):
+ docker_run_command
+ "\n\n"
+ "### Don't want to maintain your internal proxy? get in touch 🎉"
+ "\nHosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat"
+ "\nHosted Proxy Alpha: https://calendly.com/d/cx9p-5yf-2nm/litellm-introductions"
+ "\n\n"
+ "## Load Test LiteLLM Proxy Results"
+ "\n\n"
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -399,7 +399,7 @@ Support for more providers. Missing a provider or LLM Platform, raise a [feature
# Enterprise
For companies that need better security, user management and professional support

[Talk to founders](https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat)
[Talk to founders](https://calendly.com/d/cx9p-5yf-2nm/litellm-introductions)

This covers:
- ✅ **Features under the [LiteLLM Commercial License](https://docs.litellm.ai/docs/proxy/enterprise):**
Expand Down
2 changes: 1 addition & 1 deletion cookbook/benchmark/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,4 +178,4 @@ Benchmark Results for 'When will BerriAI IPO?':
```

## Support
**🤝 Schedule a 1-on-1 Session:** Book a [1-on-1 session](https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat) with Krrish and Ishaan, the founders, to discuss any issues, provide feedback, or explore how we can improve LiteLLM for you.
**🤝 Schedule a 1-on-1 Session:** Book a [1-on-1 session](https://calendly.com/d/cx9p-5yf-2nm/litellm-introductions) with Krrish and Ishaan, the founders, to discuss any issues, provide feedback, or explore how we can improve LiteLLM for you.
119 changes: 119 additions & 0 deletions cookbook/gollem_go_agent_framework/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
# Gollem Go Agent Framework with LiteLLM

A working example showing how to use [gollem](https://github.com/fugue-labs/gollem), a production-grade Go agent framework, with LiteLLM as a proxy gateway. This lets Go developers access 100+ LLM providers through a single proxy while keeping compile-time type safety for tools and structured output.

## Quick Start

### 1. Start LiteLLM Proxy

```bash
# Simple start with a single model
litellm --model gpt-4o

# Or with the example config for multi-provider access
litellm --config proxy_config.yaml
```

### 2. Run the examples

```bash
# Install Go dependencies
go mod tidy

# Basic agent
go run ./basic

# Agent with type-safe tools
go run ./tools

# Streaming responses
go run ./streaming
```

## Configuration

The included `proxy_config.yaml` sets up three providers through LiteLLM:

```yaml
model_list:
- model_name: gpt-4o # OpenAI
- model_name: claude-sonnet # Anthropic
- model_name: gemini-pro # Google Vertex AI
```

Switch providers in Go by changing a single string — no code changes needed:

```go
model := openai.NewLiteLLM("http://localhost:4000",
openai.WithModel("gpt-4o"), // OpenAI
// openai.WithModel("claude-sonnet"), // Anthropic
// openai.WithModel("gemini-pro"), // Google
)
```

## Examples

### `basic/` — Basic Agent

Connects gollem to LiteLLM and runs a simple prompt. Demonstrates the `NewLiteLLM` constructor and basic agent creation.

### `tools/` — Type-Safe Tools

Shows gollem's compile-time type-safe tool framework working through LiteLLM's tool-use passthrough. The tool parameters are Go structs with JSON tags — the schema is generated automatically at compile time.

### `streaming/` — Streaming Responses

Real-time token streaming using Go 1.23+ range-over-function iterators, proxied through LiteLLM's SSE passthrough.

## How It Works

Gollem's `openai.NewLiteLLM()` constructor creates an OpenAI-compatible provider pointed at your LiteLLM proxy. Since LiteLLM speaks the OpenAI API protocol, everything works out of the box:

- **Chat completions** — standard request/response
- **Tool use** — LiteLLM passes tool definitions and calls through transparently
- **Streaming** — Server-Sent Events proxied through LiteLLM
- **Structured output** — JSON schema response format works with supporting models

```
Go App (gollem) → LiteLLM Proxy → OpenAI / Anthropic / Google / ...
```

## Why Use This?

- **Type-safe Go**: Compile-time type checking for tools, structured output, and agent configuration — no runtime surprises
- **Single proxy, many models**: Switch between OpenAI, Anthropic, Google, and 100+ other providers by changing a model name string
- **Zero-dependency core**: gollem's core has no external dependencies — just stdlib
- **Single binary deployment**: `go build` produces one binary, no pip/venv/Docker needed
- **Cost tracking & rate limiting**: LiteLLM handles cost tracking, rate limits, and fallbacks at the proxy layer

## Environment Variables

```bash
# Required for providers you want to use (set in LiteLLM config or env)
export OPENAI_API_KEY="sk-..."
export ANTHROPIC_API_KEY="sk-ant-..."

# Optional: point to a non-default LiteLLM proxy
export LITELLM_PROXY_URL="http://localhost:4000"
```

## Troubleshooting

**Connection errors?**
- Make sure LiteLLM is running: `litellm --model gpt-4o`
- Check the URL is correct (default: `http://localhost:4000`)

**Model not found?**
- Verify the model name matches what's configured in LiteLLM
- Run `curl http://localhost:4000/models` to see available models

**Tool calls not working?**
- Ensure the underlying model supports tool use (GPT-4o, Claude, Gemini)
- Check LiteLLM logs for any provider-specific errors

## Learn More

- [gollem GitHub](https://github.com/fugue-labs/gollem)
- [gollem API Reference](https://pkg.go.dev/github.com/fugue-labs/gollem/core)
- [LiteLLM Proxy Docs](https://docs.litellm.ai/docs/simple_proxy)
- [LiteLLM Supported Models](https://docs.litellm.ai/docs/providers)
41 changes: 41 additions & 0 deletions cookbook/gollem_go_agent_framework/basic/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
// Basic gollem agent connected to a LiteLLM proxy.
//
// Usage:
//
// litellm --model gpt-4o # start proxy in another terminal
// go run ./basic
package main

import (
"context"
"fmt"
"log"
"os"

"github.com/fugue-labs/gollem/core"
"github.com/fugue-labs/gollem/provider/openai"
)

func main() {
proxyURL := "http://localhost:4000"
if u := os.Getenv("LITELLM_PROXY_URL"); u != "" {
proxyURL = u
}

// Connect to LiteLLM proxy. NewLiteLLM creates an OpenAI-compatible
// provider pointed at the given URL.
model := openai.NewLiteLLM(proxyURL,
openai.WithModel("gpt-4o"), // any model name configured in LiteLLM
)

// Create and run a simple agent.
agent := core.NewAgent[string](model,
core.WithSystemPrompt[string]("You are a helpful assistant. Be concise."),
)

result, err := agent.Run(context.Background(), "Explain quantum computing in two sentences.")
if err != nil {
log.Fatal(err)
}
fmt.Println(result.Output)
}
5 changes: 5 additions & 0 deletions cookbook/gollem_go_agent_framework/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
module github.com/BerriAI/litellm/cookbook/gollem_go_agent_framework

go 1.25.1

require github.com/fugue-labs/gollem v0.1.0
2 changes: 2 additions & 0 deletions cookbook/gollem_go_agent_framework/go.sum
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
github.com/fugue-labs/gollem v0.1.0 h1:QexYnvkb44QZFEljgAePqMIGZjgsbk0Y5GJ2jYYgfa8=
github.com/fugue-labs/gollem v0.1.0/go.mod h1:htW1YO81uysSKVOkYJtxhGCFrzm+36HBFxEWuECoHKQ=
16 changes: 16 additions & 0 deletions cookbook/gollem_go_agent_framework/proxy_config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
model_list:
- model_name: gpt-4o
litellm_params:
model: openai/gpt-4o
api_key: os.environ/OPENAI_API_KEY

- model_name: claude-sonnet
litellm_params:
model: anthropic/claude-sonnet-4-20250514
api_key: os.environ/ANTHROPIC_API_KEY

- model_name: gemini-pro
litellm_params:
model: vertex_ai/gemini-2.0-flash
vertex_project: my-project
vertex_location: us-central1
56 changes: 56 additions & 0 deletions cookbook/gollem_go_agent_framework/streaming/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
// Streaming responses from gollem through LiteLLM.
//
// Uses Go 1.23+ range-over-function iterators for real-time token
// streaming via LiteLLM's SSE passthrough.
//
// Usage:
//
// litellm --model gpt-4o
// go run ./streaming
package main

import (
"context"
"fmt"
"log"
"os"

"github.com/fugue-labs/gollem/core"
"github.com/fugue-labs/gollem/provider/openai"
)

func main() {
proxyURL := "http://localhost:4000"
if u := os.Getenv("LITELLM_PROXY_URL"); u != "" {
proxyURL = u
}

model := openai.NewLiteLLM(proxyURL,
openai.WithModel("gpt-4o"),
)

agent := core.NewAgent[string](model)

// RunStream returns a streaming result that yields tokens as they arrive.
stream, err := agent.RunStream(context.Background(), "Write a haiku about distributed systems")
if err != nil {
log.Fatal(err)
}

// StreamText yields text chunks in real-time.
// The boolean argument controls whether deltas (true) or accumulated
// text (false) is returned.
fmt.Print("Response: ")
for text, err := range stream.StreamText(true) {
if err != nil {
log.Fatal(err)
}
fmt.Print(text)
}
fmt.Println()

// After streaming completes, the final response is available.
resp := stream.Response()
fmt.Printf("\nTokens used: input=%d, output=%d\n",
resp.Usage.InputTokens, resp.Usage.OutputTokens)
}
Loading
Loading