Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion .github/workflows/scripts/cost-accuracy-test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -357,11 +357,22 @@ params = {
"order": "asc",
}

def logs_complete(logs):
# Log writes are fully async (single batched insert in PostLLMHook), so a
# row can be visible before its usage/cost are readable. Poll on the
# predicate we assert (every row has usage and cost), not just row count.
return all(
(item.get("token_usage") or {}).get("prompt_tokens") is not None
and (item.get("token_usage") or {}).get("completion_tokens") is not None
and item.get("cost") is not None
for item in logs
)

logs = []
for _ in range(60):
payload = get_json("/api/logs", params)
logs = payload.get("logs", [])
if len(logs) >= expected_count:
if len(logs) >= expected_count and logs_complete(logs):
break
time.sleep(1)

Expand Down
301 changes: 251 additions & 50 deletions .github/workflows/scripts/run-migration-tests.sh

Large diffs are not rendered by default.

14 changes: 7 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ curl -X POST http://localhost:8080/v1/chat/completions \
## Enterprise Deployments

Bifrost supports enterprise-grade, private deployments for teams running production AI systems at scale.
In addition to private networking, custom security controls, and governance, enterprise deployments unlock advanced capabilities including adaptive load balancing, clustering, guardrails, MCP gateway and and other features designed for enterprise-grade scale and reliability.
In addition to private networking, custom security controls, and governance, enterprise deployments unlock advanced capabilities including adaptive load balancing, clustering, guardrails, MCP gateway, and other features designed for enterprise-grade scale and reliability.

<img src=".github/assets/features.png" alt="Book a Demo" width="100%" style="margin-top:5px;"/>

Expand Down Expand Up @@ -88,7 +88,7 @@ In addition to private networking, custom security controls, and governance, ent

- **[Model Context Protocol (MCP)](https://docs.getbifrost.ai/mcp/overview)** - Enable AI models to use external tools (filesystem, web search, databases)
- **[Semantic Caching](https://docs.getbifrost.ai/features/semantic-caching)** - Intelligent response caching based on semantic similarity to reduce costs and latency
- **[Multimodal Support](https://docs.getbifrost.ai/quickstart/gateway/streaming)** - Support for text,images, audio, and streaming, all behind a common interface.
- **[Multimodal Support](https://docs.getbifrost.ai/quickstart/gateway/streaming)** - Support for text, images, audio, and streaming, all behind a common interface.
- **[Custom Plugins](https://docs.getbifrost.ai/enterprise/custom-plugins)** - Extensible middleware architecture for analytics, monitoring, and custom logic
- **[Governance](https://docs.getbifrost.ai/features/governance/virtual-keys)** - Usage tracking, rate limiting, and fine-grained access control

Expand Down Expand Up @@ -120,8 +120,8 @@ bifrost/
│ ├── schemas/ # Interfaces and structs used throughout Bifrost
│ └── bifrost.go # Main Bifrost implementation
├── framework/ # Framework components for data persistence
│ ├── configstore/ # Configuration storages
│ ├── logstore/ # Request logging storages
│ ├── configstore/ # Configuration storage backends
│ ├── logstore/ # Request logging storage backends
│ └── vectorstore/ # Vector storages
├── transports/ # HTTP gateway and other interface layers
│ └── bifrost-http/ # HTTP transport implementation
Expand Down Expand Up @@ -181,7 +181,7 @@ go get github.com/maximhq/bifrost/core
- base_url = "https://api.openai.com"
+ base_url = "http://localhost:8080/openai"

# Anthropic SDK
# Anthropic SDK
- base_url = "https://api.anthropic.com"
+ base_url = "http://localhost:8080/anthropic"

Expand Down Expand Up @@ -241,7 +241,7 @@ Bifrost adds virtually zero overhead to your AI requests. In sustained 5,000 RPS
- [AWS Bedrock SDK](https://docs.getbifrost.ai/integrations/bedrock-sdk/overview) - AWS Bedrock integration
- [Google GenAI SDK](https://docs.getbifrost.ai/integrations/genai-sdk/overview) - Drop-in GenAI replacement
- [LiteLLM SDK](https://docs.getbifrost.ai/integrations/litellm-sdk) - LiteLLM integration
- [Langchain SDK](https://docs.getbifrost.ai/integrations/langchain-sdk) - Langchain integration
- [LangChain SDK](https://docs.getbifrost.ai/integrations/langchain-sdk) - LangChain integration

### Enterprise

Expand All @@ -259,7 +259,7 @@ Bifrost adds virtually zero overhead to your AI requests. In sustained 5,000 RPS
Get help with:

- Quick setup assistance and troubleshooting
- Best practices and configuration tips
- Best practices and configuration tips
- Community discussions and support
- Real-time help with integrations

Expand Down
42 changes: 42 additions & 0 deletions THIRD_PARTY_NOTICES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# Third-Party Notices

Bifrost is licensed under the [Apache License, Version 2.0](LICENSE). This file lists third-party components with license terms beyond the common MIT/BSD/Apache-2.0 permissive set — specifically, code carrying Mozilla Public License 2.0 (MPL-2.0) terms — along with third-party source code embedded directly in this repository. All components below are used unmodified and combined as a "Larger Work" per MPL-2.0 Section 3.3; no Bifrost source files are themselves MPL-licensed.

This file does not enumerate the full dependency tree (see `go.sum` / `package-lock.json` in each module for that) — only the entries that need attribution beyond what those permissive licenses require.

## Embedded source code

### `framework/migrator/migrator.go`

Portions of this file are derived from [go-gormigrate/gormigrate](https://github.com/go-gormigrate/gormigrate).

```
MIT License
Copyright (c) 2016 Andrey Nering
```

Full license text is preserved in the file's header comment.

## MPL-2.0 components — Go (compiled into the binary)

### `github.com/cyphar/filepath-securejoin`

Dual-licensed: most files are BSD-3-Clause; a subset of files (see the package's own `COPYING.md`) are MPL-2.0. Used unmodified as an upstream dependency. Source: https://github.com/cyphar/filepath-securejoin

### `github.com/hashicorp/go-version`

Licensed under MPL-2.0. Copyright IBM Corp. Used unmodified as an upstream dependency. Source: https://github.com/hashicorp/go-version

## MPL-2.0 / dual-licensed components — npm (build tooling only, not shipped)

### `lightningcss`

Licensed under MPL-2.0. Pulled in transitively via `vite`/`tailwindcss` as a build-time devDependency — it runs during the UI build and is never bundled into the built output shipped to end users. Source: https://github.com/parcel-bundler/lightningcss

### `dompurify`

Dual-licensed `MPL-2.0 OR Apache-2.0`. Bifrost elects the **Apache-2.0** option; no MPL-2.0 obligations apply to Bifrost's use of this package. Source: https://github.com/cure53/DOMPurify

---

*This file covers MPL-2.0 attribution only. It is not a complete open-source license inventory and is not a substitute for legal review.*
138 changes: 138 additions & 0 deletions cmd/e2eseed/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
module github.com/maximhq/bifrost/cmd/e2eseed

go 1.26.5

replace (
github.com/maximhq/bifrost/core => ../../core
github.com/maximhq/bifrost/framework => ../../framework
)

require (
github.com/maximhq/bifrost/core v1.7.5
github.com/maximhq/bifrost/framework v1.3.15
gorm.io/driver/postgres v1.6.0
gorm.io/driver/sqlite v1.6.0
gorm.io/gorm v1.31.1
)

require (
cel.dev/expr v0.25.1 // indirect
cloud.google.com/go v0.123.0 // indirect
cloud.google.com/go/auth v0.20.0 // indirect
cloud.google.com/go/auth/oauth2adapt v0.2.8 // indirect
cloud.google.com/go/compute/metadata v0.9.0 // indirect
cloud.google.com/go/iam v1.7.0 // indirect
cloud.google.com/go/monitoring v1.24.3 // indirect
cloud.google.com/go/storage v1.62.1 // indirect
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.20.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.13.1 // indirect
github.com/Azure/azure-sdk-for-go/sdk/internal v1.11.2 // indirect
github.com/AzureAD/microsoft-authentication-library-for-go v1.6.0 // indirect
github.com/ClickHouse/ch-go v0.65.0 // indirect
github.com/ClickHouse/clickhouse-go/v2 v2.32.0 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.32.0 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.55.0 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.55.0 // indirect
github.com/andybalholm/brotli v1.2.2 // indirect
github.com/aws/aws-sdk-go-v2 v1.42.0 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.10 // indirect
github.com/aws/aws-sdk-go-v2/config v1.32.11 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.19.14 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.21 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.29 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.29 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.5 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.22 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.7 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.13 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.21 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.21 // indirect
github.com/aws/aws-sdk-go-v2/service/s3 v1.97.3 // indirect
github.com/aws/aws-sdk-go-v2/service/signin v1.0.9 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.30.15 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.19 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.41.10 // indirect
github.com/aws/smithy-go v1.27.1 // indirect
github.com/bahlo/generic-list-go v0.2.0 // indirect
github.com/buger/jsonparser v1.1.2 // indirect
github.com/bytedance/gopkg v0.1.3 // indirect
github.com/bytedance/sonic v1.15.1 // indirect
github.com/bytedance/sonic/loader v0.5.1 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/cloudwego/base64x v0.1.6 // indirect
github.com/cncf/xds/go v0.0.0-20260202195803-dba9d589def2 // indirect
github.com/envoyproxy/go-control-plane/envoy v1.37.0 // indirect
github.com/envoyproxy/protoc-gen-validate v1.3.3 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/go-faster/city v1.0.1 // indirect
github.com/go-faster/errors v0.7.1 // indirect
github.com/go-jose/go-jose/v4 v4.1.4 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/golang-jwt/jwt/v5 v5.3.1 // indirect
github.com/google/s2a-go v0.1.9 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.16 // indirect
github.com/googleapis/gax-go/v2 v2.22.0 // indirect
github.com/hashicorp/go-version v1.8.0 // indirect
github.com/invopop/jsonschema v0.13.0 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/pgx/v5 v5.9.2 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect
github.com/jinzhu/inflection v1.0.0 // indirect
github.com/jinzhu/now v1.1.5 // indirect
github.com/klauspost/compress v1.18.6 // indirect
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
github.com/kylelemons/godebug v1.1.0 // indirect
github.com/mailru/easyjson v0.9.1 // indirect
github.com/mark3labs/mcp-go v0.43.2 // indirect
github.com/mattn/go-colorable v0.1.14 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mattn/go-sqlite3 v1.14.32 // indirect
github.com/paulmach/orb v0.11.1 // indirect
github.com/pierrec/lz4/v4 v4.1.22 // indirect
github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 // indirect
github.com/rs/zerolog v1.34.0 // indirect
github.com/segmentio/asm v1.2.0 // indirect
github.com/shopspring/decimal v1.4.0 // indirect
github.com/spf13/cast v1.10.0 // indirect
github.com/spiffe/go-spiffe/v2 v2.6.0 // indirect
github.com/tidwall/gjson v1.18.0 // indirect
github.com/tidwall/match v1.1.1 // indirect
github.com/tidwall/pretty v1.2.1 // indirect
github.com/tidwall/sjson v1.2.5 // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
github.com/valyala/bytebufferpool v1.0.0 // indirect
github.com/valyala/fasthttp v1.71.0 // indirect
github.com/wk8/go-ordered-map/v2 v2.1.8 // indirect
github.com/yosida95/uritemplate/v3 v3.0.2 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/detectors/gcp v1.43.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.67.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.67.0 // indirect
go.opentelemetry.io/otel v1.43.0 // indirect
go.opentelemetry.io/otel/metric v1.43.0 // indirect
go.opentelemetry.io/otel/sdk v1.43.0 // indirect
go.opentelemetry.io/otel/sdk/metric v1.43.0 // indirect
go.opentelemetry.io/otel/trace v1.43.0 // indirect
go.starlark.net v0.0.0-20260102030733-3fee463870c9 // indirect
golang.org/x/arch v0.23.0 // indirect
golang.org/x/crypto v0.52.0 // indirect
golang.org/x/net v0.55.0 // indirect
golang.org/x/oauth2 v0.36.0 // indirect
golang.org/x/sync v0.20.0 // indirect
golang.org/x/sys v0.45.0 // indirect
golang.org/x/text v0.37.0 // indirect
golang.org/x/time v0.15.0 // indirect
google.golang.org/api v0.282.0 // indirect
google.golang.org/genproto v0.0.0-20260319201613-d00831a3d3e7 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20260414002931-afd174a4e478 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260523011958-0a33c5d7ca68 // indirect
google.golang.org/grpc v1.82.1 // indirect
google.golang.org/protobuf v1.36.12-0.20260120151049-f2248ac996af // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
gorm.io/driver/clickhouse v0.7.0 // indirect
)
Loading