Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Bifrost documentation

For better accessibility we have moved documentation [here](https://www.getmaxim.ai/bifrost/docs).
Comment thread
akshaydeo marked this conversation as resolved.
151 changes: 0 additions & 151 deletions docs/README.mdx

This file was deleted.

Empty file added docs/advanced/api-reference.mdx
Empty file.
Empty file.
145 changes: 0 additions & 145 deletions docs/architecture/README.mdx
Original file line number Diff line number Diff line change
@@ -1,145 +0,0 @@
---
title: "🏗️ Bifrost Architecture"
description: "Deep dive into Bifrost's system architecture - designed for **10,000+ RPS** with advanced concurrency management, memory optimization, and extensible plugin architecture."
---

# 🏗️ Bifrost Architecture

Deep dive into Bifrost's system architecture - designed for **10,000+ RPS** with advanced concurrency management, memory optimization, and extensible plugin architecture.

---

## 📑 Architecture Navigation

### **🎯 Core Architecture**

| Document | Description | Focus Area |
| ---------------------------------------------- | ------------------------------------------- | ---------------------------------------- |
| **[🌐 System Overview](./system-overview)** | High-level architecture & design principles | Components, interactions, data flow |
| **[🔄 Request Flow](./request-flow)** | Request processing pipeline deep dive | Processing stages, memory management |
| **[📊 Benchmarks](../benchmarks)** | Performance benchmarks & optimization | Metrics, scaling, optimization |
| **[⚙️ Concurrency](./concurrency)** | Worker pools & threading model | Goroutines, channels, resource isolation |

### **🔧 Internal Systems**

| Document | Description | Focus Area |
| ------------------------------------------------ | ----------------------------------- | --------------------------------------- |
| **[🔌 Plugin System](./plugins)** | How plugins work internally | Plugin lifecycle, interfaces, execution |
| **[🛠️ MCP System](./mcp)** | Model Context Protocol internals | Tool discovery, execution, integration |
| **[💡 Design Decisions](./design-decisions)** | Architecture rationale & trade-offs | Why we built it this way, alternatives |

---

## 🚀 Quick Start by Role

### **🔧 System Administrators**

1. **[System Overview](./system-overview)** - Deployment architecture
2. **[Benchmarks](../benchmarks)** - Scaling and capacity planning
3. **[Concurrency](./concurrency)** - Resource tuning parameters

### **👨‍💻 Backend Developers**

1. **[Request Flow](./request-flow)** - Processing pipeline internals
2. **[Plugin System](./plugins)** - Extension mechanisms
3. **[Design Decisions](./design-decisions)** - Implementation rationale

### **🏗️ Platform Engineers**

1. **[Benchmarks](../benchmarks)** - Throughput and optimization
2. **[Concurrency](./concurrency)** - Resource allocation strategies
3. **[System Overview](./system-overview)** - Integration architecture

### **🔌 Plugin Developers**

1. **[Plugin System](./plugins)** - Internal plugin architecture
2. **[Request Flow](./request-flow)** - Hook points and data flow
3. **[MCP System](./mcp)** - Tool integration patterns

---

## 🏗️ Architecture at a Glance

### **High-Performance Design Principles**

- **🔄 Asynchronous Processing** - Channel-based worker pools eliminate blocking
- **💾 Memory Pool Management** - Object reuse minimizes garbage collection
- **🏗️ Provider Isolation** - Independent resources prevent cascade failures
- **🔌 Plugin-First Architecture** - Extensible without core modifications
- **⚡ Connection Optimization** - HTTP/2, keep-alive, intelligent pooling

### **System Components Overview**

**Processing Flow:** Transport → Router → Plugins → MCP → Workers → Providers

### **Key Performance Characteristics**

| Metric | Performance | Details |
| ------------------ | ----------------- | ---------------------------------- |
| **🚀 Throughput** | 10,000+ RPS | Sustained high-load performance |
| **⚡ Latency** | 11-59μs overhead | Minimal processing overhead |
| **💾 Memory** | Optimized pooling | Object reuse minimizes GC pressure |
| **🎯 Reliability** | 100% success rate | Under 5000 RPS sustained load |

### **Architectural Features**

- **🔄 Provider Isolation** - Independent worker pools prevent cascade failures
- **💾 Memory Optimization** - Channel, message, and response object pooling
- **🎣 Extensible Hooks** - Plugin system for custom logic injection
- **🛠️ MCP Integration** - Native tool discovery and execution system
- **📊 Built-in Observability** - Prometheus metrics without external dependencies

---

## 📚 Core Concepts

### **Request Lifecycle**

1. **Transport** receives request (HTTP/SDK)
2. **Router** selects provider and manages load balancing
3. **Plugin Manager** executes pre-processing hooks
4. **MCP Manager** discovers and prepares available tools
5. **Worker Pool** processes request with dedicated provider workers
6. **Memory Pools** provide reusable objects for efficiency
7. **Plugin Manager** executes post-processing hooks
8. **Transport** returns response to client

### **Scaling Strategies**

- **Vertical Scaling** - Increase pool sizes and buffer capacities
- **Horizontal Scaling** - Deploy multiple instances with load balancing
- **Provider Scaling** - Independent worker pools per provider
- **Memory Scaling** - Configurable object pool sizes

### **Extension Points**

- **Plugin Hooks** - Pre/post request processing
- **Custom Providers** - Add new AI service integrations
- **MCP Tools** - External tool integration
- **Transport Layers** - Multiple interface options (HTTP, SDK, gRPC planned)

---

## 🔗 Related Documentation

### **Usage Documentation**

- **[🚀 Quick Start](../quickstart/README)** - Get started with Bifrost
- **[🌐 HTTP Transport](../usage/http-transport/README)** - HTTP API usage
- **[📦 Go Package](../usage/go-package/README)** - Go SDK usage

### **Configuration**

- **[🔧 Provider Setup](../usage/http-transport/configuration/providers)** - Provider configuration
- **[🔌 Plugin Setup](../usage/http-transport/configuration/plugins)** - Plugin configuration
- **[🛠️ MCP Setup](../usage/http-transport/configuration/mcp)** - MCP configuration

### **Operations**

- **[📊 Monitoring](../usage/monitoring)** - Observability and metrics
- **[🔐 Security](../usage/key-management)** - Key management and security
- **[🌐 Networking](../usage/networking)** - Network configuration

---

**💡 New to Bifrost architecture?** Start with **[System Overview](./system-overview)** for the complete picture, then dive into **[Request Flow](./request-flow)** to understand how it all works together.
Loading
Loading