TensorZero is an open-source stack for industrial-grade LLM applications. It unifies an LLM gateway, observability, optimization, evaluation, and experimentation.

⚙️🦀 Build modular and scalable LLM Applications in Rust

The smart edge and AI gateway for agents. Arch is a high-performance proxy server that handles the low-level work in building agents: like applying guardrails, routing prompts to the right agent, and unifying access to LLMs, etc. Natively designed to handle and process prompts, Arch helps you build agents faster.

Bionic is an on-premise replacement for ChatGPT, offering the advantages of Generative AI while maintaining strict data confidentiality

AICI: Prompts as (Wasm) Programs

A reactive runtime for building durable AI agents

Open-source LLMOps platform for hosting and scaling AI in your own infrastructure 🏓🦙

Scalable, fast, and disk-friendly vector search in Postgres, the successor of pgvecto.rs.

🧠 Motorhead is a memory and information retrieval server for LLMs.

Fast, streaming indexing, query, and agentic LLM applications in Rust

Govern & Secure your AI

High-scale LLM gateway, written in Rust. OpenTelemetry-based observability included

Simple, Composable, High-Performance, Safe and Web3 Friendly AI Agents and LazAI Gateway for Everyone

🧭🧭 An intelligent load balancer with smart scheduling that unifies diverse LLMs.

Robot VLM and VLA (Vision-Language-Action) inference API helping you manage multimodal prompts, RAG, and location metadata

Burgonet Gateway is an enterprise LLM gateway that provides secure access and compliance controls for AI systems

Prompt-centric framework for developing LLM applications in Rust

A Lazy, high throughput and blazing fast structured text generation backend.

[WIP] Sorai is a lightweight, high-performance, and open-source LLM proxy gateway.

Model executor runtime 💬 written in Rust