Modern Multi-LLM Orchestration Framework for Kotlin
π¨ MAJOR UPDATE COMING: v0.5.0 - Agent Framework Revolution
We're rebuilding Spice with a Microsoft Agent Framework-inspired architecture: unified graph-based runtime, OpenTelemetry-native observability, checkpoint/replay, and middleware pipeline.
β οΈ Breaking Changes Ahead | π Target: Week 10 | π Read the Roadmap β
- Current users: 0.4.x will have 6 months LTS support
 - Migration tools: Auto-migration CLI provided
 - Early access: 0.5.0-beta coming Week 8
 Join the discussion: GitHub Discussions | Discord
Spice Framework is a modern, type-safe, coroutine-first framework for building AI-powered applications in Kotlin. It provides a clean DSL for creating agents, managing tools, and orchestrating complex AI workflows with multiple LLM providers.
- π Simple yet Powerful - Get started in minutes, scale to complex multi-agent systems
 - π§ Type-Safe - Leverage Kotlin's type system for compile-time safety
 - π Async-First - Built on coroutines for efficient concurrent operations
 - π¨ Clean DSL - Intuitive API that reads like natural language
 - π Extensible - Easy to add custom agents, tools, and integrations
 
- Unified Communication - Single 
Commtype for all agent interactions - Generic Registry System - Type-safe, thread-safe component management
 - Progressive Disclosure - Simple things simple, complex things possible
 - Tool System - Built-in tools and easy custom tool creation
 - JSON Serialization - Production-grade JSON conversion for all components
 
- π Thread-Safe Context Propagation β NEW in v0.4.0 - Automatic context flow through coroutines for multi-tenant systems
 - Multi-LLM Support - OpenAI, Anthropic, Google Vertex AI, and more
 - Swarm Intelligence - Coordinate multiple agents with 5 strategies
 - AI-Powered Coordinator - LLM-enhanced meta-coordination for swarms
 - OpenTelemetry Integration - Production-grade observability with distributed tracing
 - Vector Store Integration - Built-in RAG support with multiple providers
 - MCP Protocol - External tool integration via Model Context Protocol
 - Spring Boot Starter - Seamless Spring Boot integration
 - JSON Schema Support - Export tools as standard JSON Schema for GUI/API integration
 - PSI (Program Structure Interface) - Convert DSL to LLM-friendly tree structures
 
- β Automatic Multi-Tenancy (v0.4.0) - Perfect tenant isolation with zero boilerplate via coroutine context
 - Context-Aware Tools & Services (v0.4.0) - Automatic tenantId/userId injection in all operations
 - Context Extension System (v0.4.0) - Runtime context enrichment with plugins
 - Dynamic Platform Management - Register and manage platforms at runtime
 - Policy Versioning - Version control for policies with rollback capability
 - Event Sourcing - Complete event sourcing module with Kafka integration
 - Event-Driven Architecture - Kafka integration for distributed systems
 - Advanced Monitoring - Built-in metrics and performance tracking
 
Add JitPack repository to your build file:
// settings.gradle.kts
dependencyResolutionManagement {
    repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
    repositories {
        mavenCentral()
        maven { url = uri("https://jitpack.io") }
    }
}Then add the dependency:
// build.gradle.kts
dependencies {
    implementation("com.github.no-ai-labs.spice-framework:spice-core:Tag")
    
    // Optional modules
    implementation("com.github.no-ai-labs.spice-framework:spice-springboot:Tag")
    implementation("com.github.no-ai-labs.spice-framework:spice-eventsourcing:Tag")
}Replace Tag with the latest version: 
import io.github.spice.*
import io.github.spice.dsl.*
import kotlinx.coroutines.runBlocking
fun main() = runBlocking {
    // Create a simple agent
    val assistant = buildAgent {
        id = "assistant-1"
        name = "AI Assistant"
        description = "A helpful AI assistant"
        
        // Add an inline tool
        tool("greet") {
            description = "Greet someone"
            parameter("name", "string", "Person's name")
            execute { params ->
                "Hello, ${params["name"]}! How can I help you today?"
            }
        }
        
        // Define message handling
        handle { comm ->
            when {
                comm.content.startsWith("greet ") -> {
                    val name = comm.content.removePrefix("greet ").trim()
                    val result = run("greet", mapOf("name" to name))
                    comm.reply(result.result.toString(), id)
                }
                else -> comm.reply("Say 'greet NAME' to get a greeting!", id)
            }
        }
    }
    
    // Use the agent
    val response = assistant.processComm(
        Comm(content = "greet Alice", from = "user")
    )
    println(response.content) // "Hello, Alice! How can I help you today?"
}// OpenAI Integration
val gptAgent = buildOpenAIAgent {
    id = "gpt-4"
    name = "GPT-4 Assistant"
    apiKey = System.getenv("OPENAI_API_KEY")
    model = "gpt-4"
    systemPrompt = "You are a helpful coding assistant."
}
// Anthropic Integration
val claudeAgent = buildClaudeAgent {
    id = "claude-3"
    name = "Claude Assistant"
    apiKey = System.getenv("ANTHROPIC_API_KEY")
    model = "claude-3-opus-20240229"
}
// Use them just like any other agent
val response = gptAgent.processComm(
    Comm(content = "Explain coroutines in Kotlin", from = "user")
)βββββββββββββββββββββββββββββββββββββββββββββββββββ
β                Your Application                  β
βββββββββββββββββββββββββββββββββββββββββββββββββββ€
β                  Spice DSL                      β
β         buildAgent { } β’ buildFlow { }          β
βββββββββββββββββββββββββββββββββββββββββββββββββββ€
β                 Core Layer                      β
β    Agent β’ Comm β’ Tool β’ Registry System        β
βββββββββββββββββββββββββββββββββββββββββββββββββββ€
β              Integration Layer                  β
β    LLMs β’ Vector Stores β’ MCP β’ Spring Boot    β
βββββββββββββββββββββββββββββββββββββββββββββββββββ
- Registry Pattern - Centralized management of agents, tools, and flows
 - Builder Pattern - Intuitive DSL for creating components
 - Strategy Pattern - Pluggable LLM providers and tool implementations
 - Observer Pattern - Event-driven agent communication
 
Agent- Base interface for all intelligent agentsComm- Universal communication unit (replaces legacy Message system)Tool- Reusable functions agents can executeRegistry<T>- Generic, thread-safe component registrySmartCore- Next-generation agent systemCommHub- Central message routing system
Comprehensive documentation is available in our GitHub Wiki:
- π Getting Started Guide - Installation and first steps
 - π― Core Concepts - Understanding the fundamentals
 - ποΈ Architecture Overview - System design and patterns
 - π Examples - Learn by example
 - π§ API Reference - Detailed API documentation
 
Coordinate multiple agents with AI-powered meta-coordination:
import io.github.noailabs.spice.swarm.*
// Create a swarm with AI coordinator
val llmCoordinator = buildAgent {
    name = "GPT-4 Coordinator"
    // Configure your GPT-4 or Claude agent
}
val aiSwarm = buildSwarmAgent {
    name = "AI Research Swarm"
    description = "Multi-agent research and analysis"
    // Use AI for intelligent coordination
    aiCoordinator(llmCoordinator)
    // Add member agents
    quickSwarm {
        researchAgent("researcher", "Lead Researcher")
        analysisAgent("analyst", "Data Analyst")
        specialist("expert", "Domain Expert", "Expert analysis")
    }
    config {
        debug(true)
        timeout(60000)
    }
}
// Execute with automatic strategy selection
val result = aiSwarm.processComm(Comm(
    content = "Analyze the impact of AI on healthcare",
    from = "user",
    type = CommType.TEXT
))
// Or use pre-configured swarms
val research = researchSwarm(name = "Research Team")
val creative = creativeSwarm(name = "Creative Team")
val decision = decisionSwarm(name = "Decision Team")Track everything with OpenTelemetry:
import io.github.noailabs.spice.observability.*
// Initialize observability at startup
ObservabilityConfig.initialize(
    ObservabilityConfig.Config(
        serviceName = "my-ai-app",
        serviceVersion = "1.0.0",
        otlpEndpoint = "http://localhost:4317",
        enableTracing = true,
        enableMetrics = true
    )
)
// Add tracing to any agent
val tracedAgent = buildAgent {
    name = "Research Agent"
    handle { comm ->
        // Your agent logic
        SpiceResult.success(comm.reply("Result", id))
    }
}.traced()  // Add tracing with one line!
// Create observable swarm
val observableSwarm = buildSwarmAgent {
    name = "Observable Swarm"
    quickSwarm {
        val agent1 = buildAgent { ... }.traced()
        val agent2 = buildAgent { ... }.traced()
        addAgent("agent1", agent1)
        addAgent("agent2", agent2)
    }
}
// View complete traces in Jaeger
// Track metrics in Grafana
// Monitor LLM costs in real-timeAutomatic context flow through all operations - perfect for multi-tenant systems:
import io.github.noailabs.spice.context.*
// Set context once at HTTP boundary
suspend fun handleRequest(request: HttpRequest) {
    withAgentContext(
        "tenantId" to request.tenantId,
        "userId" to request.userId,
        "correlationId" to UUID.randomUUID().toString()
    ) {
        // β
 Context flows automatically through ALL operations!
        agent.processComm(request.toComm())
    }
}
// Create context-aware tools
val agent = buildAgent {
    id = "order-agent"
    val orderService = OrderService()  // Context-aware service
    contextAwareTool("create_order") {
        description = "Create new order"
        param("items", "array", "Order items")
        execute { params, context ->
            // β
 Context automatically injected!
            val tenantId = context.tenantId!!
            val userId = context.userId!!
            // Service call automatically scoped to tenant!
            orderService.createOrder(params["items"] as List<*>)
        }
    }
}
// Context-aware services
class OrderRepository : BaseContextAwareService() {
    // Automatic tenant scoping - no manual parameters!
    suspend fun findOrders() = withTenant { tenantId ->
        database.query("WHERE tenant_id = ?", tenantId)
    }
    // Automatic tenant + user scoping
    suspend fun createOrder(items: List<String>) = withTenantAndUser { tenantId, userId ->
        Order(
            tenantId = tenantId,   // β
 From context
            userId = userId,       // β
 From context
            items = items
        )
    }
}
// Result: -20 lines of boilerplate per operation! πBenefits:
- β Zero manual parameter passing
 - β Perfect tenant isolation
 - β Thread-safe coroutine context
 - β Type-safe property access
 
See Context API Documentation and Production Examples.
// Create specialized agents
val researcher = buildAgent {
    id = "researcher"
    name = "Research Agent"
    // ... configuration
}
val analyzer = buildAgent {
    id = "analyzer"
    name = "Analysis Agent"
    // ... configuration
}
// Register them
AgentRegistry.register(researcher)
AgentRegistry.register(analyzer)
// Create a workflow
val researchFlow = buildFlow {
    id = "research-flow"
    name = "Research and Analyze"
    step("research", "researcher")
    step("analyze", "analyzer") { comm ->
        // Only analyze if research found something
        comm.content.isNotEmpty()
    }
}val ragAgent = buildAgent {
    id = "rag-agent"
    name = "RAG Assistant"
    
    // Configure vector store
    vectorStore("knowledge") {
        provider("qdrant")
        connection("localhost", 6333)
        collection("documents")
    }
    
    handle { comm ->
        // Automatic vector search tool available
        val results = run("search-knowledge", mapOf(
            "query" to comm.content,
            "topK" to 5
        ))
        // Process results...
    }
}Spice provides unified JSON serialization for all components:
import io.github.noailabs.spice.serialization.SpiceSerializer.toJson
import io.github.noailabs.spice.serialization.SpiceSerializer.toJsonSchema
// Serialize any component to JSON
val agentJson = myAgent.toJson()
val toolJson = myTool.toJson()
val vectorStoreJson = myVectorStore.toJson()
// Export tool as JSON Schema
val schema = myAgentTool.toJsonSchema()
// Handle complex metadata properly
val metadata = mapOf(
    "tags" to listOf("ai", "agent"),
    "config" to mapOf("timeout" to 30, "retries" to 3)
)
// Preserves structure instead of toString()!
val jsonMetadata = SpiceSerializer.toJsonMetadata(metadata)Spice now includes enterprise-grade multi-tenant support:
import io.github.noailabs.spice.tenant.*
// Set tenant context
TenantContext.set("tenant-123", mapOf("tier" to "premium"))
// Use tenant-aware runtime
val runtime = TenantAwareAgentRuntime(
    context = agentContext { 
        this["locale"] = "en-US"
    },
    tenantId = "tenant-123"
)
// Create tenant-aware agent
class MyTenantAgent : TenantAwareAgent(
    id = "my-agent",
    name = "Multi-Tenant Agent"
) {
    override suspend fun processCommInternal(comm: Comm): Comm {
        val tenantId = requireTenantId() // Automatically gets current tenant
        // Process with tenant isolation
        return comm.reply("Processing for tenant: $tenantId", id)
    }
}
// Tenant context propagates across coroutines
coroutineScope {
    launch(coroutineContext.withTenant("tenant-456")) {
        // This coroutine sees tenant-456
        val agent = agentRegistry.get("my-agent")
        agent.processComm(comm) // Processes in tenant-456 context
    }
}Dynamic platform registration and management:
import io.github.noailabs.spice.platform.*
// Create platform manager
val platformManager = DefaultPlatformManager()
// Register a platform
platformManager.registerPlatform(
    PlatformInfo(
        id = "shopify",
        name = "Shopify",
        type = PlatformType.MARKETPLACE,
        capabilities = setOf("orders", "inventory", "customers"),
        metadata = mapOf("apiVersion" to "2024-01")
    )
)
// Tenant-specific platform configuration
val tenantPlatformManager = DefaultTenantPlatformManager(platformManager)
tenantPlatformManager.registerPlatformForTenant(
    tenantId = "tenant-123",
    platformId = "shopify",
    config = PlatformConfig(
        platformId = "shopify",
        credentials = mapOf("apiKey" to "xxx", "apiSecret" to "yyy"),
        endpoints = mapOf("base" to "https://mystore.myshopify.com"),
        rateLimits = RateLimitConfig(
            requestsPerMinute = 40,
            burstSize = 10
        )
    )
)
// Check platform health
val health = platformManager.checkPlatformHealth("shopify")
println("Platform status: ${health.status}")Version control for configuration and policies:
import io.github.noailabs.spice.policy.*
import kotlinx.serialization.json.*
// Create policy manager
val policyManager = DefaultPolicyManager()
// Save a policy with version tracking
val v1 = policyManager.savePolicy(
    policyId = "refund-policy",
    content = buildJsonObject {
        put("maxRefundDays", 30)
        put("autoApprove", true)
        put("maxAmount", 1000)
    },
    createdBy = "admin",
    comment = "Initial refund policy"
)
// Update policy (creates v2)
val v2 = policyManager.savePolicy(
    policyId = "refund-policy",
    content = buildJsonObject {
        put("maxRefundDays", 60)  // Changed
        put("autoApprove", false) // Changed
        put("maxAmount", 1000)
    },
    createdBy = "admin",
    comment = "Extended refund period, manual approval required"
)
// View history
val history = policyManager.getPolicyHistory("refund-policy")
history.forEach { entry ->
    println("v${entry.version}: ${entry.comment} by ${entry.createdBy}")
}
// Rollback to v1
val v3 = policyManager.rollbackPolicy(
    policyId = "refund-policy",
    targetVersion = 1,
    rolledBackBy = "admin",
    comment = "Reverting to original policy"
)Spice now includes a complete event sourcing module with Kafka integration:
import io.github.noailabs.spice.eventsourcing.*
import javax.sql.DataSource
// Create event store with Kafka and PostgreSQL
val eventStore = EventStoreFactory.kafkaWithPostgres(
    kafkaCommHub = kafkaCommHub,
    dataSource = dataSource,
    config = EventStoreConfig(
        topicPrefix = "events",
        snapshotFrequency = 100
    )
)
// Or use in-memory for testing
val testEventStore = EventStoreFactory.inMemory(dataSource)
// Or use custom event publisher (e.g., RabbitMQ)
val customEventStore = EventStoreFactory.withCustomPublisher(
    dataSource = dataSource,
    eventPublisher = MyRabbitMQEventPublisher()
)
// Define domain events
class OrderCreatedEvent(
    orderId: String,
    val customerId: String,
    val items: List<OrderItem>
) : EntityCreatedEvent(
    aggregateId = orderId,
    aggregateType = "Order"
)
// Create event-sourced aggregate
class OrderAggregate(
    override val aggregateId: String
) : Aggregate() {
    
    override val aggregateType = "Order"
    
    // State
    var customerId: String? = null
    var items: MutableList<OrderItem> = mutableListOf()
    var status: OrderStatus = OrderStatus.DRAFT
    
    // Business operations
    suspend fun create(customerId: String, items: List<OrderItem>) {
        raiseEvent(OrderCreatedEvent(aggregateId, customerId, items))
    }
    
    // Apply events to update state
    override fun apply(event: DomainEvent) {
        when (event) {
            is OrderCreatedEvent -> {
                this.customerId = event.customerId
                this.items.addAll(event.items)
                this.status = OrderStatus.CREATED
            }
        }
    }
}
// Use repository for loading/saving
val repository = AggregateRepository(eventStore)
// Create and save aggregate
val order = OrderAggregate("order-123")
order.create("customer-456", items)
repository.save(order)
// Load aggregate from events
val loadedOrder = repository.load("order-123") { OrderAggregate(it) }Efficiently manage aggregate snapshots with the Memento pattern:
import io.github.noailabs.spice.eventsourcing.*
// Define a memento for your aggregate
data class CartMemento(
    override val aggregateId: String,
    override val version: Long,
    override val timestamp: Instant,
    val customerId: String,
    val items: List<CartItem>,
    val totalAmount: Money,
    val appliedCoupons: List<String>
) : AggregateMemento, Serializable {
    
    override val aggregateType = "ShoppingCart"
    
    override fun restoreState(aggregate: Aggregate) {
        require(aggregate is ShoppingCartAggregate)
        aggregate.restoreFromMemento(this)
    }
}
// Create aggregate with memento support
class ShoppingCartAggregate(
    override val aggregateId: String
) : MementoAggregate() {
    
    override val aggregateType = "ShoppingCart"
    
    // State
    private var customerId: String? = null
    private val items = mutableListOf<CartItem>()
    private var totalAmount = Money(0, "USD")
    private val appliedCoupons = mutableListOf<String>()
    
    // Create memento
    override fun doCreateMemento(): AggregateMemento {
        return CartMemento(
            aggregateId = aggregateId,
            version = version,
            timestamp = Instant.now(),
            customerId = customerId ?: "",
            items = items.toList(),
            totalAmount = totalAmount,
            appliedCoupons = appliedCoupons.toList()
        )
    }
    
    // Restore from memento
    override fun doRestoreFromMemento(memento: AggregateMemento) {
        require(memento is CartMemento)
        this.customerId = memento.customerId
        this.items.clear()
        this.items.addAll(memento.items)
        this.totalAmount = memento.totalAmount
        this.appliedCoupons.clear()
        this.appliedCoupons.addAll(memento.appliedCoupons)
    }
    
    // Event handling...
    override fun apply(event: DomainEvent) {
        // Apply events to update state
    }
}
// Use memento-aware repository
val repository = MementoAggregateRepository(
    eventStore = eventStore,
    snapshotStore = MementoSnapshotStoreFactory.inMemory(),
    snapshotFrequency = 50  // Create snapshot every 50 events
)
// Load aggregate (uses snapshot if available)
val cart = repository.load("cart-123") { ShoppingCartAggregate(it) }
// Save aggregate (creates snapshot automatically if needed)
repository.save(cart)Implement distributed transactions with compensating actions:
import io.github.noailabs.spice.eventsourcing.*
// Define a saga for order fulfillment
class OrderFulfillmentSaga(
    override val sagaId: String,
    private val orderId: String
) : Saga() {
    
    override val sagaType = "OrderFulfillment"
    
    override suspend fun execute(context: SagaContext) {
        // Step 1: Reserve inventory
        executeStep(ReserveInventoryStep(orderId), context)
        
        // Step 2: Process payment
        executeStep(ProcessPaymentStep(orderId), context)
        
        // Step 3: Create shipment
        executeStep(CreateShipmentStep(orderId), context)
        
        // Step 4: Send confirmation
        executeStep(SendConfirmationStep(orderId), context)
    }
}
// Define saga steps with compensation logic
class ProcessPaymentStep(private val orderId: String) : SagaStep {
    override val name = "ProcessPayment"
    
    override suspend fun execute(context: SagaContext) {
        // Process payment
        val paymentId = paymentService.charge(orderId)
        context.set("paymentId", paymentId)
    }
    
    override suspend fun compensate(context: SagaContext) {
        // Refund if saga fails
        val paymentId = context.get<String>("paymentId")
        paymentService.refund(paymentId)
    }
}
// Execute saga
val sagaManager = SagaManager(eventStore, InMemorySagaStore())
val saga = OrderFulfillmentSaga("saga-123", "order-123")
sagaManager.startSaga(saga, SagaContext())@SpringBootApplication
@EnableSpice
class MyApplication
@Component
class MyService(
    @Autowired private val agentRegistry: AgentRegistry
) {
    fun processRequest(message: String): String {
        val agent = agentRegistry.get("my-agent")
        val response = runBlocking {
            agent?.processComm(Comm(content = message, from = "user"))
        }
        return response?.content ?: "No response"
    }
}We welcome contributions! Please see our Contributing Guide for details.
# Clone the repository
git clone https://github.com/spice-framework/spice.git
cd spice-framework
# Build the project
./gradlew build
# Run tests
./gradlew test- β Core Agent System
 - β Generic Registry System
 - β Unified Communication (Comm)
 - β Tool Management System
 - β LLM Integrations (OpenAI, Anthropic)
 - β Spring Boot Starter
 - β JSON Serialization System
 - β PSI (Program Structure Interface) - DSL to tree conversion
 - β Swarm Intelligence (Multi-agent coordination with 5 strategies)
 - β AI-Powered Swarm Coordinator (LLM-enhanced meta-coordination)
 - β OpenTelemetry Integration (Distributed tracing & metrics)
 - β TracedAgent Wrapper (Automatic observability)
 - β MCP Protocol Support (Model Context Protocol integration)
 - β Multi-Tenant Support with ThreadLocal context propagation
 - β Dynamic Platform Management System
 - β Policy Versioning with rollback capability
 - β Tenant-aware storage abstractions
 - β Event Sourcing Module with Kafka and PostgreSQL
 - β Saga Pattern for distributed transactions
 - β CachedAgent (Performance optimization - response caching)
 - β BatchingCommBackend (Performance optimization - message batching)
 - π§ Vector Store Integrations (Qdrant implemented, others in progress)
 
This project is licensed under the MIT License - see the LICENSE file for details.
- Built with β€οΈ using Kotlin and Coroutines
 - Inspired by modern AI agent architectures
 - Special thanks to all contributors
 
- GitHub Issues: Report bugs or request features
 - Discussions: Ask questions and share ideas
 - Wiki: Comprehensive documentation
 
Ready to spice up your AI applications? πΆοΈ
Get Started β’ View Examples β’ Read Docs