How does a load balancer work with replicas for stateless

[botengyao]

Hey,

I am confused now:

From the doc here

• No session ID headers are sent or expected
• Any server node can process any request
• No state is preserved between requests
• Perfect for RESTful or stateless API scenarios
• Simplest deployment model with minimal infrastructure requirements

┌─────────────────────────────────────────────┐
│                  Client                     │
└─────────────────────────────────────────────┘
                     │
                     ▼
┌─────────────────────────────────────────────┐
│                Load Balancer                │
└─────────────────────────────────────────────┘
          │                       │
          ▼                       ▼
┌─────────────────┐     ┌─────────────────────┐
│  MCP Server #1  │     │    MCP Server #2    │
│ (Node.js)       │     │  (Node.js)          │
└─────────────────┘     └─────────────────────┘

If there are multiple replicas for MCP Server 1, and the client initializes with the first replica, but its second request is load-balanced to a second replica that hasn’t been initialized yet, the request fails.

Is the initialize phase still needed for stateless? like if a server is ever initialized then it can server any requests? I think we would need more clarification for this use case with a load balancer in front.

[botengyao]

cc @ihrpr

[ihrpr]

In a stateless setup, the load balancer simply routes incoming requests to any available node without tracking session state. Here's what happens:

• When a client makes its initial request, the load balancer routes it to any available server (let's say Node 1).
• Node 1 responds with its capabilities, and the client sends an initialization notification to confirm it's ready.
• For subsequent requests, the load balancer might route to a different server (Node 2) — this is perfectly fine because stateless servers don't maintain session state between requests.
  Each MCP server instance can process requests independently, regardless of whether it was the one that handled the initial capabilities exchange.

The key point is that with stateless servers, there's no need for sticky sessions or for requests to go back to the same server that handled initialization. Any server instance can handle any request at any time, as long as it's configured to provide the same capabilities

[botengyao]

Thanks @ihrpr for the reply.

There is one scenario that server 1 and server 2 are same and horizontally scaled.

Let's say the server 1 replies the initialization response from a client. And then a client continues to send the tool use request to an LB that is routed to server 2, does server 2 require the initialization process?

If server 2 doesn't require it, then sever 1 will also work without the initialization phase, am I understanding correctly?