Architecture overview of pomelo

The architecture overview of pomelo

This article is about the design and techinal choice of pomelo. Why we choose node.js, why we use this architecture, why we design that way. It is based on our experience of game development and a lot of studies on previous game server solutions.

Why we choose node.js?

Node.js is astonishingly suitable for game server development. In the definition of node.js, fast, scalable, realtime, network, all these features are quite suitable for game server.

Game server is a high density network application, it requires realtime, the network io advantage makes node.js and game server a perfect match. But IO is not the only reason for us to choose node.js, here is all the reasons we choose node.js:

• Network IO and scalablity. Perfect match for game servers, which requires realtime, scalabiltiy, high density network IO.
• The node.js thread model. Single thread is quite suitable for game server, it helps us to handle all the troubles about concurrency, lock and other annoying questions. Multi-process, single thread is ideal for game server.
• The language advantages. Javascript is powerful, popular and performance good. Further more, if you choose HTML 5 for client, you can reuse a lot of code logic between server and client.

The game server runtime architecture

A scalable game server runtime architecture must be multi-process, single process does not scale, even node.js. gritsgame from google and browserquest from mozilla all use node.js as game server, but only single process, which means their online users are limited.

A typical mutli-process MMO runtime is following:

Memo of runtime architecture

• Clients connect to the servers through websocket
• Connectors do not handle logic, it just forward the messages to backend server
• The backend servers include area, chat, status and other type servers, they all handle logic. Most of the game related logics are handled in area servers.
• The backend servers will send back the result to connector, connector than broadcast to related clients.
• Master manages all these servers, including startup, monitor, close etc.

The difference between game servers and web servers

It looks like web servers and game servers are simliar, but actually it's not:

• Long connection VS short connection. The game servers must connect with socket, which is critical for realtime application. Longconnection architecture makes it all different, since all the servers are tightly coupled.
• Difference partition strategies. Game server is based on area based partition strategy, because it can minimize cross process invocation. But web servers can be partitioned based on any load balance strategies, which makes web app more scalable.
• Stateful VS stateless. Because of the partition strategy, the game server is stateful, which limits game server's scalability.
• Broadcast VS request/response. Not like web, Game servers need a lot of broadcasts, even a small action must be notified to related players. These broadcasts makes network communication a big burden.

The runtime architecture is complicated, we need a framework to simplify it.

Not like web, there is not so much open source game frameworks, not event this architecture. Pomelo is a rescue, it let you write as little code as possible to support this complicated runtime architecture.

Introduction to pomleo framework

The following is all components of pomelo:

Besides framework, pomelo also provides a lot of libraries and tools. Libraries like ai, aoi is specific for games, pomelo-schedule, seq-queue, datasync is for common usage. Tools include robots, admin console, etc. are all useful. All these components are organized in npm module, which makes framework easy to extend. We'll talk about extensibility later. But the core of pomelo is framework, you can call it multi-process, realtime application framework. It is a common purpose architecture, not only for game. For example, you can write scalable , distribute chat application with pomelo.

The design goal of pomelo

• Servers abstraction and extension.

In web app, servers are stateless, loosely coupled, so there is no need for a framework to manager all these servers. Game apps, however, are different. All these servers work tightly together, and there are a lot of server types. We need to support all these server types and servers extension.

• Request/response, broadcast abstraction

We need a request, response mechanism, or more specificly, a request/broadcast mechanism. Since broadcast is the most usual action in game servers, and potentially a performance bottleneck.

• Servers rpc communication

Servers need to talk to each other, although we try to avoid it.We need a rpc framework as simple as possible.

Introduction to servers abstraction and extention

Servers types

Pomelo divide servers in two types: frontend and backend, here it is:

The frontend servers responsibilty:

• handle client connection
• maintain session information
• forward request to backend
• broadcast messages to clients

The backend servers responsibility:

• handle logic, including rpc and frontend logic
• push result back to fronend

The server duck type

Duck type is commonly used in OOP of dynamic language. Servers, however can also use duck type idea. There is only two type of interfaces for a server:

• handle client request, we call it handler
• handle rpc call, we call it remote All we have to do is to define handler and remote, which can define what the server looks like.

The implementation of server abstraction

The simplest way is to correspond directory structure and server. The directory structure can be look as the serialization version of server. Here is the example:

We just name the server 'area', the behavior of the server is determined by the code in 'handler' and 'remote'. All we need to do is to fill the code in handler and remote. To make the server run, we need a small config file called servers.json. Here is the example file:

{
  "development":
    {
    "connector": [
      {"id": "connector-server-1", "host": "127.0.0.1", "port": 3150, "wsPort":3010},
      {"id": "connector-server-2", "host": "127.0.0.1", "port": 3151, "wsPort":3011}
    ],
    "area": [
      {"id": "area-server-1", "host": "127.0.0.1", "port": 3250, "area": 1},
      {"id": "area-server-2", "host": "127.0.0.1", "port": 3251, "area": 2},
      {"id": "area-server-3", "host": "127.0.0.1", "port": 3252, "area": 3}
    ],
    "chat":[
      {"id":"chat-server-1","host":"127.0.0.1","port":3450}
    ]
    }
}

Client request, response, broadcast

Although we use long connection, but the request/response api looks like web. Here is the example:

The api looks like ajax request, although it's actually a long connection cross server request. Based on convention over configuration rules, there is no config. Pomelo also have filter, broadcast and other mechanisms, you make see the details in pomelo framework reference

rpc abstraction

The rpc framework is really simple, it can automatically choose route strategy and route to the target server with no configuration.Here is the picture:

The picture above defined a rpc interface in chatRemotejs, the definication is following:

chatRemote.kick = function(uid, player, cb) {
}

The rpc cient just invoke like this:

app.rpc.chat.chatRemote.kick(session, uid, player, function(data){
});

Notice the session parameter, it's crucial for router. The framework will help you send the message to certain server.

conclusion

The above framework mechanism help us handle server abstraction and extension, client request and broadcast, server communication. All these are the basic ground of pomelo, above these, we can construct libraries and tools, or framework in another abstract level.