Craftium

✨ Imagine the features of Minecraft: open world, procedural generation, fully destructible voxel environments... but open source, without Java, easily extensible in Lua, and with the modern Gymnasium API designed for RL... This is Craftium! ✨

[Docs] [GitHub] [Paper (ArXiv)]

Craftium is a fully open-source platform for creating Reinforcement Learning (RL) environments and tasks that provides a Gymnasium wrapper for the Minetest voxel game engine. Craftium forks Minetest to support:

• Connecting to Craftium's Python process via TCP (sending observations, and other data; receiving action commands)
• Executing Craftium actions as keyboard and mouse controls.
• Extesions to the Lua API for creating RL environments and tasks.
• Minetest's client/server synchronization.

📚 Craftium's documentation can be found here.

📑 If you use craftium in your projects or research please consider citing the project, and don't hesitate to let us know what you have created using the library! 🤗

Features ✨

• Super extensible 🧩: Minetest is built with modding in mind! The game engine is completely extensible using the Lua programming language. Easily create mods to implement the environment that suits the needs of your research! See environments for a showcase of what is possible with craftium.

• Extensive documentation 📚: Craftium extensively documents how to use existing environments and create new ones. The documentation also includes a low-level reference of the exported Python classes and gymnasium Wrappers, extensions to the Lua API, and code examples. Moreover, Craftium greatly benefits from existing Minetest documentation like the wiki, reference, and modding book.

• Modern RL API 🤖: Craftium slightly modifies Mintest to communicate with Python, and implements the well-known Gymnasium's Env API. This opens the door to a huge number of existing tools and libraries compatible with this API, such as stable-baselines3 or CleanRL (check examples section!).

• Fully open source 🤠: Craftium is based on the Minetest and Gymnasium, both open-source projects.

• No more Java ⚡: The Minecraft game is written in Java, not a very friendly language for clusters and high-performance applications. Contrarily, Minetest is written in C++, much more friendly for clusters, and highly performant!

Installation ⚙️

First, clone craftium using the --recurse-submodules flag (important: the flag fetches submodules needed by the library) and cd into the project's main directory:

git clone --recurse-submodules https://github.com/mikelma/craftium.git # if you prefer ssh: [email protected]:mikelma/craftium.git
cd craftium

craftium depends on Minetest, which in turn, depends on a series of libraries that we need to install. Minetest's repo contains instructions on how to setup the build environment for many Linux distros (and MacOS). In Ubuntu/Debian the following command installs all the required libraries:

sudo apt install g++ make libc6-dev cmake libpng-dev libjpeg-dev libgl1-mesa-dev libsqlite3-dev libogg-dev libvorbis-dev libopenal-dev libcurl4-gnutls-dev libfreetype6-dev zlib1g-dev libgmp-dev libjsoncpp-dev libzstd-dev libluajit-5.1-dev gettext libsdl2-dev

Additionally, craftium requires Python's header files to build a dedicated Python module implemented in C (mt_server):

sudo apt install libpython3-dev

Then, check that setuptools is updated and run the installation command in the craftium repo's directory:

pip install -U setuptools

and,

pip install .

This last command should compile minetest, install python dependencies, and, finally, craftium. If the installation process fails, please consider submitting an issue here.

Getting started 💡

Craftium implements Gymnasium's Env API, making it compatible with many popular reinforcement learning libraries and tools like stable-baselines3 and CleanRL.

Thanks to the mentioned API, using craftium environments is as simple as using any gymnasium environment. The example below shows the implementation of a random agent in one of the craftium's environments:

import gymnasium as gym
import craftium

env = gym.make("Craftium/ChopTree-v0")

observation, info = env.reset()

for t in range(100):
    action = env.action_space.sample()  # get a random action
    observation, reward, terminated, truncated, _info = env.step(action)

    if terminated or truncated:
        observation, info = env.reset()

env.close()

Training a CNN-based agent using PPO is as simple as:

from stable_baselines3 import PPO
from stable_baselines3.common import logger
import gymnasium as gym
import craftium

env = gym.make("Craftium/ChopTree-v0")

model = PPO("CnnPolicy", env, verbose=1)

new_logger = logger.configure("logs-ppo-agent", ["stdout", "csv"])
model.set_logger(new_logger)
model.learn(total_timesteps=100_000)

env.close()

This example trains a CNN-based agent for 10K timesteps in the Craftium/ChopTree-v0 environment using PPO. Additionally, we set up a custom logger that records training statistics to a CSV file inside the logs-ppo-agent/ directory.

Craftium is not only limited to "typical" RL scenarios, its versatility makes it the ideal platform for meta-RL, open-ended learning, continual and lifelong RL, or unsupervised environment design. As a showcase, Craftium provides open-world and procedurally generated tasks (see environments for more info).

This code snippet initializes a procedurally generated dungeon environment with 5 rooms and a maximum number of 4 monsters per room:

env, map_str = craftium.make_dungeon_env(
    mapgen_kwargs=dict(
        n_rooms=5,
        max_monsters_per_room=5,
    ),
    return_map_str=True,
)
print(map_str.split("-")[1])

Examples 🤓

By implementing Gymnasium's API, craftium supports many existing tools and libraries. Check these scripts for some examples:

• Stable baselines3: sb3_train.py
• Ray rllib: sb3_train.py
• CleanRL: cleanrl_ppo_train.py

Citing Craftium 📑

@article{malagon2024craftium,
  title={Craftium: An Extensible Framework for Creating Reinforcement Learning Environments},
  author={Mikel Malag{\'o}n and Josu Ceberio and Jose A. Lozano},
  journal={arXiv preprint arXiv:2407.03969},
  year={2024}
}

Contributing 🏋️

We appreciate contributions to craftium! craftium is in early development, so many possible improvements and bugs are expected. If you have found a bug or have a suggestion, please consider opening an issue if it isn't already covered. In case you want to submit a fix or an improvement to the library, pull requests are also very welcome!

License 📖

Craftium is based on minetest and its source code is distributed with this library. Craftium maintains the same licenses as minetest: MIT for code and CC-BY-SA 3.0 for content.

Acknowledgements 🤗

We thank the minetest and gymnasium developers and community for maintaining such an excellent open-source project. We also thank minerl and other projects integrating minetest and gymnasium (here and here) for serving as inspiration for this project.

We are also grateful to:

• @vadel for helping with obscure bugs 🐛.
• @abarrainkua for reading preliminary versions of the paper.
• Pascu for the technical support in HPC.