[Docs] [Lite] AReaLite release documentation#202
Conversation
| + 🔪 **Cutting-Edge Performance:** AReaL can produce models with cutting-edge reasoning capabilities in math and coding. We are also actively working on agentic tasks. | ||
| - ⚡ **\[NEW\] Light-weight & AI-centric:** Our new release **AReaLite** follows an | ||
| **AI-centric** design that prioritizes better development experiences for AI | ||
| researchers. As a result, **AReaLite** delivers most AReaL functionalities with a much |
There was a problem hiding this comment.
Can you also mention about the performance of new codebase?
| **[2025/02/24] (v0.1)** Our initial release includes reproducible results for 1.5B and 7B LRMs. Check our [v0.1 technical blog](/blog/AReaL_v0_1.md). | ||
| **\[2025/07/31\] (v0.4, AReaLite)** We introduce **AReaLite**, a **light-weight** | ||
| version of AReaL designed specifically for AI researchers and rapid prototyping. | ||
| AReaLite features an **AI-centric** API design that prioritizes ease of use and |
There was a problem hiding this comment.
AI centric to RL centric or Algorithm First? AI centric is not very common expression
There was a problem hiding this comment.
Here we use "AI-centric" because it is a contrast to "system centric". Another point is that now AReaLite is designed to run in SPMD paradigm, which is familiar to AI researchers that are not so experienced in RL. Therefore using RL-centric would be less accurate.
I think "Algorithm First" is more appropriate: "algorithm first" - "system first".
Could you share some insights as well? @garrett4wade
There was a problem hiding this comment.
"algorithm first" - "system first" would be fine for this cover letter. We can discuss offline later.
| New highlights in AReaLite: | ||
|
|
||
| *Fig.1. Left: Execution timeline of synchronous RL training. Right: Execution timeline of one-step overlap RL system.* | ||
| - Follows an *AI-centric* API design instead of the *system-centric* architecture in old |
| Train Qwen3 1.7B locally (Remember to modify `dataset.path` in the script below): | ||
| ```bash | ||
| bash examples/run_async_ppo.sh | ||
| python3 examples/arealite/convert_config.py -f <config_path> -o <output_path> |
There was a problem hiding this comment.
Haven't found this file in https://github.com/inclusionAI/AReaL/pull/154/files
There was a problem hiding this comment.
This is a standalone script which will be updated later.
| pipelining | ||
| - [ ] RL for vision-language models (VLM) | ||
| - [x] Multi-turn agentic RL | ||
| - [ ] Function calling and tool use |
There was a problem hiding this comment.
This should be supported now
There was a problem hiding this comment.
Currently AReaLite uses FSDP and AReaL still not supports Megatron for these features ... But these features are in the schedule, and they will come in one or two months.
VLM and function calling can be checked.
nuzant
had a problem deploying
to
AReaL-unittests
GitHub Actions
Failure
nuzant
had a problem deploying
to
AReaL-lite-unittests
GitHub Actions
Failure
nuzant
had a problem deploying
to
AReaL-unittests
GitHub Actions
Failure
garrett4wade
had a problem deploying
to
AReaL-unittests
GitHub Actions
Error
4 participants
No description provided.