Add CI for testing AReaLite#150
Merged
Merged
Conversation
added 25 commits
July 3, 2025 20:41
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR adds continuous integration support for AReaLite by introducing CI scripts and a GitHub Actions workflow, and updates test configurations to point to the new model path.
- Introduce bash scripts to clone the repo, build the Docker environment image, and run tests on a remote node
- Add a GitHub Actions workflow to orchestrate the CI steps via SSH
- Update all tests to use the new
Qwen/Qwen2-0.5Bmodel identifier instead of the old absolute path
Reviewed Changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| ci/clone_repo.sh | New script to clone the repository at a specific commit |
| ci/build_env_image.sh | New script to build (or skip) the Docker environment image |
| ci/test_arealite.sh | New script to run tests inside the Docker environment |
| .github/workflows/test-arealite.yml | GitHub Actions workflow to invoke the CI scripts via SSH |
| arealite/tests/test_sglang_client.py | Update MODEL_PATH to the new model identifier |
| arealite/tests/test_sft.py | Update EngineConfig.path to use the new model identifier |
| arealite/tests/test_rollout_controller.py | Update MODEL_PATH to the new model identifier |
| arealite/tests/test_rollout.py | Update MODEL_PATH to the new model identifier |
| arealite/tests/test_grpo.py | Update MODEL_PATH to the new model identifier |
| arealite/tests/test_engine.py | Update MODEL_PATH to the new model identifier |
Comments suppressed due to low confidence (1)
.github/workflows/test-arealite.yml:14
- Add
cancel-in-progress: trueunderconcurrencyto automatically cancel previous runs in the same group.
concurrency:
| @@ -0,0 +1,28 @@ | |||
| #!/usr/bin/env bash | |||
|
|
|||
| set -e | |||
There was a problem hiding this comment.
Consider adding set -u (or set -eu) to treat unset variables as errors and avoid silent failures.
Suggested change
| set -e | |
| set -eu |
| --name $RUN_ID \ | ||
| --gpus all \ | ||
| --shm-size=8g \ | ||
| -v $(pwd):/workspace \ |
There was a problem hiding this comment.
Quote the volume path to handle directories with spaces, e.g., -v "$(pwd)":/workspace.
Suggested change
| -v $(pwd):/workspace \ | |
| -v "$(pwd)":/workspace \ |
| --name $RUN_ID \ | ||
| --gpus all \ | ||
| --shm-size=8g \ | ||
| -v $(pwd):/workspace \ |
There was a problem hiding this comment.
Quote the volume path to handle directories with spaces, e.g., -v "$(pwd)":/workspace.
Suggested change
| -v $(pwd):/workspace \ | |
| -v "$(pwd)":/workspace \ |
| EXPR_NAME = "test_sglang_client" | ||
| TRIAL_NAME = "test_sglang_client" | ||
| MODEL_PATH = "/storage/openpsi/models/Qwen__Qwen3-1.7B/" | ||
| MODEL_PATH = "Qwen/Qwen2-0.5B" |
There was a problem hiding this comment.
[nitpick] This MODEL_PATH is duplicated across multiple tests; consider extracting it into a shared fixture or constant to reduce duplication and ease future updates.
garrett4wade
added a commit
that referenced
this pull request
Jul 10, 2025
* initial proposal * add arealite * . * change api * . * remove LOG_ROOT * remove MODEL_SAVE_PATH * remove PARAM_REALLOC_PATH, DATASET_CACHE * prepare for testing * prepare for testing * ready for run * local run * tests mainly pass * format * . * amend cluster.py * . * . * client test pass * pass rollout test * remove unused imports * add arealite readme * change api * . * . * . * . * . * . * . * . * format * . * implement iteraptable generation (#112) Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com> * . * fix * . * . * . * pass controller generate batch test * . * refactor rollout controller into worker and controller * . * . * . * change to async rollout * pass rollout controller test * pass test * . * update readme * . * sft debug * . * add lisence * remove unused files * remove unsed args in ppo * add hf engine wrapper (#116) * add hf engine * fix issues * fix ppo bugs and add test * add hf client interface and modify cli args * fix bugs * fix issues * Merge fw/refactor * Finish hf wrapper test * add test --------- Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com> * format * format * . * refine hf engine * . * fix * add fsdp engine and sft tests * . * . * . * pass ppo unittest * pass ppo and rollout controller tests * clear unused imports * rename ppo to grpo * change reward function organization * reorganize code * add dataset api * . * . * . * format * chmod fix * . * rename workflow to collector * refactor llm_client location * . * . * fix llm server api * refactor config structure * . * fix tests * . * . * . * Fix unresolved issue in SFTTrainer PR (#139) * . * . * efficient loading * format * . * . * . * . * . * . * Add CI for testing AReaLite (#150) * ci: add test-arealite * ci: add checkout before running test-arealite * ci: add USERNAME * ci: add test script * ci: add GitHub mirror * ci: fix typo * ci: clone one commit * ci: fix condition * ci: set command timeout to 60m * ci: enable pip cache * ci: optimize container lifecycle * ci: split into many stages * ci(test-arealite): fix typo * ci: fix wrong env * ci: fix pytest * ci: uninstall transformer-engine * ci: uninstall transformer-engine * ci: fix model paths * ci: show stdout/stderr * ci: fix not clean up * ci: backup sglang * ci: remove tmp repo dir when run * ci: fix docker run exit 1 condition * ci(test-arealite): limit the concurrency and extend command timeout * . * merge fw/refactor * revert some changes * fix --------- Co-authored-by: meizhiyu.mzy <meizhiyu.mzy@antgroup.com> Co-authored-by: Chayenne <zhaochen20@outlook.com> Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com> Co-authored-by: Jayon02 <qiujiangc@outlook.com> Co-authored-by: root <meizhiyu.mzy> Co-authored-by: Zijian Zhang <futrime@outlook.com>
Jayon02
pushed a commit
to Jayon02/AReaL
that referenced
this pull request
Jul 10, 2025
commit 8d4b8dc Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com> Date: Thu Jul 10 13:14:10 2025 +0800 [Doc] Add an instruction about how to run the SFT example. (areal-project#164) commit 3bf9c85 Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com> Date: Thu Jul 10 12:56:24 2025 +0800 [Fix] Merge previous contributions from fw/refactor to lite (areal-project#163) * initial proposal * add arealite * . * change api * . * remove LOG_ROOT * remove MODEL_SAVE_PATH * remove PARAM_REALLOC_PATH, DATASET_CACHE * prepare for testing * prepare for testing * ready for run * local run * tests mainly pass * format * . * amend cluster.py * . * . * client test pass * pass rollout test * remove unused imports * add arealite readme * change api * . * . * . * . * . * . * . * . * format * . * implement iteraptable generation (areal-project#112) Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com> * . * fix * . * . * . * pass controller generate batch test * . * refactor rollout controller into worker and controller * . * . * . * change to async rollout * pass rollout controller test * pass test * . * update readme * . * sft debug * . * add lisence * remove unused files * remove unsed args in ppo * add hf engine wrapper (areal-project#116) * add hf engine * fix issues * fix ppo bugs and add test * add hf client interface and modify cli args * fix bugs * fix issues * Merge fw/refactor * Finish hf wrapper test * add test --------- Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com> * format * format * . * refine hf engine * . * fix * add fsdp engine and sft tests * . * . * . * pass ppo unittest * pass ppo and rollout controller tests * clear unused imports * rename ppo to grpo * change reward function organization * reorganize code * add dataset api * . * . * . * format * chmod fix * . * rename workflow to collector * refactor llm_client location * . * . * fix llm server api * refactor config structure * . * fix tests * . * . * . * Fix unresolved issue in SFTTrainer PR (areal-project#139) * . * . * efficient loading * format * . * . * . * . * . * . * Add CI for testing AReaLite (areal-project#150) * ci: add test-arealite * ci: add checkout before running test-arealite * ci: add USERNAME * ci: add test script * ci: add GitHub mirror * ci: fix typo * ci: clone one commit * ci: fix condition * ci: set command timeout to 60m * ci: enable pip cache * ci: optimize container lifecycle * ci: split into many stages * ci(test-arealite): fix typo * ci: fix wrong env * ci: fix pytest * ci: uninstall transformer-engine * ci: uninstall transformer-engine * ci: fix model paths * ci: show stdout/stderr * ci: fix not clean up * ci: backup sglang * ci: remove tmp repo dir when run * ci: fix docker run exit 1 condition * ci(test-arealite): limit the concurrency and extend command timeout * . * merge fw/refactor * revert some changes * fix --------- Co-authored-by: meizhiyu.mzy <meizhiyu.mzy@antgroup.com> Co-authored-by: Chayenne <zhaochen20@outlook.com> Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com> Co-authored-by: Jayon02 <qiujiangc@outlook.com> Co-authored-by: root <meizhiyu.mzy> Co-authored-by: Zijian Zhang <futrime@outlook.com> commit d48bf00 Merge: 42c717b b9dbd4a Author: 博惟 <bowei.fw@antgroup.com> Date: Thu Jul 10 12:53:30 2025 +0800 Merge branch 'main' of https://github.com/inclusionAI/AReaL into lite commit 42c717b Merge: c38cffc a203c7c Author: 博惟 <bowei.fw@antgroup.com> Date: Thu Jul 10 11:15:01 2025 +0800 Merge branch 'lite' of https://github.com/inclusionAI/AReaL into lite commit c38cffc Author: 博惟 <bowei.fw@antgroup.com> Date: Thu Jul 10 11:10:10 2025 +0800 PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods Merge branch fw/lite-dev of git@code.alipay.com:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/340 Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com> * support fsdp engine and sglang remote engine * minor fix * . * refactor trainer * add close * rm mb_spec * fix commit b9dbd4a Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com> Date: Wed Jul 9 10:50:19 2025 +0800 Update to persistent wechat QR code. (areal-project#159) commit 17ea7fe Author: xssstory <33601810+xssstory@users.noreply.github.com> Date: Mon Jul 7 15:49:13 2025 +0800 fix math reward verifier (areal-project#156) * PullRequest: 293 fix get_param_realloc_path Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh https://code.alipay.com/inclusionAI/AReaL/pull_requests/293 Reviewed-by: 博惟 <bowei.fw@antgroup.com> * fix get_param_realloc_path * PullRequest: 297 bugfix: reward is always -5 Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh https://code.alipay.com/inclusionAI/AReaL/pull_requests/297 Reviewed-by: 博惟 <bowei.fw@antgroup.com> * bugfix: reward is always -5 * PullRequest: 321 fix checkpoint save dir Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh https://code.alipay.com/inclusionAI/AReaL/pull_requests/321 Reviewed-by: 博惟 <bowei.fw@antgroup.com> * fix checkpoint save dir * PullRequest: 328 [Doc] update installation Merge branch sxj/doc of git@code.alipay.com:inclusionAI/AReaL.git into gh https://code.alipay.com/inclusionAI/AReaL/pull_requests/328 Reviewed-by: 博惟 <bowei.fw@antgroup.com> * [Doc] update installation * PullRequest: 329 bugfix: math verifier blocks the async training Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh https://code.alipay.com/inclusionAI/AReaL/pull_requests/329 Reviewed-by: 博惟 <bowei.fw@antgroup.com> * bugfix: math verifier block the async training * format --------- Co-authored-by: 冰临 <shenxujie.sxj@antgroup.com> Co-authored-by: garrett4wade <fuwth17@gmail.com>
garrett4wade
added a commit
that referenced
this pull request
Jul 16, 2025
…nit test. (#161) * refactor hf engine * format file * revert file format * Squashed commit of the following: commit 8d4b8dc Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com> Date: Thu Jul 10 13:14:10 2025 +0800 [Doc] Add an instruction about how to run the SFT example. (#164) commit 3bf9c85 Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com> Date: Thu Jul 10 12:56:24 2025 +0800 [Fix] Merge previous contributions from fw/refactor to lite (#163) * initial proposal * add arealite * . * change api * . * remove LOG_ROOT * remove MODEL_SAVE_PATH * remove PARAM_REALLOC_PATH, DATASET_CACHE * prepare for testing * prepare for testing * ready for run * local run * tests mainly pass * format * . * amend cluster.py * . * . * client test pass * pass rollout test * remove unused imports * add arealite readme * change api * . * . * . * . * . * . * . * . * format * . * implement iteraptable generation (#112) Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com> * . * fix * . * . * . * pass controller generate batch test * . * refactor rollout controller into worker and controller * . * . * . * change to async rollout * pass rollout controller test * pass test * . * update readme * . * sft debug * . * add lisence * remove unused files * remove unsed args in ppo * add hf engine wrapper (#116) * add hf engine * fix issues * fix ppo bugs and add test * add hf client interface and modify cli args * fix bugs * fix issues * Merge fw/refactor * Finish hf wrapper test * add test --------- Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com> * format * format * . * refine hf engine * . * fix * add fsdp engine and sft tests * . * . * . * pass ppo unittest * pass ppo and rollout controller tests * clear unused imports * rename ppo to grpo * change reward function organization * reorganize code * add dataset api * . * . * . * format * chmod fix * . * rename workflow to collector * refactor llm_client location * . * . * fix llm server api * refactor config structure * . * fix tests * . * . * . * Fix unresolved issue in SFTTrainer PR (#139) * . * . * efficient loading * format * . * . * . * . * . * . * Add CI for testing AReaLite (#150) * ci: add test-arealite * ci: add checkout before running test-arealite * ci: add USERNAME * ci: add test script * ci: add GitHub mirror * ci: fix typo * ci: clone one commit * ci: fix condition * ci: set command timeout to 60m * ci: enable pip cache * ci: optimize container lifecycle * ci: split into many stages * ci(test-arealite): fix typo * ci: fix wrong env * ci: fix pytest * ci: uninstall transformer-engine * ci: uninstall transformer-engine * ci: fix model paths * ci: show stdout/stderr * ci: fix not clean up * ci: backup sglang * ci: remove tmp repo dir when run * ci: fix docker run exit 1 condition * ci(test-arealite): limit the concurrency and extend command timeout * . * merge fw/refactor * revert some changes * fix --------- Co-authored-by: meizhiyu.mzy <meizhiyu.mzy@antgroup.com> Co-authored-by: Chayenne <zhaochen20@outlook.com> Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com> Co-authored-by: Jayon02 <qiujiangc@outlook.com> Co-authored-by: root <meizhiyu.mzy> Co-authored-by: Zijian Zhang <futrime@outlook.com> commit d48bf00 Merge: 42c717b b9dbd4a Author: 博惟 <bowei.fw@antgroup.com> Date: Thu Jul 10 12:53:30 2025 +0800 Merge branch 'main' of https://github.com/inclusionAI/AReaL into lite commit 42c717b Merge: c38cffc a203c7c Author: 博惟 <bowei.fw@antgroup.com> Date: Thu Jul 10 11:15:01 2025 +0800 Merge branch 'lite' of https://github.com/inclusionAI/AReaL into lite commit c38cffc Author: 博惟 <bowei.fw@antgroup.com> Date: Thu Jul 10 11:10:10 2025 +0800 PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods Merge branch fw/lite-dev of git@code.alipay.com:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/340 Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com> * support fsdp engine and sglang remote engine * minor fix * . * refactor trainer * add close * rm mb_spec * fix commit b9dbd4a Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com> Date: Wed Jul 9 10:50:19 2025 +0800 Update to persistent wechat QR code. (#159) commit 17ea7fe Author: xssstory <33601810+xssstory@users.noreply.github.com> Date: Mon Jul 7 15:49:13 2025 +0800 fix math reward verifier (#156) * PullRequest: 293 fix get_param_realloc_path Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh https://code.alipay.com/inclusionAI/AReaL/pull_requests/293 Reviewed-by: 博惟 <bowei.fw@antgroup.com> * fix get_param_realloc_path * PullRequest: 297 bugfix: reward is always -5 Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh https://code.alipay.com/inclusionAI/AReaL/pull_requests/297 Reviewed-by: 博惟 <bowei.fw@antgroup.com> * bugfix: reward is always -5 * PullRequest: 321 fix checkpoint save dir Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh https://code.alipay.com/inclusionAI/AReaL/pull_requests/321 Reviewed-by: 博惟 <bowei.fw@antgroup.com> * fix checkpoint save dir * PullRequest: 328 [Doc] update installation Merge branch sxj/doc of git@code.alipay.com:inclusionAI/AReaL.git into gh https://code.alipay.com/inclusionAI/AReaL/pull_requests/328 Reviewed-by: 博惟 <bowei.fw@antgroup.com> * [Doc] update installation * PullRequest: 329 bugfix: math verifier blocks the async training Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh https://code.alipay.com/inclusionAI/AReaL/pull_requests/329 Reviewed-by: 博惟 <bowei.fw@antgroup.com> * bugfix: math verifier block the async training * format --------- Co-authored-by: 冰临 <shenxujie.sxj@antgroup.com> Co-authored-by: garrett4wade <fuwth17@gmail.com> * add autotp for hf * refactor test * fix bugs * fix issues * format files * Squashed commit of the following: commit 9ed043f Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com> Date: Tue Jul 15 10:24:48 2025 +0800 format (#174) commit 8cc9b1f Author: Night <32424487+PrinsYin@users.noreply.github.com> Date: Mon Jul 14 19:22:00 2025 -0700 added LocalSGlangEngine and test (#170) * added LocalSGLangEngine * upload test file * add build args * fix sgl_local generate * improved sgl local robustness * test * test updated * added fallback when sgl engine isn't initialized * finish test local engine * added LocalSGlangEngine and test * format and fix format and fix, raise when generate missing field format * change cli_args.py * add comment header format --------- Co-authored-by: ChangyiYang <changyiyang2023@gmail.com> --------- Co-authored-by: Jayon02 <12012211@mail..sustech.edu.cn> Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
garrett4wade
added a commit
that referenced
this pull request
Aug 1, 2025
* PullRequest: 293 fix get_param_realloc_path
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/293
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix get_param_realloc_path
* PullRequest: 297 bugfix: reward is always -5
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/297
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* bugfix: reward is always -5
* PullRequest: 321 fix checkpoint save dir
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/321
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix checkpoint save dir
* add api
* add directory structure
* add tests template
* format
* checkout previous impl
* checkout previous implementations
* checkout prev impl
* add remote sglang engine
* format
* add readme
* PullRequest: 331 [lite] Support remote sglang engine with corresponding testcases.
Merge branch fw/lite of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/331
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* add test for sglang remote engine
* fix
* PullRequest: 336 add wrapper
Merge branch lite-util-wrapper of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/336
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add wrapper
* PullRequest: 332 [lite] Support FSDP engines
Merge branch mzy/lite/engines of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/332
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* fsdp2 engine
* fix utils
* add fsdp engine test
* .
* fsdp engine test passed
* unsqueeze immediately before model inputs and after model outputts
* add optimizer save/load, add position id calculation for input
* .
* format
* not to squeeze
* add train and eval api
* .
* .
* improve fsdp engine data preprocessing
* format
* PullRequest: 337 [lite] Add SFT trainer example.
* trainer log
* minor changes
* add update weights from disk
* fix type annotation
* PullRequest: 339 [Fix] Fix some minor issues to pass all tests.
Merge branch fw/lite of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/339
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* chore: empty commit
* ci: build images on demand
* ci: fix on demand condition
* ci: fix env sha
* PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods
Merge branch fw/lite-dev of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/340
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* fix
* [Fix] Merge previous contributions from fw/refactor to lite (#163)
* initial proposal
* add arealite
* .
* change api
* .
* remove LOG_ROOT
* remove MODEL_SAVE_PATH
* remove PARAM_REALLOC_PATH, DATASET_CACHE
* prepare for testing
* prepare for testing
* ready for run
* local run
* tests mainly pass
* format
* .
* amend cluster.py
* .
* .
* client test pass
* pass rollout test
* remove unused imports
* add arealite readme
* change api
* .
* .
* .
* .
* .
* .
* .
* .
* format
* .
* implement iteraptable generation (#112)
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
* .
* fix
* .
* .
* .
* pass controller generate batch test
* .
* refactor rollout controller into worker and controller
* .
* .
* .
* change to async rollout
* pass rollout controller test
* pass test
* .
* update readme
* .
* sft debug
* .
* add lisence
* remove unused files
* remove unsed args in ppo
* add hf engine wrapper (#116)
* add hf engine
* fix issues
* fix ppo bugs and add test
* add hf client interface and modify cli args
* fix bugs
* fix issues
* Merge fw/refactor
* Finish hf wrapper test
* add test
---------
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
* format
* format
* .
* refine hf engine
* .
* fix
* add fsdp engine and sft tests
* .
* .
* .
* pass ppo unittest
* pass ppo and rollout controller tests
* clear unused imports
* rename ppo to grpo
* change reward function organization
* reorganize code
* add dataset api
* .
* .
* .
* format
* chmod fix
* .
* rename workflow to collector
* refactor llm_client location
* .
* .
* fix llm server api
* refactor config structure
* .
* fix tests
* .
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139)
* .
* .
* efficient loading
* format
* .
* .
* .
* .
* .
* .
* Add CI for testing AReaLite (#150)
* ci: add test-arealite
* ci: add checkout before running test-arealite
* ci: add USERNAME
* ci: add test script
* ci: add GitHub mirror
* ci: fix typo
* ci: clone one commit
* ci: fix condition
* ci: set command timeout to 60m
* ci: enable pip cache
* ci: optimize container lifecycle
* ci: split into many stages
* ci(test-arealite): fix typo
* ci: fix wrong env
* ci: fix pytest
* ci: uninstall transformer-engine
* ci: uninstall transformer-engine
* ci: fix model paths
* ci: show stdout/stderr
* ci: fix not clean up
* ci: backup sglang
* ci: remove tmp repo dir when run
* ci: fix docker run exit 1 condition
* ci(test-arealite): limit the concurrency and extend command timeout
* .
* merge fw/refactor
* revert some changes
* fix
---------
Co-authored-by: meizhiyu.mzy <meizhiyu.mzy@antgroup.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
Co-authored-by: Jayon02 <qiujiangc@outlook.com>
Co-authored-by: root <meizhiyu.mzy>
Co-authored-by: Zijian Zhang <futrime@outlook.com>
* [Doc] Add an instruction about how to run the SFT example. (#164)
* [Fix] Fix CI running condition for lite. (#172)
* .
* fix
* .
* .
* fix
* fix
* fix
* added LocalSGlangEngine and test (#170)
* added LocalSGLangEngine
* upload test file
* add build args
* fix sgl_local generate
* improved sgl local robustness
* test
* test updated
* added fallback when sgl engine isn't initialized
* finish test local engine
* added LocalSGlangEngine and test
* format and fix
format and fix, raise when generate missing field
format
* change cli_args.py
* add comment header
format
---------
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
* format (#174)
* [Feat][Refactor]Support DeepSpeed AutoTP; Refactor hf_engine.py and unit test. (#161)
* refactor hf engine
* format file
* revert file format
* Squashed commit of the following:
commit 8d4b8dc90fceb144bf00f8125f938b57f696e166
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date: Thu Jul 10 13:14:10 2025 +0800
[Doc] Add an instruction about how to run the SFT example. (#164)
commit 3bf9c85e400a2fe1c6a1e488d27d458ede8cea45
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date: Thu Jul 10 12:56:24 2025 +0800
[Fix] Merge previous contributions from fw/refactor to lite (#163)
* initial proposal
* add arealite
* .
* change api
* .
* remove LOG_ROOT
* remove MODEL_SAVE_PATH
* remove PARAM_REALLOC_PATH, DATASET_CACHE
* prepare for testing
* prepare for testing
* ready for run
* local run
* tests mainly pass
* format
* .
* amend cluster.py
* .
* .
* client test pass
* pass rollout test
* remove unused imports
* add arealite readme
* change api
* .
* .
* .
* .
* .
* .
* .
* .
* format
* .
* implement iteraptable generation (#112)
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
* .
* fix
* .
* .
* .
* pass controller generate batch test
* .
* refactor rollout controller into worker and controller
* .
* .
* .
* change to async rollout
* pass rollout controller test
* pass test
* .
* update readme
* .
* sft debug
* .
* add lisence
* remove unused files
* remove unsed args in ppo
* add hf engine wrapper (#116)
* add hf engine
* fix issues
* fix ppo bugs and add test
* add hf client interface and modify cli args
* fix bugs
* fix issues
* Merge fw/refactor
* Finish hf wrapper test
* add test
---------
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
* format
* format
* .
* refine hf engine
* .
* fix
* add fsdp engine and sft tests
* .
* .
* .
* pass ppo unittest
* pass ppo and rollout controller tests
* clear unused imports
* rename ppo to grpo
* change reward function organization
* reorganize code
* add dataset api
* .
* .
* .
* format
* chmod fix
* .
* rename workflow to collector
* refactor llm_client location
* .
* .
* fix llm server api
* refactor config structure
* .
* fix tests
* .
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139)
* .
* .
* efficient loading
* format
* .
* .
* .
* .
* .
* .
* Add CI for testing AReaLite (#150)
* ci: add test-arealite
* ci: add checkout before running test-arealite
* ci: add USERNAME
* ci: add test script
* ci: add GitHub mirror
* ci: fix typo
* ci: clone one commit
* ci: fix condition
* ci: set command timeout to 60m
* ci: enable pip cache
* ci: optimize container lifecycle
* ci: split into many stages
* ci(test-arealite): fix typo
* ci: fix wrong env
* ci: fix pytest
* ci: uninstall transformer-engine
* ci: uninstall transformer-engine
* ci: fix model paths
* ci: show stdout/stderr
* ci: fix not clean up
* ci: backup sglang
* ci: remove tmp repo dir when run
* ci: fix docker run exit 1 condition
* ci(test-arealite): limit the concurrency and extend command timeout
* .
* merge fw/refactor
* revert some changes
* fix
---------
Co-authored-by: meizhiyu.mzy <meizhiyu.mzy@antgroup.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
Co-authored-by: Jayon02 <qiujiangc@outlook.com>
Co-authored-by: root <meizhiyu.mzy>
Co-authored-by: Zijian Zhang <futrime@outlook.com>
commit d48bf007cfb0821c35a18a673f3c2e8f51a8ee45
Merge: 42c717b b9dbd4a
Author: 博惟 <bowei.fw@antgroup.com>
Date: Thu Jul 10 12:53:30 2025 +0800
Merge branch 'main' of https://github.com/inclusionAI/AReaL into lite
commit 42c717b6e45b3b345f29cff5882ba5374b79faee
Merge: c38cffc a203c7c
Author: 博惟 <bowei.fw@antgroup.com>
Date: Thu Jul 10 11:15:01 2025 +0800
Merge branch 'lite' of https://github.com/inclusionAI/AReaL into lite
commit c38cffc02385acf382196ba6b720b2d8b91a5675
Author: 博惟 <bowei.fw@antgroup.com>
Date: Thu Jul 10 11:10:10 2025 +0800
PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods
Merge branch fw/lite-dev of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/340
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* fix
commit b9dbd4a2c18539648dca373c9c8c1347e4e9db68
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date: Wed Jul 9 10:50:19 2025 +0800
Update to persistent wechat QR code. (#159)
commit 17ea7fe94dc0bd9299082a069383d7372ed13ac9
Author: xssstory <33601810+xssstory@users.noreply.github.com>
Date: Mon Jul 7 15:49:13 2025 +0800
fix math reward verifier (#156)
* PullRequest: 293 fix get_param_realloc_path
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/293
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix get_param_realloc_path
* PullRequest: 297 bugfix: reward is always -5
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/297
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* bugfix: reward is always -5
* PullRequest: 321 fix checkpoint save dir
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/321
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix checkpoint save dir
* PullRequest: 328 [Doc] update installation
Merge branch sxj/doc of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/328
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* [Doc] update installation
* PullRequest: 329 bugfix: math verifier blocks the async training
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/329
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* bugfix: math verifier block the async training
* format
---------
Co-authored-by: 冰临 <shenxujie.sxj@antgroup.com>
Co-authored-by: garrett4wade <fuwth17@gmail.com>
* add autotp for hf
* refactor test
* fix bugs
* fix issues
* format files
* Squashed commit of the following:
commit 9ed043f6ab83cf9c9b38e0251ad08009a73243e8
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date: Tue Jul 15 10:24:48 2025 +0800
format (#174)
commit 8cc9b1feb5ed03cfa42f3ed588dd5f6ec16e5e55
Author: Night <32424487+PrinsYin@users.noreply.github.com>
Date: Mon Jul 14 19:22:00 2025 -0700
added LocalSGlangEngine and test (#170)
* added LocalSGLangEngine
* upload test file
* add build args
* fix sgl_local generate
* improved sgl local robustness
* test
* test updated
* added fallback when sgl engine isn't initialized
* finish test local engine
* added LocalSGlangEngine and test
* format and fix
format and fix, raise when generate missing field
format
* change cli_args.py
* add comment header
format
---------
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
---------
Co-authored-by: Jayon02 <12012211@mail..sustech.edu.cn>
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
* fix ci (#175)
* [Feature] Add pre-commit (#178)
* fix ci
* .
* .
* .
* [lite] [refactor] Add GSM8k GRPO example. (#179)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* fix ci
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* change doc (#180)
* [Fix] [lite] Merge from the internal repo to fix GRPO bugs and refactor the train engine (#181)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* Implement fsdp distributed update (#183)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* added remote nccl weight update
feat: implement update_weights_from_distributed in fsdp_engine.py
unfinishd test, raise PR first
coroutine for each server
chore: change uploads weights behavior, change test order
fix small bug
fixed test
* fix rebase
* add test.sh
* updated, test stil fails
* .
* .
* .
* fix: full_tensor() should happen in all rank (#187)
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
Co-authored-by: 博惟 <bowei.fw@antgroup.com>
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
Co-authored-by: ChangyiYang <112288487+ChangyiYang@users.noreply.github.com>
* [Feature] [lite] Merge from internal dev repo (#189)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* format
* format
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* [Refactor] Rename files in arealite before release. (#190)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* format
* format
* .
* rename
* rename
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* add quickstart (#194)
* [doc] [lite] Add customization docs for AReaLite. (#191)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* .
* .
* .
* .
* .
* .
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* [lite] [feature] Bump to SGLang v0.4.9.post2 and use NCCL to update weights (#196)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* .
* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .
* PullRequest: 423 [lite] Remove the boba example for github release.
Merge branch fw/remove-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* [Docs] [lite] Add example code walkthrough documentation. (#197)
* .
* .
* fix grammar and minor revise
* [lite] [doc] Add AReaLite design doc as README (#198)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* .
* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .
* PullRequest: 423 [lite] Remove the boba example for github release.
Merge branch fw/remove-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* update readme
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* Add self-hosted runner support (#199)
* ci: add testsz for self hosted runner
* ci: use self-hosted runner
* ci: run ci on push
* ci: remove cd to tmp dir
* ci: fix
* ci: add environment back
* ci: set concurrency
* ci: fix build env image
* Update .github/workflows/test-arealite.yml
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update ci/build_env_image.sh
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* ci: add some test code
* ci: simplify
* ci: try to fix
* ci: test
* ci: test2
* ci: test3
* ci: test3
* ci: tidy up
* ci: simplify
* ci: do not trigger on push
* ci: test arealite/tests/
* ci: use UUID as run id
* ci: use openssl rand to genreate run id
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* [WIP][feat] Initial support for VLMs, add Qwen2VL SFT test and Qwen2.5VL GRPO test (#188)
* vlm_sft_test
* vlm_sft_test
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139)
* .
* .
* efficient loading
* format
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139)
* .
* .
* efficient loading
* format
* .
* .
* image_process0701
* image_process0701
* image_process0701_2
* image_process0701_2
* image_process0701_3
* image_process0701_3
* .
* .
* .
* .
* .
* .
* imageprocess0702
* imageprocess0702
* image_process0702_2
* image_process0702_2
* image_process0702_3
* image_process0702_3
* image_process0702_4
* image_process0702_4
* image_process0702_5
* image_process0702_5
* image_process0703_1
* image_process0703_1
* 0703_2
* 0703_2
* 0703_3
* 0703_3
* 0703_4
* 0703_4
* 0703_4
* 0703_4
* 0703_5
* 0703_5
* 0703_6
* 0703_6
* 0703_7
* 0703_7
* 0703_8
* 0703_8
* 0703_9
* 0703_9
* 0703_11
* 0703_11
* 0703_12
* 0703_12
* 0703_13
* 0703_13
* 0703_14
* 0703_14
* 0703_15
* 0703_15
* 0703_16
* 0703_16
* 0703-17
* 0703-17
* 0703_18
* 0703_18
* 0703_18
* 0703_18
* 0703_19
* 0703_19
* 0704_1
* 0704_1
* 0704_2
* 0704_2
* 0704_3
* 0704_3
* .
* .
* 0707_1
* 0707_1
* 0707_2
* 0707_2
* 0703_3
* 0703_3
* r
* p
* fix
* fix
* refactor
* 0707_6
* 0707_7
* refactor1
* 0707_undone
* 0708_1
* 0708_2
* 0708_3
* 0708_7
* 0708_4
* 0709_1
* 0709_2
* 0709_3
* 0709_4
* 0709_5
* 0709_
* 0709_6
* 0709_7
* 0709_7
* 0709_8
* 0709_9
* 0710_1
* 0710_2
* 0710_2
* 0710_3
* 0710_3
* 0710_3
* 0710_5
* 0710_4
* merge_2
* merge_3
* 0711_1
* 0711_2
* 0711_3
* 0711_4
* 0711_6
* 0711_7
* 0711_8
* 0711_8
* 0711_9
* 0711_10
* 0711-11
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* 0714_1
* 0714_2
* 0714_3
* 0714_3
* 0714_5
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* 0715_1
* 0715_2
* 0715_2
* 0716_1
* 0716_2
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* 0716_3
* 0716_4
* 0716_4
* 0716_5
* 0717_1
* 0717_3
* 0717_3
* 0717_4
* 0717_5
* 0717_6
* 0717_6
* 0717_6
* 0718_2
* 0718_4
* 0718_5
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* merge_0721
* 0721_1
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* 0721_2
* 0721_3
* 0721_4
* .
* 0721_formal
* 0721_formal
* 0721_merge4
* 0721_merge5
* 0721_6
* 0721_merge6
* 0721_merge7
* 0721_8
* 0722_1
* 0722_2
* 0722_3
* 0722_4
* 0722_4
* 0722_5
* 0722_6
* 0722_7
* 0723_1
* reformatted
* clang-reformatted
* clang-reformatted2
* 0723_1
* 0723_1
* 0723_1
* 0723_merge3
* 0723_4
* 0723_reformatted_5
* 0724_1
* 0724_1
* 0724_merge1
* 0724_merge2
* 0724_merge3
* 0724_merge3
* 0724_merge4
* 0724_merge5
* 0724_merge6
* 0724_merge7
* 0724_4
* 0724-merge8
* 0724_merge8
* 0725_1
* 0725_6
* 0725_7
* 0725_4padded_image
* 0725_9padded_image
* 0725_10padded_image
* 0725
* 0725_12
* 0725_format
---------
Co-authored-by: bowei.fw <bowei.fw@antgroup.com>
Co-authored-by: nuzant <meizhiyu.mzy@antgroup.com>
Co-authored-by: 朱晗 <lichangye.lcy@antgroup.com>
* [lite] [fix] Fix a performance issue and several minor issues before release (#203)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* .
* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .
* PullRequest: 423 [lite] Remove the boba example for github release.
Merge branch fw/remove-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* update readme
* PullRequest: 431 [Fix] Fix environment of lite
Merge branch fw/lite-dev of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/431
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* change requirements
* .
* .
* .
* PullRequest: 440 [FIX] fix update weight from disk
Merge branch sxj/lite-fix-disk-update of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/440
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* [FIX] fix update weight from disk
* PullRequest: 442 [lite] Refactor `RemoteSGLangEngine` into two parts: `RemoteSGLangEngine` and `WorkflowExecutor`.
Merge branch mzy/workflow-executor of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/442
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* refactor workflow executor
* .
* fix tests and eval
* .
* .
* revert workflow executor into remote sglang engine
* .
* PullRequest: 456 [lite] [Bug] Use `ProcessPoolExecutor` to calculate reward to avoid rollout slow down
Merge branch mzy/lite/fix-reward of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/456?tab=comment
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix reward
* .
* .
* .
* PullRequest: 460 [lite][fix] add a warning when reward computation timeout
Merge branch fw/lite-fix of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/460
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* add a warning when reward computation timeout
* PullRequest: 465 [lite][fix] Fix issues raised by tsao
Merge branch fw/lite-fix of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/465
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* fix
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
Co-authored-by: 冰临 <shenxujie.sxj@antgroup.com>
* [lite] [refactor] Rename the `arealite` folder to `areal` (#205)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* .
* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .
* PullRequest: 423 [lite] Remove the boba example for github release.
Merge branch fw/remove-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* update readme
* PullRequest: 431 [Fix] Fix environment of lite
Merge branch fw/lite-dev of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/431
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* change requirements
* .
* .
* .
* PullRequest: 440 [FIX] fix update weight from disk
Merge branch sxj/lite-fix-disk-update of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/440
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* [FIX] fix update weight from disk
* PullRequest: 442 [lite] Refactor `RemoteSGLangEngine` into two parts: `RemoteSGLangEngine` and `WorkflowExecutor`.
Merge branch mzy/workflow-executor of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/442
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* refactor workflow executor
* .
* fix tests and eval
* .
* .
* revert workflow executor into remote sglang engine
* .
* PullRequest: 456 [lite] [Bug] Use `ProcessPoolExecutor` to calculate reward to avoid rollout slow down
Merge branch mzy/lite/fix-reward of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/456?tab=comment
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix reward
* .
* .
* .
* PullRequest: 460 [lite][fix] add a warning when reward computation timeout
Merge branch fw/lite-fix of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/460
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* add a warning when reward computation timeout
* PullRequest: 465 [lite][fix] Fix issues raised by tsao
Merge branch fw/lite-fix of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/465
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* fix
* PullRequest: 463 [FEAT] add config converter
Merge branch sxj/lite-add-converter of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/463
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* [FEAT] add config converter
* [FIX] fix bugs
* [FIX] fix typos
* [FIX] fix cpu allocation
* [FIX] del vllm
* PullRequest: 466 [lite] Add seeding in training scripts and pad to max_tokens_per_mb to avoid memory fragmentation
Merge branch fw/lite-fix of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/466
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* add seed
* .
* add option for pad to max len
* .
* PullRequest: 467 [lite][fix] remove logging functionality in stats_logger
Merge branch fw/lite-fix of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/467
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* remove logging functionality in stats_logger
* .
* rename arealite to areal
* rename arealite to areal
* .
* local experiment run
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
Co-authored-by: 冰临 <shenxujie.sxj@antgroup.com>
* [lite] [fix] Fix running the legacy experiment in the v0.4.9.post2 sglang environment (#207)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update we…
garrett4wade
added a commit
that referenced
this pull request
Aug 7, 2025
* 0707_1
* 0707_1
* 0707_2
* 0707_2
* 0703_3
* 0703_3
* r
* add api
* add directory structure
* add tests template
* p
* format
* fix
* fix
* checkout previous impl
* checkout previous implementations
* checkout prev impl
* refactor
* add remote sglang engine
* format
* 0707_6
* 0707_7
* add readme
* refactor1
* 0707_undone
* 0708_1
* 0708_2
* 0708_3
* 0708_7
* 0708_4
* 0709_1
* 0709_2
* 0709_3
* PullRequest: 331 [lite] Support remote sglang engine with corresponding testcases.
Merge branch fw/lite of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/331
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* add test for sglang remote engine
* fix
* 0709_4
* 0709_5
* 0709_
* 0709_6
* 0709_7
* PullRequest: 336 add wrapper
Merge branch lite-util-wrapper of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/336
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add wrapper
* 0709_7
* 0709_8
* 0709_9
* PullRequest: 332 [lite] Support FSDP engines
Merge branch mzy/lite/engines of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/332
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* fsdp2 engine
* fix utils
* add fsdp engine test
* .
* fsdp engine test passed
* unsqueeze immediately before model inputs and after model outputts
* add optimizer save/load, add position id calculation for input
* .
* format
* not to squeeze
* add train and eval api
* .
* .
* improve fsdp engine data preprocessing
* format
* PullRequest: 337 [lite] Add SFT trainer example.
* trainer log
* minor changes
* add update weights from disk
* fix type annotation
* PullRequest: 339 [Fix] Fix some minor issues to pass all tests.
Merge branch fw/lite of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/339
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* chore: empty commit
* ci: build images on demand
* ci: fix on demand condition
* ci: fix env sha
* PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods
Merge branch fw/lite-dev of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/340
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* fix
* 0710_1
* 0710_2
* [Fix] Merge previous contributions from fw/refactor to lite (#163)
* initial proposal
* add arealite
* .
* change api
* .
* remove LOG_ROOT
* remove MODEL_SAVE_PATH
* remove PARAM_REALLOC_PATH, DATASET_CACHE
* prepare for testing
* prepare for testing
* ready for run
* local run
* tests mainly pass
* format
* .
* amend cluster.py
* .
* .
* client test pass
* pass rollout test
* remove unused imports
* add arealite readme
* change api
* .
* .
* .
* .
* .
* .
* .
* .
* format
* .
* implement iteraptable generation (#112)
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
* .
* fix
* .
* .
* .
* pass controller generate batch test
* .
* refactor rollout controller into worker and controller
* .
* .
* .
* change to async rollout
* pass rollout controller test
* pass test
* .
* update readme
* .
* sft debug
* .
* add lisence
* remove unused files
* remove unsed args in ppo
* add hf engine wrapper (#116)
* add hf engine
* fix issues
* fix ppo bugs and add test
* add hf client interface and modify cli args
* fix bugs
* fix issues
* Merge fw/refactor
* Finish hf wrapper test
* add test
---------
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
* format
* format
* .
* refine hf engine
* .
* fix
* add fsdp engine and sft tests
* .
* .
* .
* pass ppo unittest
* pass ppo and rollout controller tests
* clear unused imports
* rename ppo to grpo
* change reward function organization
* reorganize code
* add dataset api
* .
* .
* .
* format
* chmod fix
* .
* rename workflow to collector
* refactor llm_client location
* .
* .
* fix llm server api
* refactor config structure
* .
* fix tests
* .
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139)
* .
* .
* efficient loading
* format
* .
* .
* .
* .
* .
* .
* Add CI for testing AReaLite (#150)
* ci: add test-arealite
* ci: add checkout before running test-arealite
* ci: add USERNAME
* ci: add test script
* ci: add GitHub mirror
* ci: fix typo
* ci: clone one commit
* ci: fix condition
* ci: set command timeout to 60m
* ci: enable pip cache
* ci: optimize container lifecycle
* ci: split into many stages
* ci(test-arealite): fix typo
* ci: fix wrong env
* ci: fix pytest
* ci: uninstall transformer-engine
* ci: uninstall transformer-engine
* ci: fix model paths
* ci: show stdout/stderr
* ci: fix not clean up
* ci: backup sglang
* ci: remove tmp repo dir when run
* ci: fix docker run exit 1 condition
* ci(test-arealite): limit the concurrency and extend command timeout
* .
* merge fw/refactor
* revert some changes
* fix
---------
Co-authored-by: meizhiyu.mzy <meizhiyu.mzy@antgroup.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
Co-authored-by: Jayon02 <qiujiangc@outlook.com>
Co-authored-by: root <meizhiyu.mzy>
Co-authored-by: Zijian Zhang <futrime@outlook.com>
* 0710_2
* 0710_3
* [Doc] Add an instruction about how to run the SFT example. (#164)
* 0710_3
* 0710_3
* 0710_5
* 0710_4
* merge_2
* merge_3
* 0711_1
* 0711_2
* 0711_3
* 0711_4
* 0711_6
* 0711_7
* 0711_8
* 0711_8
* 0711_9
* 0711_10
* 0711-11
* [Fix] Fix CI running condition for lite. (#172)
* .
* fix
* .
* .
* fix
* fix
* fix
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* 0714_1
* 0714_2
* 0714_3
* 0714_3
* 0714_5
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* added LocalSGlangEngine and test (#170)
* added LocalSGLangEngine
* upload test file
* add build args
* fix sgl_local generate
* improved sgl local robustness
* test
* test updated
* added fallback when sgl engine isn't initialized
* finish test local engine
* added LocalSGlangEngine and test
* format and fix
format and fix, raise when generate missing field
format
* change cli_args.py
* add comment header
format
---------
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
* format (#174)
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* 0715_1
* 0715_2
* 0715_2
* [Feat][Refactor]Support DeepSpeed AutoTP; Refactor hf_engine.py and unit test. (#161)
* refactor hf engine
* format file
* revert file format
* Squashed commit of the following:
commit 8d4b8dc90fceb144bf00f8125f938b57f696e166
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date: Thu Jul 10 13:14:10 2025 +0800
[Doc] Add an instruction about how to run the SFT example. (#164)
commit 3bf9c85e400a2fe1c6a1e488d27d458ede8cea45
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date: Thu Jul 10 12:56:24 2025 +0800
[Fix] Merge previous contributions from fw/refactor to lite (#163)
* initial proposal
* add arealite
* .
* change api
* .
* remove LOG_ROOT
* remove MODEL_SAVE_PATH
* remove PARAM_REALLOC_PATH, DATASET_CACHE
* prepare for testing
* prepare for testing
* ready for run
* local run
* tests mainly pass
* format
* .
* amend cluster.py
* .
* .
* client test pass
* pass rollout test
* remove unused imports
* add arealite readme
* change api
* .
* .
* .
* .
* .
* .
* .
* .
* format
* .
* implement iteraptable generation (#112)
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
* .
* fix
* .
* .
* .
* pass controller generate batch test
* .
* refactor rollout controller into worker and controller
* .
* .
* .
* change to async rollout
* pass rollout controller test
* pass test
* .
* update readme
* .
* sft debug
* .
* add lisence
* remove unused files
* remove unsed args in ppo
* add hf engine wrapper (#116)
* add hf engine
* fix issues
* fix ppo bugs and add test
* add hf client interface and modify cli args
* fix bugs
* fix issues
* Merge fw/refactor
* Finish hf wrapper test
* add test
---------
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
* format
* format
* .
* refine hf engine
* .
* fix
* add fsdp engine and sft tests
* .
* .
* .
* pass ppo unittest
* pass ppo and rollout controller tests
* clear unused imports
* rename ppo to grpo
* change reward function organization
* reorganize code
* add dataset api
* .
* .
* .
* format
* chmod fix
* .
* rename workflow to collector
* refactor llm_client location
* .
* .
* fix llm server api
* refactor config structure
* .
* fix tests
* .
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139)
* .
* .
* efficient loading
* format
* .
* .
* .
* .
* .
* .
* Add CI for testing AReaLite (#150)
* ci: add test-arealite
* ci: add checkout before running test-arealite
* ci: add USERNAME
* ci: add test script
* ci: add GitHub mirror
* ci: fix typo
* ci: clone one commit
* ci: fix condition
* ci: set command timeout to 60m
* ci: enable pip cache
* ci: optimize container lifecycle
* ci: split into many stages
* ci(test-arealite): fix typo
* ci: fix wrong env
* ci: fix pytest
* ci: uninstall transformer-engine
* ci: uninstall transformer-engine
* ci: fix model paths
* ci: show stdout/stderr
* ci: fix not clean up
* ci: backup sglang
* ci: remove tmp repo dir when run
* ci: fix docker run exit 1 condition
* ci(test-arealite): limit the concurrency and extend command timeout
* .
* merge fw/refactor
* revert some changes
* fix
---------
Co-authored-by: meizhiyu.mzy <meizhiyu.mzy@antgroup.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
Co-authored-by: Jayon02 <qiujiangc@outlook.com>
Co-authored-by: root <meizhiyu.mzy>
Co-authored-by: Zijian Zhang <futrime@outlook.com>
commit d48bf007cfb0821c35a18a673f3c2e8f51a8ee45
Merge: 42c717b b9dbd4a
Author: 博惟 <bowei.fw@antgroup.com>
Date: Thu Jul 10 12:53:30 2025 +0800
Merge branch 'main' of https://github.com/inclusionAI/AReaL into lite
commit 42c717b6e45b3b345f29cff5882ba5374b79faee
Merge: c38cffc a203c7c
Author: 博惟 <bowei.fw@antgroup.com>
Date: Thu Jul 10 11:15:01 2025 +0800
Merge branch 'lite' of https://github.com/inclusionAI/AReaL into lite
commit c38cffc02385acf382196ba6b720b2d8b91a5675
Author: 博惟 <bowei.fw@antgroup.com>
Date: Thu Jul 10 11:10:10 2025 +0800
PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods
Merge branch fw/lite-dev of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/340
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* fix
commit b9dbd4a2c18539648dca373c9c8c1347e4e9db68
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date: Wed Jul 9 10:50:19 2025 +0800
Update to persistent wechat QR code. (#159)
commit 17ea7fe94dc0bd9299082a069383d7372ed13ac9
Author: xssstory <33601810+xssstory@users.noreply.github.com>
Date: Mon Jul 7 15:49:13 2025 +0800
fix math reward verifier (#156)
* PullRequest: 293 fix get_param_realloc_path
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/293
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix get_param_realloc_path
* PullRequest: 297 bugfix: reward is always -5
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/297
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* bugfix: reward is always -5
* PullRequest: 321 fix checkpoint save dir
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/321
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix checkpoint save dir
* PullRequest: 328 [Doc] update installation
Merge branch sxj/doc of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/328
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* [Doc] update installation
* PullRequest: 329 bugfix: math verifier blocks the async training
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/329
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* bugfix: math verifier block the async training
* format
---------
Co-authored-by: 冰临 <shenxujie.sxj@antgroup.com>
Co-authored-by: garrett4wade <fuwth17@gmail.com>
* add autotp for hf
* refactor test
* fix bugs
* fix issues
* format files
* Squashed commit of the following:
commit 9ed043f6ab83cf9c9b38e0251ad08009a73243e8
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date: Tue Jul 15 10:24:48 2025 +0800
format (#174)
commit 8cc9b1feb5ed03cfa42f3ed588dd5f6ec16e5e55
Author: Night <32424487+PrinsYin@users.noreply.github.com>
Date: Mon Jul 14 19:22:00 2025 -0700
added LocalSGlangEngine and test (#170)
* added LocalSGLangEngine
* upload test file
* add build args
* fix sgl_local generate
* improved sgl local robustness
* test
* test updated
* added fallback when sgl engine isn't initialized
* finish test local engine
* added LocalSGlangEngine and test
* format and fix
format and fix, raise when generate missing field
format
* change cli_args.py
* add comment header
format
---------
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
---------
Co-authored-by: Jayon02 <12012211@mail..sustech.edu.cn>
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
* fix ci (#175)
* [Feature] Add pre-commit (#178)
* fix ci
* .
* .
* .
* [lite] [refactor] Add GSM8k GRPO example. (#179)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* fix ci
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* 0716_1
* 0716_2
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* change doc (#180)
* [Fix] [lite] Merge from the internal repo to fix GRPO bugs and refactor the train engine (#181)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* 0716_3
* 0716_4
* 0716_4
* 0716_5
* 0717_1
* 0717_3
* 0717_3
* 0717_4
* 0717_5
* 0717_6
* 0717_6
* 0717_6
* 0718_2
* 0718_4
* 0718_5
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* merge_0721
* 0721_1
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* 0721_2
* 0721_3
* Implement fsdp distributed update (#183)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* added remote nccl weight update
feat: implement update_weights_from_distributed in fsdp_engine.py
unfinishd test, raise PR first
coroutine for each server
chore: change uploads weights behavior, change test order
fix small bug
fixed test
* fix rebase
* add test.sh
* updated, test stil fails
* .
* .
* .
* fix: full_tensor() should happen in all rank (#187)
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
Co-authored-by: 博惟 <bowei.fw@antgroup.com>
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
Co-authored-by: ChangyiYang <112288487+ChangyiYang@users.noreply.github.com>
* [Feature] [lite] Merge from internal dev repo (#189)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* format
* format
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* [Refactor] Rename files in arealite before release. (#190)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* format
* format
* .
* rename
* rename
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* 0721_4
* .
* 0721_formal
* 0721_formal
* 0721_merge4
* 0721_merge5
* 0721_6
* 0721_merge6
* 0721_merge7
* 0721_8
* 0722_1
* 0722_2
* 0722_3
* add quickstart (#194)
* [doc] [lite] Add customization docs for AReaLite. (#191)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* .
* .
* .
* .
* .
* .
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* 0722_4
* 0722_4
* 0722_5
* 0722_6
* 0722_7
* 0723_1
* reformatted
* clang-reformatted
* clang-reformatted2
* 0723_1
* 0723_1
* 0723_1
* 0723_merge3
* 0723_4
* 0723_reformatted_5
* 0724_1
* 0724_1
* 0724_merge1
* 0724_merge2
* 0724_merge3
* 0724_merge3
* 0724_merge4
* [lite] [feature] Bump to SGLang v0.4.9.post2 and use NCCL to update weights (#196)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* .
* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .
* PullRequest: 423 [lite] Remove the boba example for github release.
Merge branch fw/remove-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* 0724_merge5
* 0724_merge6
* 0724_merge7
* [Docs] [lite] Add example code walkthrough documentation. (#197)
* .
* .
* fix grammar and minor revise
* [lite] [doc] Add AReaLite design doc as README (#198)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* .
* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .
* PullRequest: 423 [lite] Remove the boba example for github release.
Merge branch fw/remove-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* update readme
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* 0724_4
* 0724-merge8
* 0724_merge8
* 0725_1
* 0725_6
* 0725_7
* 0725_4padded_image
* 0725_9padded_image
* 0725_10padded_image
* 0725
* 0725_12
* 0725_format
* Add self-hosted runner support (#199)
* ci: add testsz for self hosted runner
* ci: use self-hosted runner
* ci: run ci on push
* ci: remove cd to tmp dir
* ci: fix
* ci: add environment back
* ci: set concurrency
* ci: fix build env image
* Update .github/workflows/test-arealite.yml
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update ci/build_env_image.sh
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* ci: add some test code
* ci: simplify
* ci: try to fix
* ci: test
* ci: test2
* ci: test3
* ci: test3
* ci: tidy up
* ci: simplify
* ci: do not trigger on push
* ci: test arealite/tests/
* ci: use UUID as run id
* ci: use openssl rand to genreate run id
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* [WIP][feat] Initial support for VLMs, add Qwen2VL SFT test and Qwen2.5VL GRPO test (#188)
* vlm_sft_test
* vlm_sft_test
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139)
* .
* .
* efficient loading
* format
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139)
* .
* .
* efficient loading
* format
* .
* .
* image_process0701
* image_process0701
* image_process0701_2
* image_process0701_2
* image_process0701_3
* image_process0701_3
* .
* .
* .
* .
* .
* .
* imageprocess0702
* imageprocess0702
* image_process0702_2
* image_process0702_2
* image_process0702_3
* image_process0702_3
* image_process0702_4
* image_process0702_4
* image_process0702_5
* image_process0702_5
* image_process0703_1
* image_process0703_1
* 0703_2
* 0703_2
* 0703_3
* 0703_3
* 0703_4
* 0703_4
* 0703_4
* 0703_4
* 0703_5
* 0703_5
* 0703_6
* 0703_6
* 0703_7
* 0703_7
* 0703_8
* 0703_8
* 0703_9
* 0703_9
* 0703_11
* 0703_11
* 0703_12
* 0703_12
* 0703_13
* 0703_13
* 0703_14
* 0703_14
* 0703_15
* 0703_15
* 0703_16
* 0703_16
* 0703-17
* 0703-17
* 0703_18
* 0703_18
* 0703_18
* 0703_18
* 0703_19
* 0703_19
* 0704_1
* 0704_1
* 0704_2
* 0704_2
* 0704_3
* 0704_3
* .
* .
* 0707_1
* 0707_1
* 0707_2
* 0707_2
* 0703_3
* 0703_3
* r
* p
* fix
* fix
* refactor
* 0707_6
* 0707_7
* refactor1
* 0707_undone
* 0708_1
* 0708_2
* 0708_3
* 0708_7
* 0708_4
* 0709_1
* 0709_2
* 0709_3
* 0709_4
* 0709_5
* 0709_
* 0709_6
* 0709_7
* 0709_7
* 0709_8
* 0709_9
* 0710_1
* 0710_2
* 0710_2
* 0710_3
* 0710_3
* 0710_3
* 0710_5
* 0710_4
* merge_2
* merge_3
* 0711_1
* 0711_2
* 0711_3
* 0711_4
* 0711_6
* 0711_7
* 0711_8
* 0711_8
* 0711_9
* 0711_10
* 0711-11
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* 0714_1
* 0714_2
* 0714_3
* 0714_3
* 0714_5
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* 0715_1
* 0715_2
* 0715_2
* 0716_1
* 0716_2
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* 0716_3
* 0716_4
* 0716_4
* 0716_5
* 0717_1
* 0717_3
* 0717_3
* 0717_4
* 0717_5
* 0717_6
* 0717_6
* 0717_6
* 0718_2
* 0718_4
* 0718_5
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* merge_0721
* 0721_1
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* 0721_2
* 0721_3
* 0721_4
* .
* 0721_formal
* 0721_formal
* 0721_merge4
* 0721_merge5
* 0721_6
* 0721_merge6
* 0721_merge7
* 0721_8
* 0722_1
* 0722_2
* 0722_3
* 0722_4
* 0722_4
* 0722_5
* 0722_6
* 0722_7
* 0723_1
* reformatted
* clang-reformatted
* clang-reformatted2
* 0723_1
* 0723_1
* 0723_1
* 0723_merge3
* 0723_4
* 0723_reformatted_5
* 0724_1
* 0724_1
* 0724_merge1
* 0724_merge2
* 0724_merge3
* 0724_merge3
* 0724_merge4
* 0724_merge5
* 0724_merge6
* 0724_merge7
* 0724_4
* 0724-merge8
* 0724_merge8
* 0725_1
* 0725_6
* 0725_7
* 0725_4padded_image
* 0725_9padded_image
* 0725_10padded_image
* 0725
* 0725_12
* 0725_format
---------
Co-authored-by: bowei.fw <bowei.fw@antgroup.com>
Co-authored-by: nuzant <meizhiyu.mzy@antgroup.com>
Co-authored-by: 朱晗 <lichangye.lcy@antgroup.com>
* 0731
* 0731_1
* 0731_2
* 0731_2
* 0731_3
* 0731_4
* 0801_1
* 0801_2
* 0804_1
* 0804_2
* 0804_2
* 0804_5
* 0805_3
* 0805_2
* 0806
* 0806_merge1
* 0806_merge2
* 0806_format1
* 0806_merge3
* 0806_4
* 0806_6
* 0806_7
* 0806_formatted2
* fix
* revert examples
* .
* .
---------
Co-authored-by: 朱晗 <lichangye.lcy@antgroup.com>
Co-authored-by: 博惟 <bowei.fw@antgroup.com>
Co-authored-by: garrett4wade <fuwth17@gmail.com>
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
mjbmjb
pushed a commit
to mjbmjb/AReaL
that referenced
this pull request
Sep 22, 2025
* PullRequest: 293 fix get_param_realloc_path
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/293
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix get_param_realloc_path
* PullRequest: 297 bugfix: reward is always -5
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/297
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* bugfix: reward is always -5
* PullRequest: 321 fix checkpoint save dir
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/321
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix checkpoint save dir
* add api
* add directory structure
* add tests template
* format
* checkout previous impl
* checkout previous implementations
* checkout prev impl
* add remote sglang engine
* format
* add readme
* PullRequest: 331 [lite] Support remote sglang engine with corresponding testcases.
Merge branch fw/lite of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/331
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* add test for sglang remote engine
* fix
* PullRequest: 336 add wrapper
Merge branch lite-util-wrapper of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/336
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add wrapper
* PullRequest: 332 [lite] Support FSDP engines
Merge branch mzy/lite/engines of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/332
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* fsdp2 engine
* fix utils
* add fsdp engine test
* .
* fsdp engine test passed
* unsqueeze immediately before model inputs and after model outputts
* add optimizer save/load, add position id calculation for input
* .
* format
* not to squeeze
* add train and eval api
* .
* .
* improve fsdp engine data preprocessing
* format
* PullRequest: 337 [lite] Add SFT trainer example.
* trainer log
* minor changes
* add update weights from disk
* fix type annotation
* PullRequest: 339 [Fix] Fix some minor issues to pass all tests.
Merge branch fw/lite of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/339
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* chore: empty commit
* ci: build images on demand
* ci: fix on demand condition
* ci: fix env sha
* PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods
Merge branch fw/lite-dev of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/340
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* fix
* [Fix] Merge previous contributions from fw/refactor to lite (#163)
* initial proposal
* add arealite
* .
* change api
* .
* remove LOG_ROOT
* remove MODEL_SAVE_PATH
* remove PARAM_REALLOC_PATH, DATASET_CACHE
* prepare for testing
* prepare for testing
* ready for run
* local run
* tests mainly pass
* format
* .
* amend cluster.py
* .
* .
* client test pass
* pass rollout test
* remove unused imports
* add arealite readme
* change api
* .
* .
* .
* .
* .
* .
* .
* .
* format
* .
* implement iteraptable generation (#112)
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
* .
* fix
* .
* .
* .
* pass controller generate batch test
* .
* refactor rollout controller into worker and controller
* .
* .
* .
* change to async rollout
* pass rollout controller test
* pass test
* .
* update readme
* .
* sft debug
* .
* add lisence
* remove unused files
* remove unsed args in ppo
* add hf engine wrapper (#116)
* add hf engine
* fix issues
* fix ppo bugs and add test
* add hf client interface and modify cli args
* fix bugs
* fix issues
* Merge fw/refactor
* Finish hf wrapper test
* add test
---------
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
* format
* format
* .
* refine hf engine
* .
* fix
* add fsdp engine and sft tests
* .
* .
* .
* pass ppo unittest
* pass ppo and rollout controller tests
* clear unused imports
* rename ppo to grpo
* change reward function organization
* reorganize code
* add dataset api
* .
* .
* .
* format
* chmod fix
* .
* rename workflow to collector
* refactor llm_client location
* .
* .
* fix llm server api
* refactor config structure
* .
* fix tests
* .
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139)
* .
* .
* efficient loading
* format
* .
* .
* .
* .
* .
* .
* Add CI for testing AReaLite (#150)
* ci: add test-arealite
* ci: add checkout before running test-arealite
* ci: add USERNAME
* ci: add test script
* ci: add GitHub mirror
* ci: fix typo
* ci: clone one commit
* ci: fix condition
* ci: set command timeout to 60m
* ci: enable pip cache
* ci: optimize container lifecycle
* ci: split into many stages
* ci(test-arealite): fix typo
* ci: fix wrong env
* ci: fix pytest
* ci: uninstall transformer-engine
* ci: uninstall transformer-engine
* ci: fix model paths
* ci: show stdout/stderr
* ci: fix not clean up
* ci: backup sglang
* ci: remove tmp repo dir when run
* ci: fix docker run exit 1 condition
* ci(test-arealite): limit the concurrency and extend command timeout
* .
* merge fw/refactor
* revert some changes
* fix
---------
Co-authored-by: meizhiyu.mzy <meizhiyu.mzy@antgroup.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
Co-authored-by: Jayon02 <qiujiangc@outlook.com>
Co-authored-by: root <meizhiyu.mzy>
Co-authored-by: Zijian Zhang <futrime@outlook.com>
* [Doc] Add an instruction about how to run the SFT example. (#164)
* [Fix] Fix CI running condition for lite. (#172)
* .
* fix
* .
* .
* fix
* fix
* fix
* added LocalSGlangEngine and test (#170)
* added LocalSGLangEngine
* upload test file
* add build args
* fix sgl_local generate
* improved sgl local robustness
* test
* test updated
* added fallback when sgl engine isn't initialized
* finish test local engine
* added LocalSGlangEngine and test
* format and fix
format and fix, raise when generate missing field
format
* change cli_args.py
* add comment header
format
---------
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
* format (#174)
* [Feat][Refactor]Support DeepSpeed AutoTP; Refactor hf_engine.py and unit test. (#161)
* refactor hf engine
* format file
* revert file format
* Squashed commit of the following:
commit 8d4b8dc90fceb144bf00f8125f938b57f696e166
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date: Thu Jul 10 13:14:10 2025 +0800
[Doc] Add an instruction about how to run the SFT example. (#164)
commit 3bf9c85e400a2fe1c6a1e488d27d458ede8cea45
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date: Thu Jul 10 12:56:24 2025 +0800
[Fix] Merge previous contributions from fw/refactor to lite (#163)
* initial proposal
* add arealite
* .
* change api
* .
* remove LOG_ROOT
* remove MODEL_SAVE_PATH
* remove PARAM_REALLOC_PATH, DATASET_CACHE
* prepare for testing
* prepare for testing
* ready for run
* local run
* tests mainly pass
* format
* .
* amend cluster.py
* .
* .
* client test pass
* pass rollout test
* remove unused imports
* add arealite readme
* change api
* .
* .
* .
* .
* .
* .
* .
* .
* format
* .
* implement iteraptable generation (#112)
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
* .
* fix
* .
* .
* .
* pass controller generate batch test
* .
* refactor rollout controller into worker and controller
* .
* .
* .
* change to async rollout
* pass rollout controller test
* pass test
* .
* update readme
* .
* sft debug
* .
* add lisence
* remove unused files
* remove unsed args in ppo
* add hf engine wrapper (#116)
* add hf engine
* fix issues
* fix ppo bugs and add test
* add hf client interface and modify cli args
* fix bugs
* fix issues
* Merge fw/refactor
* Finish hf wrapper test
* add test
---------
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
* format
* format
* .
* refine hf engine
* .
* fix
* add fsdp engine and sft tests
* .
* .
* .
* pass ppo unittest
* pass ppo and rollout controller tests
* clear unused imports
* rename ppo to grpo
* change reward function organization
* reorganize code
* add dataset api
* .
* .
* .
* format
* chmod fix
* .
* rename workflow to collector
* refactor llm_client location
* .
* .
* fix llm server api
* refactor config structure
* .
* fix tests
* .
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139)
* .
* .
* efficient loading
* format
* .
* .
* .
* .
* .
* .
* Add CI for testing AReaLite (#150)
* ci: add test-arealite
* ci: add checkout before running test-arealite
* ci: add USERNAME
* ci: add test script
* ci: add GitHub mirror
* ci: fix typo
* ci: clone one commit
* ci: fix condition
* ci: set command timeout to 60m
* ci: enable pip cache
* ci: optimize container lifecycle
* ci: split into many stages
* ci(test-arealite): fix typo
* ci: fix wrong env
* ci: fix pytest
* ci: uninstall transformer-engine
* ci: uninstall transformer-engine
* ci: fix model paths
* ci: show stdout/stderr
* ci: fix not clean up
* ci: backup sglang
* ci: remove tmp repo dir when run
* ci: fix docker run exit 1 condition
* ci(test-arealite): limit the concurrency and extend command timeout
* .
* merge fw/refactor
* revert some changes
* fix
---------
Co-authored-by: meizhiyu.mzy <meizhiyu.mzy@antgroup.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
Co-authored-by: Jayon02 <qiujiangc@outlook.com>
Co-authored-by: root <meizhiyu.mzy>
Co-authored-by: Zijian Zhang <futrime@outlook.com>
commit d48bf007cfb0821c35a18a673f3c2e8f51a8ee45
Merge: 42c717b 743896d
Author: 博惟 <bowei.fw@antgroup.com>
Date: Thu Jul 10 12:53:30 2025 +0800
Merge branch 'main' of https://github.com/inclusionAI/AReaL into lite
commit 42c717b6e45b3b345f29cff5882ba5374b79faee
Merge: c38cffc a203c7c
Author: 博惟 <bowei.fw@antgroup.com>
Date: Thu Jul 10 11:15:01 2025 +0800
Merge branch 'lite' of https://github.com/inclusionAI/AReaL into lite
commit c38cffc02385acf382196ba6b720b2d8b91a5675
Author: 博惟 <bowei.fw@antgroup.com>
Date: Thu Jul 10 11:10:10 2025 +0800
PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods
Merge branch fw/lite-dev of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/340
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* fix
commit 743896dbd3938fb879dc67cb9990462236778193
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date: Wed Jul 9 10:50:19 2025 +0800
Update to persistent wechat QR code. (#159)
commit 80e16378237dd89878bdf79e5992932550df9493
Author: xssstory <33601810+xssstory@users.noreply.github.com>
Date: Mon Jul 7 15:49:13 2025 +0800
fix math reward verifier (#156)
* PullRequest: 293 fix get_param_realloc_path
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/293
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix get_param_realloc_path
* PullRequest: 297 bugfix: reward is always -5
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/297
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* bugfix: reward is always -5
* PullRequest: 321 fix checkpoint save dir
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/321
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix checkpoint save dir
* PullRequest: 328 [Doc] update installation
Merge branch sxj/doc of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/328
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* [Doc] update installation
* PullRequest: 329 bugfix: math verifier blocks the async training
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/329
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* bugfix: math verifier block the async training
* format
---------
Co-authored-by: 冰临 <shenxujie.sxj@antgroup.com>
Co-authored-by: garrett4wade <fuwth17@gmail.com>
* add autotp for hf
* refactor test
* fix bugs
* fix issues
* format files
* Squashed commit of the following:
commit 9ed043f6ab83cf9c9b38e0251ad08009a73243e8
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date: Tue Jul 15 10:24:48 2025 +0800
format (#174)
commit 8cc9b1feb5ed03cfa42f3ed588dd5f6ec16e5e55
Author: Night <32424487+PrinsYin@users.noreply.github.com>
Date: Mon Jul 14 19:22:00 2025 -0700
added LocalSGlangEngine and test (#170)
* added LocalSGLangEngine
* upload test file
* add build args
* fix sgl_local generate
* improved sgl local robustness
* test
* test updated
* added fallback when sgl engine isn't initialized
* finish test local engine
* added LocalSGlangEngine and test
* format and fix
format and fix, raise when generate missing field
format
* change cli_args.py
* add comment header
format
---------
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
---------
Co-authored-by: Jayon02 <12012211@mail..sustech.edu.cn>
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
* fix ci (#175)
* [Feature] Add pre-commit (#178)
* fix ci
* .
* .
* .
* [lite] [refactor] Add GSM8k GRPO example. (#179)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* fix ci
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* change doc (#180)
* [Fix] [lite] Merge from the internal repo to fix GRPO bugs and refactor the train engine (#181)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* Implement fsdp distributed update (#183)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* added remote nccl weight update
feat: implement update_weights_from_distributed in fsdp_engine.py
unfinishd test, raise PR first
coroutine for each server
chore: change uploads weights behavior, change test order
fix small bug
fixed test
* fix rebase
* add test.sh
* updated, test stil fails
* .
* .
* .
* fix: full_tensor() should happen in all rank (#187)
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
Co-authored-by: 博惟 <bowei.fw@antgroup.com>
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
Co-authored-by: ChangyiYang <112288487+ChangyiYang@users.noreply.github.com>
* [Feature] [lite] Merge from internal dev repo (#189)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* format
* format
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* [Refactor] Rename files in arealite before release. (#190)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* format
* format
* .
* rename
* rename
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* add quickstart (#194)
* [doc] [lite] Add customization docs for AReaLite. (#191)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* .
* .
* .
* .
* .
* .
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* [lite] [feature] Bump to SGLang v0.4.9.post2 and use NCCL to update weights (#196)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* .
* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .
* PullRequest: 423 [lite] Remove the boba example for github release.
Merge branch fw/remove-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* [Docs] [lite] Add example code walkthrough documentation. (#197)
* .
* .
* fix grammar and minor revise
* [lite] [doc] Add AReaLite design doc as README (#198)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* .
* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .
* PullRequest: 423 [lite] Remove the boba example for github release.
Merge branch fw/remove-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* update readme
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* Add self-hosted runner support (#199)
* ci: add testsz for self hosted runner
* ci: use self-hosted runner
* ci: run ci on push
* ci: remove cd to tmp dir
* ci: fix
* ci: add environment back
* ci: set concurrency
* ci: fix build env image
* Update .github/workflows/test-arealite.yml
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update ci/build_env_image.sh
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* ci: add some test code
* ci: simplify
* ci: try to fix
* ci: test
* ci: test2
* ci: test3
* ci: test3
* ci: tidy up
* ci: simplify
* ci: do not trigger on push
* ci: test arealite/tests/
* ci: use UUID as run id
* ci: use openssl rand to genreate run id
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* [WIP][feat] Initial support for VLMs, add Qwen2VL SFT test and Qwen2.5VL GRPO test (#188)
* vlm_sft_test
* vlm_sft_test
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139)
* .
* .
* efficient loading
* format
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139)
* .
* .
* efficient loading
* format
* .
* .
* image_process0701
* image_process0701
* image_process0701_2
* image_process0701_2
* image_process0701_3
* image_process0701_3
* .
* .
* .
* .
* .
* .
* imageprocess0702
* imageprocess0702
* image_process0702_2
* image_process0702_2
* image_process0702_3
* image_process0702_3
* image_process0702_4
* image_process0702_4
* image_process0702_5
* image_process0702_5
* image_process0703_1
* image_process0703_1
* 0703_2
* 0703_2
* 0703_3
* 0703_3
* 0703_4
* 0703_4
* 0703_4
* 0703_4
* 0703_5
* 0703_5
* 0703_6
* 0703_6
* 0703_7
* 0703_7
* 0703_8
* 0703_8
* 0703_9
* 0703_9
* 0703_11
* 0703_11
* 0703_12
* 0703_12
* 0703_13
* 0703_13
* 0703_14
* 0703_14
* 0703_15
* 0703_15
* 0703_16
* 0703_16
* 0703-17
* 0703-17
* 0703_18
* 0703_18
* 0703_18
* 0703_18
* 0703_19
* 0703_19
* 0704_1
* 0704_1
* 0704_2
* 0704_2
* 0704_3
* 0704_3
* .
* .
* 0707_1
* 0707_1
* 0707_2
* 0707_2
* 0703_3
* 0703_3
* r
* p
* fix
* fix
* refactor
* 0707_6
* 0707_7
* refactor1
* 0707_undone
* 0708_1
* 0708_2
* 0708_3
* 0708_7
* 0708_4
* 0709_1
* 0709_2
* 0709_3
* 0709_4
* 0709_5
* 0709_
* 0709_6
* 0709_7
* 0709_7
* 0709_8
* 0709_9
* 0710_1
* 0710_2
* 0710_2
* 0710_3
* 0710_3
* 0710_3
* 0710_5
* 0710_4
* merge_2
* merge_3
* 0711_1
* 0711_2
* 0711_3
* 0711_4
* 0711_6
* 0711_7
* 0711_8
* 0711_8
* 0711_9
* 0711_10
* 0711-11
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* 0714_1
* 0714_2
* 0714_3
* 0714_3
* 0714_5
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* 0715_1
* 0715_2
* 0715_2
* 0716_1
* 0716_2
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* 0716_3
* 0716_4
* 0716_4
* 0716_5
* 0717_1
* 0717_3
* 0717_3
* 0717_4
* 0717_5
* 0717_6
* 0717_6
* 0717_6
* 0718_2
* 0718_4
* 0718_5
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* merge_0721
* 0721_1
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* 0721_2
* 0721_3
* 0721_4
* .
* 0721_formal
* 0721_formal
* 0721_merge4
* 0721_merge5
* 0721_6
* 0721_merge6
* 0721_merge7
* 0721_8
* 0722_1
* 0722_2
* 0722_3
* 0722_4
* 0722_4
* 0722_5
* 0722_6
* 0722_7
* 0723_1
* reformatted
* clang-reformatted
* clang-reformatted2
* 0723_1
* 0723_1
* 0723_1
* 0723_merge3
* 0723_4
* 0723_reformatted_5
* 0724_1
* 0724_1
* 0724_merge1
* 0724_merge2
* 0724_merge3
* 0724_merge3
* 0724_merge4
* 0724_merge5
* 0724_merge6
* 0724_merge7
* 0724_4
* 0724-merge8
* 0724_merge8
* 0725_1
* 0725_6
* 0725_7
* 0725_4padded_image
* 0725_9padded_image
* 0725_10padded_image
* 0725
* 0725_12
* 0725_format
---------
Co-authored-by: bowei.fw <bowei.fw@antgroup.com>
Co-authored-by: nuzant <meizhiyu.mzy@antgroup.com>
Co-authored-by: 朱晗 <lichangye.lcy@antgroup.com>
* [lite] [fix] Fix a performance issue and several minor issues before release (#203)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* .
* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .
* PullRequest: 423 [lite] Remove the boba example for github release.
Merge branch fw/remove-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* update readme
* PullRequest: 431 [Fix] Fix environment of lite
Merge branch fw/lite-dev of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/431
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* change requirements
* .
* .
* .
* PullRequest: 440 [FIX] fix update weight from disk
Merge branch sxj/lite-fix-disk-update of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/440
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* [FIX] fix update weight from disk
* PullRequest: 442 [lite] Refactor `RemoteSGLangEngine` into two parts: `RemoteSGLangEngine` and `WorkflowExecutor`.
Merge branch mzy/workflow-executor of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/442
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* refactor workflow executor
* .
* fix tests and eval
* .
* .
* revert workflow executor into remote sglang engine
* .
* PullRequest: 456 [lite] [Bug] Use `ProcessPoolExecutor` to calculate reward to avoid rollout slow down
Merge branch mzy/lite/fix-reward of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/456?tab=comment
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix reward
* .
* .
* .
* PullRequest: 460 [lite][fix] add a warning when reward computation timeout
Merge branch fw/lite-fix of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/460
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* add a warning when reward computation timeout
* PullRequest: 465 [lite][fix] Fix issues raised by tsao
Merge branch fw/lite-fix of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/465
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* fix
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
Co-authored-by: 冰临 <shenxujie.sxj@antgroup.com>
* [lite] [refactor] Rename the `arealite` folder to `areal` (#205)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* .
* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .
* PullRequest: 423 [lite] Remove the boba example for github release.
Merge branch fw/remove-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* update readme
* PullRequest: 431 [Fix] Fix environment of lite
Merge branch fw/lite-dev of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/431
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* change requirements
* .
* .
* .
* PullRequest: 440 [FIX] fix update weight from disk
Merge branch sxj/lite-fix-disk-update of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/440
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* [FIX] fix update weight from disk
* PullRequest: 442 [lite] Refactor `RemoteSGLangEngine` into two parts: `RemoteSGLangEngine` and `WorkflowExecutor`.
Merge branch mzy/workflow-executor of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/442
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* refactor workflow executor
* .
* fix tests and eval
* .
* .
* revert workflow executor into remote sglang engine
* .
* PullRequest: 456 [lite] [Bug] Use `ProcessPoolExecutor` to calculate reward to avoid rollout slow down
Merge branch mzy/lite/fix-reward of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/456?tab=comment
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix reward
* .
* .
* .
* PullRequest: 460 [lite][fix] add a warning when reward computation timeout
Merge branch fw/lite-fix of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/460
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* add a warning when reward computation timeout
* PullRequest: 465 [lite][fix] Fix issues raised by tsao
Merge branch fw/lite-fix of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/465
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* fix
* PullRequest: 463 [FEAT] add config converter
Merge branch sxj/lite-add-converter of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/463
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* [FEAT] add config converter
* [FIX] fix bugs
* [FIX] fix typos
* [FIX] fix cpu allocation
* [FIX] del vllm
* PullRequest: 466 [lite] Add seeding in training scripts and pad to max_tokens_per_mb to avoid memory fragmentation
Merge branch fw/lite-fix of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/466
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* add seed
* .
* add option for pad to max len
* .
* PullRequest: 467 [lite][fix] remove logging functionality in stats_logger
Merge branch fw/lite-fix of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/467
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* remove logging functionality in stats_logger
* .
* rename arealite to areal
* rename arealite to areal
* .
* local experiment run
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
Co-authored-by: 冰临 <shenxujie.sxj@antgroup.com>
* [lite] [fix] Fix running the legacy experiment in the v0.4.9.post2 sglang environment (#207)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update we…
mjbmjb
pushed a commit
to mjbmjb/AReaL
that referenced
this pull request
Sep 22, 2025
* 0707_1
* 0707_1
* 0707_2
* 0707_2
* 0703_3
* 0703_3
* r
* add api
* add directory structure
* add tests template
* p
* format
* fix
* fix
* checkout previous impl
* checkout previous implementations
* checkout prev impl
* refactor
* add remote sglang engine
* format
* 0707_6
* 0707_7
* add readme
* refactor1
* 0707_undone
* 0708_1
* 0708_2
* 0708_3
* 0708_7
* 0708_4
* 0709_1
* 0709_2
* 0709_3
* PullRequest: 331 [lite] Support remote sglang engine with corresponding testcases.
Merge branch fw/lite of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/331
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* add test for sglang remote engine
* fix
* 0709_4
* 0709_5
* 0709_
* 0709_6
* 0709_7
* PullRequest: 336 add wrapper
Merge branch lite-util-wrapper of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/336
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add wrapper
* 0709_7
* 0709_8
* 0709_9
* PullRequest: 332 [lite] Support FSDP engines
Merge branch mzy/lite/engines of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/332
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* fsdp2 engine
* fix utils
* add fsdp engine test
* .
* fsdp engine test passed
* unsqueeze immediately before model inputs and after model outputts
* add optimizer save/load, add position id calculation for input
* .
* format
* not to squeeze
* add train and eval api
* .
* .
* improve fsdp engine data preprocessing
* format
* PullRequest: 337 [lite] Add SFT trainer example.
* trainer log
* minor changes
* add update weights from disk
* fix type annotation
* PullRequest: 339 [Fix] Fix some minor issues to pass all tests.
Merge branch fw/lite of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/339
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* chore: empty commit
* ci: build images on demand
* ci: fix on demand condition
* ci: fix env sha
* PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods
Merge branch fw/lite-dev of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/340
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* fix
* 0710_1
* 0710_2
* [Fix] Merge previous contributions from fw/refactor to lite (#163)
* initial proposal
* add arealite
* .
* change api
* .
* remove LOG_ROOT
* remove MODEL_SAVE_PATH
* remove PARAM_REALLOC_PATH, DATASET_CACHE
* prepare for testing
* prepare for testing
* ready for run
* local run
* tests mainly pass
* format
* .
* amend cluster.py
* .
* .
* client test pass
* pass rollout test
* remove unused imports
* add arealite readme
* change api
* .
* .
* .
* .
* .
* .
* .
* .
* format
* .
* implement iteraptable generation (#112)
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
* .
* fix
* .
* .
* .
* pass controller generate batch test
* .
* refactor rollout controller into worker and controller
* .
* .
* .
* change to async rollout
* pass rollout controller test
* pass test
* .
* update readme
* .
* sft debug
* .
* add lisence
* remove unused files
* remove unsed args in ppo
* add hf engine wrapper (#116)
* add hf engine
* fix issues
* fix ppo bugs and add test
* add hf client interface and modify cli args
* fix bugs
* fix issues
* Merge fw/refactor
* Finish hf wrapper test
* add test
---------
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
* format
* format
* .
* refine hf engine
* .
* fix
* add fsdp engine and sft tests
* .
* .
* .
* pass ppo unittest
* pass ppo and rollout controller tests
* clear unused imports
* rename ppo to grpo
* change reward function organization
* reorganize code
* add dataset api
* .
* .
* .
* format
* chmod fix
* .
* rename workflow to collector
* refactor llm_client location
* .
* .
* fix llm server api
* refactor config structure
* .
* fix tests
* .
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139)
* .
* .
* efficient loading
* format
* .
* .
* .
* .
* .
* .
* Add CI for testing AReaLite (#150)
* ci: add test-arealite
* ci: add checkout before running test-arealite
* ci: add USERNAME
* ci: add test script
* ci: add GitHub mirror
* ci: fix typo
* ci: clone one commit
* ci: fix condition
* ci: set command timeout to 60m
* ci: enable pip cache
* ci: optimize container lifecycle
* ci: split into many stages
* ci(test-arealite): fix typo
* ci: fix wrong env
* ci: fix pytest
* ci: uninstall transformer-engine
* ci: uninstall transformer-engine
* ci: fix model paths
* ci: show stdout/stderr
* ci: fix not clean up
* ci: backup sglang
* ci: remove tmp repo dir when run
* ci: fix docker run exit 1 condition
* ci(test-arealite): limit the concurrency and extend command timeout
* .
* merge fw/refactor
* revert some changes
* fix
---------
Co-authored-by: meizhiyu.mzy <meizhiyu.mzy@antgroup.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
Co-authored-by: Jayon02 <qiujiangc@outlook.com>
Co-authored-by: root <meizhiyu.mzy>
Co-authored-by: Zijian Zhang <futrime@outlook.com>
* 0710_2
* 0710_3
* [Doc] Add an instruction about how to run the SFT example. (#164)
* 0710_3
* 0710_3
* 0710_5
* 0710_4
* merge_2
* merge_3
* 0711_1
* 0711_2
* 0711_3
* 0711_4
* 0711_6
* 0711_7
* 0711_8
* 0711_8
* 0711_9
* 0711_10
* 0711-11
* [Fix] Fix CI running condition for lite. (#172)
* .
* fix
* .
* .
* fix
* fix
* fix
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* 0714_1
* 0714_2
* 0714_3
* 0714_3
* 0714_5
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* added LocalSGlangEngine and test (#170)
* added LocalSGLangEngine
* upload test file
* add build args
* fix sgl_local generate
* improved sgl local robustness
* test
* test updated
* added fallback when sgl engine isn't initialized
* finish test local engine
* added LocalSGlangEngine and test
* format and fix
format and fix, raise when generate missing field
format
* change cli_args.py
* add comment header
format
---------
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
* format (#174)
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* 0715_1
* 0715_2
* 0715_2
* [Feat][Refactor]Support DeepSpeed AutoTP; Refactor hf_engine.py and unit test. (#161)
* refactor hf engine
* format file
* revert file format
* Squashed commit of the following:
commit a45164235353ea77babfe4d9ccf34b7b8549dd0e
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date: Thu Jul 10 13:14:10 2025 +0800
[Doc] Add an instruction about how to run the SFT example. (#164)
commit 1690c4471f0f8e93fa69665bb72f42ee080f0590
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date: Thu Jul 10 12:56:24 2025 +0800
[Fix] Merge previous contributions from fw/refactor to lite (#163)
* initial proposal
* add arealite
* .
* change api
* .
* remove LOG_ROOT
* remove MODEL_SAVE_PATH
* remove PARAM_REALLOC_PATH, DATASET_CACHE
* prepare for testing
* prepare for testing
* ready for run
* local run
* tests mainly pass
* format
* .
* amend cluster.py
* .
* .
* client test pass
* pass rollout test
* remove unused imports
* add arealite readme
* change api
* .
* .
* .
* .
* .
* .
* .
* .
* format
* .
* implement iteraptable generation (#112)
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
* .
* fix
* .
* .
* .
* pass controller generate batch test
* .
* refactor rollout controller into worker and controller
* .
* .
* .
* change to async rollout
* pass rollout controller test
* pass test
* .
* update readme
* .
* sft debug
* .
* add lisence
* remove unused files
* remove unsed args in ppo
* add hf engine wrapper (#116)
* add hf engine
* fix issues
* fix ppo bugs and add test
* add hf client interface and modify cli args
* fix bugs
* fix issues
* Merge fw/refactor
* Finish hf wrapper test
* add test
---------
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
* format
* format
* .
* refine hf engine
* .
* fix
* add fsdp engine and sft tests
* .
* .
* .
* pass ppo unittest
* pass ppo and rollout controller tests
* clear unused imports
* rename ppo to grpo
* change reward function organization
* reorganize code
* add dataset api
* .
* .
* .
* format
* chmod fix
* .
* rename workflow to collector
* refactor llm_client location
* .
* .
* fix llm server api
* refactor config structure
* .
* fix tests
* .
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139)
* .
* .
* efficient loading
* format
* .
* .
* .
* .
* .
* .
* Add CI for testing AReaLite (#150)
* ci: add test-arealite
* ci: add checkout before running test-arealite
* ci: add USERNAME
* ci: add test script
* ci: add GitHub mirror
* ci: fix typo
* ci: clone one commit
* ci: fix condition
* ci: set command timeout to 60m
* ci: enable pip cache
* ci: optimize container lifecycle
* ci: split into many stages
* ci(test-arealite): fix typo
* ci: fix wrong env
* ci: fix pytest
* ci: uninstall transformer-engine
* ci: uninstall transformer-engine
* ci: fix model paths
* ci: show stdout/stderr
* ci: fix not clean up
* ci: backup sglang
* ci: remove tmp repo dir when run
* ci: fix docker run exit 1 condition
* ci(test-arealite): limit the concurrency and extend command timeout
* .
* merge fw/refactor
* revert some changes
* fix
---------
Co-authored-by: meizhiyu.mzy <meizhiyu.mzy@antgroup.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
Co-authored-by: Jayon02 <qiujiangc@outlook.com>
Co-authored-by: root <meizhiyu.mzy>
Co-authored-by: Zijian Zhang <futrime@outlook.com>
commit d5696a0056eb5622f5cc4c43857d2f423a3efe2c
Merge: 8fc5aa1 743896d
Author: 博惟 <bowei.fw@antgroup.com>
Date: Thu Jul 10 12:53:30 2025 +0800
Merge branch 'main' of https://github.com/inclusionAI/AReaL into lite
commit 8fc5aa11b154ea31fba5a1600cf10dc74ee6f42d
Merge: aa2f819 b588dc3
Author: 博惟 <bowei.fw@antgroup.com>
Date: Thu Jul 10 11:15:01 2025 +0800
Merge branch 'lite' of https://github.com/inclusionAI/AReaL into lite
commit aa2f8196989673556bc25bdf8228c1d8e31834c5
Author: 博惟 <bowei.fw@antgroup.com>
Date: Thu Jul 10 11:10:10 2025 +0800
PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods
Merge branch fw/lite-dev of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/340
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* fix
commit 743896dbd3938fb879dc67cb9990462236778193
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date: Wed Jul 9 10:50:19 2025 +0800
Update to persistent wechat QR code. (#159)
commit 80e16378237dd89878bdf79e5992932550df9493
Author: xssstory <33601810+xssstory@users.noreply.github.com>
Date: Mon Jul 7 15:49:13 2025 +0800
fix math reward verifier (#156)
* PullRequest: 293 fix get_param_realloc_path
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/293
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix get_param_realloc_path
* PullRequest: 297 bugfix: reward is always -5
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/297
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* bugfix: reward is always -5
* PullRequest: 321 fix checkpoint save dir
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/321
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix checkpoint save dir
* PullRequest: 328 [Doc] update installation
Merge branch sxj/doc of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/328
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* [Doc] update installation
* PullRequest: 329 bugfix: math verifier blocks the async training
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/329
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* bugfix: math verifier block the async training
* format
---------
Co-authored-by: 冰临 <shenxujie.sxj@antgroup.com>
Co-authored-by: garrett4wade <fuwth17@gmail.com>
* add autotp for hf
* refactor test
* fix bugs
* fix issues
* format files
* Squashed commit of the following:
commit 93f024b2d173b180a291cc58f73b04bec493a7fb
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date: Tue Jul 15 10:24:48 2025 +0800
format (#174)
commit 4bac0a556eca458c0b2f080453408f242bb79b3c
Author: Night <32424487+PrinsYin@users.noreply.github.com>
Date: Mon Jul 14 19:22:00 2025 -0700
added LocalSGlangEngine and test (#170)
* added LocalSGLangEngine
* upload test file
* add build args
* fix sgl_local generate
* improved sgl local robustness
* test
* test updated
* added fallback when sgl engine isn't initialized
* finish test local engine
* added LocalSGlangEngine and test
* format and fix
format and fix, raise when generate missing field
format
* change cli_args.py
* add comment header
format
---------
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
---------
Co-authored-by: Jayon02 <12012211@mail..sustech.edu.cn>
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
* fix ci (#175)
* [Feature] Add pre-commit (#178)
* fix ci
* .
* .
* .
* [lite] [refactor] Add GSM8k GRPO example. (#179)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* fix ci
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* 0716_1
* 0716_2
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* change doc (#180)
* [Fix] [lite] Merge from the internal repo to fix GRPO bugs and refactor the train engine (#181)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* 0716_3
* 0716_4
* 0716_4
* 0716_5
* 0717_1
* 0717_3
* 0717_3
* 0717_4
* 0717_5
* 0717_6
* 0717_6
* 0717_6
* 0718_2
* 0718_4
* 0718_5
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* merge_0721
* 0721_1
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* 0721_2
* 0721_3
* Implement fsdp distributed update (#183)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* added remote nccl weight update
feat: implement update_weights_from_distributed in fsdp_engine.py
unfinishd test, raise PR first
coroutine for each server
chore: change uploads weights behavior, change test order
fix small bug
fixed test
* fix rebase
* add test.sh
* updated, test stil fails
* .
* .
* .
* fix: full_tensor() should happen in all rank (#187)
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
Co-authored-by: 博惟 <bowei.fw@antgroup.com>
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
Co-authored-by: ChangyiYang <112288487+ChangyiYang@users.noreply.github.com>
* [Feature] [lite] Merge from internal dev repo (#189)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* format
* format
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* [Refactor] Rename files in arealite before release. (#190)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* format
* format
* .
* rename
* rename
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* 0721_4
* .
* 0721_formal
* 0721_formal
* 0721_merge4
* 0721_merge5
* 0721_6
* 0721_merge6
* 0721_merge7
* 0721_8
* 0722_1
* 0722_2
* 0722_3
* add quickstart (#194)
* [doc] [lite] Add customization docs for AReaLite. (#191)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* .
* .
* .
* .
* .
* .
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* 0722_4
* 0722_4
* 0722_5
* 0722_6
* 0722_7
* 0723_1
* reformatted
* clang-reformatted
* clang-reformatted2
* 0723_1
* 0723_1
* 0723_1
* 0723_merge3
* 0723_4
* 0723_reformatted_5
* 0724_1
* 0724_1
* 0724_merge1
* 0724_merge2
* 0724_merge3
* 0724_merge3
* 0724_merge4
* [lite] [feature] Bump to SGLang v0.4.9.post2 and use NCCL to update weights (#196)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* .
* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .
* PullRequest: 423 [lite] Remove the boba example for github release.
Merge branch fw/remove-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* 0724_merge5
* 0724_merge6
* 0724_merge7
* [Docs] [lite] Add example code walkthrough documentation. (#197)
* .
* .
* fix grammar and minor revise
* [lite] [doc] Add AReaLite design doc as README (#198)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* .
* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .
* PullRequest: 423 [lite] Remove the boba example for github release.
Merge branch fw/remove-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* update readme
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* 0724_4
* 0724-merge8
* 0724_merge8
* 0725_1
* 0725_6
* 0725_7
* 0725_4padded_image
* 0725_9padded_image
* 0725_10padded_image
* 0725
* 0725_12
* 0725_format
* Add self-hosted runner support (#199)
* ci: add testsz for self hosted runner
* ci: use self-hosted runner
* ci: run ci on push
* ci: remove cd to tmp dir
* ci: fix
* ci: add environment back
* ci: set concurrency
* ci: fix build env image
* Update .github/workflows/test-arealite.yml
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update ci/build_env_image.sh
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* ci: add some test code
* ci: simplify
* ci: try to fix
* ci: test
* ci: test2
* ci: test3
* ci: test3
* ci: tidy up
* ci: simplify
* ci: do not trigger on push
* ci: test arealite/tests/
* ci: use UUID as run id
* ci: use openssl rand to genreate run id
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* [WIP][feat] Initial support for VLMs, add Qwen2VL SFT test and Qwen2.5VL GRPO test (#188)
* vlm_sft_test
* vlm_sft_test
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139)
* .
* .
* efficient loading
* format
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139)
* .
* .
* efficient loading
* format
* .
* .
* image_process0701
* image_process0701
* image_process0701_2
* image_process0701_2
* image_process0701_3
* image_process0701_3
* .
* .
* .
* .
* .
* .
* imageprocess0702
* imageprocess0702
* image_process0702_2
* image_process0702_2
* image_process0702_3
* image_process0702_3
* image_process0702_4
* image_process0702_4
* image_process0702_5
* image_process0702_5
* image_process0703_1
* image_process0703_1
* 0703_2
* 0703_2
* 0703_3
* 0703_3
* 0703_4
* 0703_4
* 0703_4
* 0703_4
* 0703_5
* 0703_5
* 0703_6
* 0703_6
* 0703_7
* 0703_7
* 0703_8
* 0703_8
* 0703_9
* 0703_9
* 0703_11
* 0703_11
* 0703_12
* 0703_12
* 0703_13
* 0703_13
* 0703_14
* 0703_14
* 0703_15
* 0703_15
* 0703_16
* 0703_16
* 0703-17
* 0703-17
* 0703_18
* 0703_18
* 0703_18
* 0703_18
* 0703_19
* 0703_19
* 0704_1
* 0704_1
* 0704_2
* 0704_2
* 0704_3
* 0704_3
* .
* .
* 0707_1
* 0707_1
* 0707_2
* 0707_2
* 0703_3
* 0703_3
* r
* p
* fix
* fix
* refactor
* 0707_6
* 0707_7
* refactor1
* 0707_undone
* 0708_1
* 0708_2
* 0708_3
* 0708_7
* 0708_4
* 0709_1
* 0709_2
* 0709_3
* 0709_4
* 0709_5
* 0709_
* 0709_6
* 0709_7
* 0709_7
* 0709_8
* 0709_9
* 0710_1
* 0710_2
* 0710_2
* 0710_3
* 0710_3
* 0710_3
* 0710_5
* 0710_4
* merge_2
* merge_3
* 0711_1
* 0711_2
* 0711_3
* 0711_4
* 0711_6
* 0711_7
* 0711_8
* 0711_8
* 0711_9
* 0711_10
* 0711-11
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* 0714_1
* 0714_2
* 0714_3
* 0714_3
* 0714_5
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* 0715_1
* 0715_2
* 0715_2
* 0716_1
* 0716_2
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* 0716_3
* 0716_4
* 0716_4
* 0716_5
* 0717_1
* 0717_3
* 0717_3
* 0717_4
* 0717_5
* 0717_6
* 0717_6
* 0717_6
* 0718_2
* 0718_4
* 0718_5
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* merge_0721
* 0721_1
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* 0721_2
* 0721_3
* 0721_4
* .
* 0721_formal
* 0721_formal
* 0721_merge4
* 0721_merge5
* 0721_6
* 0721_merge6
* 0721_merge7
* 0721_8
* 0722_1
* 0722_2
* 0722_3
* 0722_4
* 0722_4
* 0722_5
* 0722_6
* 0722_7
* 0723_1
* reformatted
* clang-reformatted
* clang-reformatted2
* 0723_1
* 0723_1
* 0723_1
* 0723_merge3
* 0723_4
* 0723_reformatted_5
* 0724_1
* 0724_1
* 0724_merge1
* 0724_merge2
* 0724_merge3
* 0724_merge3
* 0724_merge4
* 0724_merge5
* 0724_merge6
* 0724_merge7
* 0724_4
* 0724-merge8
* 0724_merge8
* 0725_1
* 0725_6
* 0725_7
* 0725_4padded_image
* 0725_9padded_image
* 0725_10padded_image
* 0725
* 0725_12
* 0725_format
---------
Co-authored-by: bowei.fw <bowei.fw@antgroup.com>
Co-authored-by: nuzant <meizhiyu.mzy@antgroup.com>
Co-authored-by: 朱晗 <lichangye.lcy@antgroup.com>
* 0731
* 0731_1
* 0731_2
* 0731_2
* 0731_3
* 0731_4
* 0801_1
* 0801_2
* 0804_1
* 0804_2
* 0804_2
* 0804_5
* 0805_3
* 0805_2
* 0806
* 0806_merge1
* 0806_merge2
* 0806_format1
* 0806_merge3
* 0806_4
* 0806_6
* 0806_7
* 0806_formatted2
* fix
* revert examples
* .
* .
---------
Co-authored-by: 朱晗 <lichangye.lcy@antgroup.com>
Co-authored-by: 博惟 <bowei.fw@antgroup.com>
Co-authored-by: garrett4wade <fuwth17@gmail.com>
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
nv8300
pushed a commit
to nv8300/AReaL
that referenced
this pull request
Oct 13, 2025
* 0707_1
* 0707_1
* 0707_2
* 0707_2
* 0703_3
* 0703_3
* r
* add api
* add directory structure
* add tests template
* p
* format
* fix
* fix
* checkout previous impl
* checkout previous implementations
* checkout prev impl
* refactor
* add remote sglang engine
* format
* 0707_6
* 0707_7
* add readme
* refactor1
* 0707_undone
* 0708_1
* 0708_2
* 0708_3
* 0708_7
* 0708_4
* 0709_1
* 0709_2
* 0709_3
* PullRequest: 331 [lite] Support remote sglang engine with corresponding testcases.
Merge branch fw/lite of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/331
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* add test for sglang remote engine
* fix
* 0709_4
* 0709_5
* 0709_
* 0709_6
* 0709_7
* PullRequest: 336 add wrapper
Merge branch lite-util-wrapper of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/336
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add wrapper
* 0709_7
* 0709_8
* 0709_9
* PullRequest: 332 [lite] Support FSDP engines
Merge branch mzy/lite/engines of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/332
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* fsdp2 engine
* fix utils
* add fsdp engine test
* .
* fsdp engine test passed
* unsqueeze immediately before model inputs and after model outputts
* add optimizer save/load, add position id calculation for input
* .
* format
* not to squeeze
* add train and eval api
* .
* .
* improve fsdp engine data preprocessing
* format
* PullRequest: 337 [lite] Add SFT trainer example.
* trainer log
* minor changes
* add update weights from disk
* fix type annotation
* PullRequest: 339 [Fix] Fix some minor issues to pass all tests.
Merge branch fw/lite of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/339
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* chore: empty commit
* ci: build images on demand
* ci: fix on demand condition
* ci: fix env sha
* PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods
Merge branch fw/lite-dev of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/340
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* fix
* 0710_1
* 0710_2
* [Fix] Merge previous contributions from fw/refactor to lite (#163)
* initial proposal
* add arealite
* .
* change api
* .
* remove LOG_ROOT
* remove MODEL_SAVE_PATH
* remove PARAM_REALLOC_PATH, DATASET_CACHE
* prepare for testing
* prepare for testing
* ready for run
* local run
* tests mainly pass
* format
* .
* amend cluster.py
* .
* .
* client test pass
* pass rollout test
* remove unused imports
* add arealite readme
* change api
* .
* .
* .
* .
* .
* .
* .
* .
* format
* .
* implement iteraptable generation (#112)
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
* .
* fix
* .
* .
* .
* pass controller generate batch test
* .
* refactor rollout controller into worker and controller
* .
* .
* .
* change to async rollout
* pass rollout controller test
* pass test
* .
* update readme
* .
* sft debug
* .
* add lisence
* remove unused files
* remove unsed args in ppo
* add hf engine wrapper (#116)
* add hf engine
* fix issues
* fix ppo bugs and add test
* add hf client interface and modify cli args
* fix bugs
* fix issues
* Merge fw/refactor
* Finish hf wrapper test
* add test
---------
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
* format
* format
* .
* refine hf engine
* .
* fix
* add fsdp engine and sft tests
* .
* .
* .
* pass ppo unittest
* pass ppo and rollout controller tests
* clear unused imports
* rename ppo to grpo
* change reward function organization
* reorganize code
* add dataset api
* .
* .
* .
* format
* chmod fix
* .
* rename workflow to collector
* refactor llm_client location
* .
* .
* fix llm server api
* refactor config structure
* .
* fix tests
* .
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139)
* .
* .
* efficient loading
* format
* .
* .
* .
* .
* .
* .
* Add CI for testing AReaLite (#150)
* ci: add test-arealite
* ci: add checkout before running test-arealite
* ci: add USERNAME
* ci: add test script
* ci: add GitHub mirror
* ci: fix typo
* ci: clone one commit
* ci: fix condition
* ci: set command timeout to 60m
* ci: enable pip cache
* ci: optimize container lifecycle
* ci: split into many stages
* ci(test-arealite): fix typo
* ci: fix wrong env
* ci: fix pytest
* ci: uninstall transformer-engine
* ci: uninstall transformer-engine
* ci: fix model paths
* ci: show stdout/stderr
* ci: fix not clean up
* ci: backup sglang
* ci: remove tmp repo dir when run
* ci: fix docker run exit 1 condition
* ci(test-arealite): limit the concurrency and extend command timeout
* .
* merge fw/refactor
* revert some changes
* fix
---------
Co-authored-by: meizhiyu.mzy <meizhiyu.mzy@antgroup.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
Co-authored-by: Jayon02 <qiujiangc@outlook.com>
Co-authored-by: root <meizhiyu.mzy>
Co-authored-by: Zijian Zhang <futrime@outlook.com>
* 0710_2
* 0710_3
* [Doc] Add an instruction about how to run the SFT example. (#164)
* 0710_3
* 0710_3
* 0710_5
* 0710_4
* merge_2
* merge_3
* 0711_1
* 0711_2
* 0711_3
* 0711_4
* 0711_6
* 0711_7
* 0711_8
* 0711_8
* 0711_9
* 0711_10
* 0711-11
* [Fix] Fix CI running condition for lite. (#172)
* .
* fix
* .
* .
* fix
* fix
* fix
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* 0714_1
* 0714_2
* 0714_3
* 0714_3
* 0714_5
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* added LocalSGlangEngine and test (#170)
* added LocalSGLangEngine
* upload test file
* add build args
* fix sgl_local generate
* improved sgl local robustness
* test
* test updated
* added fallback when sgl engine isn't initialized
* finish test local engine
* added LocalSGlangEngine and test
* format and fix
format and fix, raise when generate missing field
format
* change cli_args.py
* add comment header
format
---------
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
* format (#174)
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* 0715_1
* 0715_2
* 0715_2
* [Feat][Refactor]Support DeepSpeed AutoTP; Refactor hf_engine.py and unit test. (#161)
* refactor hf engine
* format file
* revert file format
* Squashed commit of the following:
commit 8d4b8dc90fceb144bf00f8125f938b57f696e166
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date: Thu Jul 10 13:14:10 2025 +0800
[Doc] Add an instruction about how to run the SFT example. (#164)
commit 3bf9c85e400a2fe1c6a1e488d27d458ede8cea45
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date: Thu Jul 10 12:56:24 2025 +0800
[Fix] Merge previous contributions from fw/refactor to lite (#163)
* initial proposal
* add arealite
* .
* change api
* .
* remove LOG_ROOT
* remove MODEL_SAVE_PATH
* remove PARAM_REALLOC_PATH, DATASET_CACHE
* prepare for testing
* prepare for testing
* ready for run
* local run
* tests mainly pass
* format
* .
* amend cluster.py
* .
* .
* client test pass
* pass rollout test
* remove unused imports
* add arealite readme
* change api
* .
* .
* .
* .
* .
* .
* .
* .
* format
* .
* implement iteraptable generation (#112)
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
* .
* fix
* .
* .
* .
* pass controller generate batch test
* .
* refactor rollout controller into worker and controller
* .
* .
* .
* change to async rollout
* pass rollout controller test
* pass test
* .
* update readme
* .
* sft debug
* .
* add lisence
* remove unused files
* remove unsed args in ppo
* add hf engine wrapper (#116)
* add hf engine
* fix issues
* fix ppo bugs and add test
* add hf client interface and modify cli args
* fix bugs
* fix issues
* Merge fw/refactor
* Finish hf wrapper test
* add test
---------
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
* format
* format
* .
* refine hf engine
* .
* fix
* add fsdp engine and sft tests
* .
* .
* .
* pass ppo unittest
* pass ppo and rollout controller tests
* clear unused imports
* rename ppo to grpo
* change reward function organization
* reorganize code
* add dataset api
* .
* .
* .
* format
* chmod fix
* .
* rename workflow to collector
* refactor llm_client location
* .
* .
* fix llm server api
* refactor config structure
* .
* fix tests
* .
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139)
* .
* .
* efficient loading
* format
* .
* .
* .
* .
* .
* .
* Add CI for testing AReaLite (#150)
* ci: add test-arealite
* ci: add checkout before running test-arealite
* ci: add USERNAME
* ci: add test script
* ci: add GitHub mirror
* ci: fix typo
* ci: clone one commit
* ci: fix condition
* ci: set command timeout to 60m
* ci: enable pip cache
* ci: optimize container lifecycle
* ci: split into many stages
* ci(test-arealite): fix typo
* ci: fix wrong env
* ci: fix pytest
* ci: uninstall transformer-engine
* ci: uninstall transformer-engine
* ci: fix model paths
* ci: show stdout/stderr
* ci: fix not clean up
* ci: backup sglang
* ci: remove tmp repo dir when run
* ci: fix docker run exit 1 condition
* ci(test-arealite): limit the concurrency and extend command timeout
* .
* merge fw/refactor
* revert some changes
* fix
---------
Co-authored-by: meizhiyu.mzy <meizhiyu.mzy@antgroup.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
Co-authored-by: Jayon02 <qiujiangc@outlook.com>
Co-authored-by: root <meizhiyu.mzy>
Co-authored-by: Zijian Zhang <futrime@outlook.com>
commit d48bf007cfb0821c35a18a673f3c2e8f51a8ee45
Merge: 42c717b 743896d
Author: 博惟 <bowei.fw@antgroup.com>
Date: Thu Jul 10 12:53:30 2025 +0800
Merge branch 'main' of https://github.com/inclusionAI/AReaL into lite
commit 42c717b6e45b3b345f29cff5882ba5374b79faee
Merge: c38cffc a203c7c
Author: 博惟 <bowei.fw@antgroup.com>
Date: Thu Jul 10 11:15:01 2025 +0800
Merge branch 'lite' of https://github.com/inclusionAI/AReaL into lite
commit c38cffc02385acf382196ba6b720b2d8b91a5675
Author: 博惟 <bowei.fw@antgroup.com>
Date: Thu Jul 10 11:10:10 2025 +0800
PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods
Merge branch fw/lite-dev of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/340
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* fix
commit 743896dbd3938fb879dc67cb9990462236778193
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date: Wed Jul 9 10:50:19 2025 +0800
Update to persistent wechat QR code. (#159)
commit 80e16378237dd89878bdf79e5992932550df9493
Author: xssstory <33601810+xssstory@users.noreply.github.com>
Date: Mon Jul 7 15:49:13 2025 +0800
fix math reward verifier (#156)
* PullRequest: 293 fix get_param_realloc_path
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/293
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix get_param_realloc_path
* PullRequest: 297 bugfix: reward is always -5
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/297
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* bugfix: reward is always -5
* PullRequest: 321 fix checkpoint save dir
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/321
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix checkpoint save dir
* PullRequest: 328 [Doc] update installation
Merge branch sxj/doc of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/328
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* [Doc] update installation
* PullRequest: 329 bugfix: math verifier blocks the async training
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/329
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* bugfix: math verifier block the async training
* format
---------
Co-authored-by: 冰临 <shenxujie.sxj@antgroup.com>
Co-authored-by: garrett4wade <fuwth17@gmail.com>
* add autotp for hf
* refactor test
* fix bugs
* fix issues
* format files
* Squashed commit of the following:
commit 9ed043f6ab83cf9c9b38e0251ad08009a73243e8
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date: Tue Jul 15 10:24:48 2025 +0800
format (#174)
commit 8cc9b1feb5ed03cfa42f3ed588dd5f6ec16e5e55
Author: Night <32424487+PrinsYin@users.noreply.github.com>
Date: Mon Jul 14 19:22:00 2025 -0700
added LocalSGlangEngine and test (#170)
* added LocalSGLangEngine
* upload test file
* add build args
* fix sgl_local generate
* improved sgl local robustness
* test
* test updated
* added fallback when sgl engine isn't initialized
* finish test local engine
* added LocalSGlangEngine and test
* format and fix
format and fix, raise when generate missing field
format
* change cli_args.py
* add comment header
format
---------
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
---------
Co-authored-by: Jayon02 <12012211@mail..sustech.edu.cn>
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
* fix ci (#175)
* [Feature] Add pre-commit (#178)
* fix ci
* .
* .
* .
* [lite] [refactor] Add GSM8k GRPO example. (#179)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* fix ci
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* 0716_1
* 0716_2
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* change doc (#180)
* [Fix] [lite] Merge from the internal repo to fix GRPO bugs and refactor the train engine (#181)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* 0716_3
* 0716_4
* 0716_4
* 0716_5
* 0717_1
* 0717_3
* 0717_3
* 0717_4
* 0717_5
* 0717_6
* 0717_6
* 0717_6
* 0718_2
* 0718_4
* 0718_5
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* merge_0721
* 0721_1
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* 0721_2
* 0721_3
* Implement fsdp distributed update (#183)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* added remote nccl weight update
feat: implement update_weights_from_distributed in fsdp_engine.py
unfinishd test, raise PR first
coroutine for each server
chore: change uploads weights behavior, change test order
fix small bug
fixed test
* fix rebase
* add test.sh
* updated, test stil fails
* .
* .
* .
* fix: full_tensor() should happen in all rank (#187)
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
Co-authored-by: 博惟 <bowei.fw@antgroup.com>
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
Co-authored-by: ChangyiYang <112288487+ChangyiYang@users.noreply.github.com>
* [Feature] [lite] Merge from internal dev repo (#189)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* format
* format
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* [Refactor] Rename files in arealite before release. (#190)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* format
* format
* .
* rename
* rename
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* 0721_4
* .
* 0721_formal
* 0721_formal
* 0721_merge4
* 0721_merge5
* 0721_6
* 0721_merge6
* 0721_merge7
* 0721_8
* 0722_1
* 0722_2
* 0722_3
* add quickstart (#194)
* [doc] [lite] Add customization docs for AReaLite. (#191)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* .
* .
* .
* .
* .
* .
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* 0722_4
* 0722_4
* 0722_5
* 0722_6
* 0722_7
* 0723_1
* reformatted
* clang-reformatted
* clang-reformatted2
* 0723_1
* 0723_1
* 0723_1
* 0723_merge3
* 0723_4
* 0723_reformatted_5
* 0724_1
* 0724_1
* 0724_merge1
* 0724_merge2
* 0724_merge3
* 0724_merge3
* 0724_merge4
* [lite] [feature] Bump to SGLang v0.4.9.post2 and use NCCL to update weights (#196)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* .
* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .
* PullRequest: 423 [lite] Remove the boba example for github release.
Merge branch fw/remove-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* 0724_merge5
* 0724_merge6
* 0724_merge7
* [Docs] [lite] Add example code walkthrough documentation. (#197)
* .
* .
* fix grammar and minor revise
* [lite] [doc] Add AReaLite design doc as README (#198)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* .
* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .
* PullRequest: 423 [lite] Remove the boba example for github release.
Merge branch fw/remove-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* update readme
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* 0724_4
* 0724-merge8
* 0724_merge8
* 0725_1
* 0725_6
* 0725_7
* 0725_4padded_image
* 0725_9padded_image
* 0725_10padded_image
* 0725
* 0725_12
* 0725_format
* Add self-hosted runner support (#199)
* ci: add testsz for self hosted runner
* ci: use self-hosted runner
* ci: run ci on push
* ci: remove cd to tmp dir
* ci: fix
* ci: add environment back
* ci: set concurrency
* ci: fix build env image
* Update .github/workflows/test-arealite.yml
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update ci/build_env_image.sh
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* ci: add some test code
* ci: simplify
* ci: try to fix
* ci: test
* ci: test2
* ci: test3
* ci: test3
* ci: tidy up
* ci: simplify
* ci: do not trigger on push
* ci: test arealite/tests/
* ci: use UUID as run id
* ci: use openssl rand to genreate run id
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* [WIP][feat] Initial support for VLMs, add Qwen2VL SFT test and Qwen2.5VL GRPO test (#188)
* vlm_sft_test
* vlm_sft_test
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139)
* .
* .
* efficient loading
* format
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139)
* .
* .
* efficient loading
* format
* .
* .
* image_process0701
* image_process0701
* image_process0701_2
* image_process0701_2
* image_process0701_3
* image_process0701_3
* .
* .
* .
* .
* .
* .
* imageprocess0702
* imageprocess0702
* image_process0702_2
* image_process0702_2
* image_process0702_3
* image_process0702_3
* image_process0702_4
* image_process0702_4
* image_process0702_5
* image_process0702_5
* image_process0703_1
* image_process0703_1
* 0703_2
* 0703_2
* 0703_3
* 0703_3
* 0703_4
* 0703_4
* 0703_4
* 0703_4
* 0703_5
* 0703_5
* 0703_6
* 0703_6
* 0703_7
* 0703_7
* 0703_8
* 0703_8
* 0703_9
* 0703_9
* 0703_11
* 0703_11
* 0703_12
* 0703_12
* 0703_13
* 0703_13
* 0703_14
* 0703_14
* 0703_15
* 0703_15
* 0703_16
* 0703_16
* 0703-17
* 0703-17
* 0703_18
* 0703_18
* 0703_18
* 0703_18
* 0703_19
* 0703_19
* 0704_1
* 0704_1
* 0704_2
* 0704_2
* 0704_3
* 0704_3
* .
* .
* 0707_1
* 0707_1
* 0707_2
* 0707_2
* 0703_3
* 0703_3
* r
* p
* fix
* fix
* refactor
* 0707_6
* 0707_7
* refactor1
* 0707_undone
* 0708_1
* 0708_2
* 0708_3
* 0708_7
* 0708_4
* 0709_1
* 0709_2
* 0709_3
* 0709_4
* 0709_5
* 0709_
* 0709_6
* 0709_7
* 0709_7
* 0709_8
* 0709_9
* 0710_1
* 0710_2
* 0710_2
* 0710_3
* 0710_3
* 0710_3
* 0710_5
* 0710_4
* merge_2
* merge_3
* 0711_1
* 0711_2
* 0711_3
* 0711_4
* 0711_6
* 0711_7
* 0711_8
* 0711_8
* 0711_9
* 0711_10
* 0711-11
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* 0714_1
* 0714_2
* 0714_3
* 0714_3
* 0714_5
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* 0715_1
* 0715_2
* 0715_2
* 0716_1
* 0716_2
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* 0716_3
* 0716_4
* 0716_4
* 0716_5
* 0717_1
* 0717_3
* 0717_3
* 0717_4
* 0717_5
* 0717_6
* 0717_6
* 0717_6
* 0718_2
* 0718_4
* 0718_5
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* merge_0721
* 0721_1
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* 0721_2
* 0721_3
* 0721_4
* .
* 0721_formal
* 0721_formal
* 0721_merge4
* 0721_merge5
* 0721_6
* 0721_merge6
* 0721_merge7
* 0721_8
* 0722_1
* 0722_2
* 0722_3
* 0722_4
* 0722_4
* 0722_5
* 0722_6
* 0722_7
* 0723_1
* reformatted
* clang-reformatted
* clang-reformatted2
* 0723_1
* 0723_1
* 0723_1
* 0723_merge3
* 0723_4
* 0723_reformatted_5
* 0724_1
* 0724_1
* 0724_merge1
* 0724_merge2
* 0724_merge3
* 0724_merge3
* 0724_merge4
* 0724_merge5
* 0724_merge6
* 0724_merge7
* 0724_4
* 0724-merge8
* 0724_merge8
* 0725_1
* 0725_6
* 0725_7
* 0725_4padded_image
* 0725_9padded_image
* 0725_10padded_image
* 0725
* 0725_12
* 0725_format
---------
Co-authored-by: bowei.fw <bowei.fw@antgroup.com>
Co-authored-by: nuzant <meizhiyu.mzy@antgroup.com>
Co-authored-by: 朱晗 <lichangye.lcy@antgroup.com>
* 0731
* 0731_1
* 0731_2
* 0731_2
* 0731_3
* 0731_4
* 0801_1
* 0801_2
* 0804_1
* 0804_2
* 0804_2
* 0804_5
* 0805_3
* 0805_2
* 0806
* 0806_merge1
* 0806_merge2
* 0806_format1
* 0806_merge3
* 0806_4
* 0806_6
* 0806_7
* 0806_formatted2
* fix
* revert examples
* .
* .
---------
Co-authored-by: 朱晗 <lichangye.lcy@antgroup.com>
Co-authored-by: 博惟 <bowei.fw@antgroup.com>
Co-authored-by: garrett4wade <fuwth17@gmail.com>
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
SathyaGnanakumar
pushed a commit
to danielkiely/AReaL
that referenced
this pull request
Apr 29, 2026
* PullRequest: 293 fix get_param_realloc_path
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/293
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix get_param_realloc_path
* PullRequest: 297 bugfix: reward is always -5
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/297
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* bugfix: reward is always -5
* PullRequest: 321 fix checkpoint save dir
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/321
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix checkpoint save dir
* add api
* add directory structure
* add tests template
* format
* checkout previous impl
* checkout previous implementations
* checkout prev impl
* add remote sglang engine
* format
* add readme
* PullRequest: 331 [lite] Support remote sglang engine with corresponding testcases.
Merge branch fw/lite of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/331
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* add test for sglang remote engine
* fix
* PullRequest: 336 add wrapper
Merge branch lite-util-wrapper of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/336
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add wrapper
* PullRequest: 332 [lite] Support FSDP engines
Merge branch mzy/lite/engines of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/332
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* fsdp2 engine
* fix utils
* add fsdp engine test
* .
* fsdp engine test passed
* unsqueeze immediately before model inputs and after model outputts
* add optimizer save/load, add position id calculation for input
* .
* format
* not to squeeze
* add train and eval api
* .
* .
* improve fsdp engine data preprocessing
* format
* PullRequest: 337 [lite] Add SFT trainer example.
* trainer log
* minor changes
* add update weights from disk
* fix type annotation
* PullRequest: 339 [Fix] Fix some minor issues to pass all tests.
Merge branch fw/lite of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/339
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* chore: empty commit
* ci: build images on demand
* ci: fix on demand condition
* ci: fix env sha
* PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods
Merge branch fw/lite-dev of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/340
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* fix
* [Fix] Merge previous contributions from fw/refactor to lite (#163)
* initial proposal
* add arealite
* .
* change api
* .
* remove LOG_ROOT
* remove MODEL_SAVE_PATH
* remove PARAM_REALLOC_PATH, DATASET_CACHE
* prepare for testing
* prepare for testing
* ready for run
* local run
* tests mainly pass
* format
* .
* amend cluster.py
* .
* .
* client test pass
* pass rollout test
* remove unused imports
* add arealite readme
* change api
* .
* .
* .
* .
* .
* .
* .
* .
* format
* .
* implement iteraptable generation (#112)
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
* .
* fix
* .
* .
* .
* pass controller generate batch test
* .
* refactor rollout controller into worker and controller
* .
* .
* .
* change to async rollout
* pass rollout controller test
* pass test
* .
* update readme
* .
* sft debug
* .
* add lisence
* remove unused files
* remove unsed args in ppo
* add hf engine wrapper (#116)
* add hf engine
* fix issues
* fix ppo bugs and add test
* add hf client interface and modify cli args
* fix bugs
* fix issues
* Merge fw/refactor
* Finish hf wrapper test
* add test
---------
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
* format
* format
* .
* refine hf engine
* .
* fix
* add fsdp engine and sft tests
* .
* .
* .
* pass ppo unittest
* pass ppo and rollout controller tests
* clear unused imports
* rename ppo to grpo
* change reward function organization
* reorganize code
* add dataset api
* .
* .
* .
* format
* chmod fix
* .
* rename workflow to collector
* refactor llm_client location
* .
* .
* fix llm server api
* refactor config structure
* .
* fix tests
* .
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139)
* .
* .
* efficient loading
* format
* .
* .
* .
* .
* .
* .
* Add CI for testing AReaLite (#150)
* ci: add test-arealite
* ci: add checkout before running test-arealite
* ci: add USERNAME
* ci: add test script
* ci: add GitHub mirror
* ci: fix typo
* ci: clone one commit
* ci: fix condition
* ci: set command timeout to 60m
* ci: enable pip cache
* ci: optimize container lifecycle
* ci: split into many stages
* ci(test-arealite): fix typo
* ci: fix wrong env
* ci: fix pytest
* ci: uninstall transformer-engine
* ci: uninstall transformer-engine
* ci: fix model paths
* ci: show stdout/stderr
* ci: fix not clean up
* ci: backup sglang
* ci: remove tmp repo dir when run
* ci: fix docker run exit 1 condition
* ci(test-arealite): limit the concurrency and extend command timeout
* .
* merge fw/refactor
* revert some changes
* fix
---------
Co-authored-by: meizhiyu.mzy <meizhiyu.mzy@antgroup.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
Co-authored-by: Jayon02 <qiujiangc@outlook.com>
Co-authored-by: root <meizhiyu.mzy>
Co-authored-by: Zijian Zhang <futrime@outlook.com>
* [Doc] Add an instruction about how to run the SFT example. (#164)
* [Fix] Fix CI running condition for lite. (#172)
* .
* fix
* .
* .
* fix
* fix
* fix
* added LocalSGlangEngine and test (#170)
* added LocalSGLangEngine
* upload test file
* add build args
* fix sgl_local generate
* improved sgl local robustness
* test
* test updated
* added fallback when sgl engine isn't initialized
* finish test local engine
* added LocalSGlangEngine and test
* format and fix
format and fix, raise when generate missing field
format
* change cli_args.py
* add comment header
format
---------
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
* format (#174)
* [Feat][Refactor]Support DeepSpeed AutoTP; Refactor hf_engine.py and unit test. (#161)
* refactor hf engine
* format file
* revert file format
* Squashed commit of the following:
commit 8d4b8dc90fceb144bf00f8125f938b57f696e166
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date: Thu Jul 10 13:14:10 2025 +0800
[Doc] Add an instruction about how to run the SFT example. (#164)
commit 3bf9c85e400a2fe1c6a1e488d27d458ede8cea45
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date: Thu Jul 10 12:56:24 2025 +0800
[Fix] Merge previous contributions from fw/refactor to lite (#163)
* initial proposal
* add arealite
* .
* change api
* .
* remove LOG_ROOT
* remove MODEL_SAVE_PATH
* remove PARAM_REALLOC_PATH, DATASET_CACHE
* prepare for testing
* prepare for testing
* ready for run
* local run
* tests mainly pass
* format
* .
* amend cluster.py
* .
* .
* client test pass
* pass rollout test
* remove unused imports
* add arealite readme
* change api
* .
* .
* .
* .
* .
* .
* .
* .
* format
* .
* implement iteraptable generation (#112)
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
* .
* fix
* .
* .
* .
* pass controller generate batch test
* .
* refactor rollout controller into worker and controller
* .
* .
* .
* change to async rollout
* pass rollout controller test
* pass test
* .
* update readme
* .
* sft debug
* .
* add lisence
* remove unused files
* remove unsed args in ppo
* add hf engine wrapper (#116)
* add hf engine
* fix issues
* fix ppo bugs and add test
* add hf client interface and modify cli args
* fix bugs
* fix issues
* Merge fw/refactor
* Finish hf wrapper test
* add test
---------
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
* format
* format
* .
* refine hf engine
* .
* fix
* add fsdp engine and sft tests
* .
* .
* .
* pass ppo unittest
* pass ppo and rollout controller tests
* clear unused imports
* rename ppo to grpo
* change reward function organization
* reorganize code
* add dataset api
* .
* .
* .
* format
* chmod fix
* .
* rename workflow to collector
* refactor llm_client location
* .
* .
* fix llm server api
* refactor config structure
* .
* fix tests
* .
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139)
* .
* .
* efficient loading
* format
* .
* .
* .
* .
* .
* .
* Add CI for testing AReaLite (#150)
* ci: add test-arealite
* ci: add checkout before running test-arealite
* ci: add USERNAME
* ci: add test script
* ci: add GitHub mirror
* ci: fix typo
* ci: clone one commit
* ci: fix condition
* ci: set command timeout to 60m
* ci: enable pip cache
* ci: optimize container lifecycle
* ci: split into many stages
* ci(test-arealite): fix typo
* ci: fix wrong env
* ci: fix pytest
* ci: uninstall transformer-engine
* ci: uninstall transformer-engine
* ci: fix model paths
* ci: show stdout/stderr
* ci: fix not clean up
* ci: backup sglang
* ci: remove tmp repo dir when run
* ci: fix docker run exit 1 condition
* ci(test-arealite): limit the concurrency and extend command timeout
* .
* merge fw/refactor
* revert some changes
* fix
---------
Co-authored-by: meizhiyu.mzy <meizhiyu.mzy@antgroup.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
Co-authored-by: Jayon02 <qiujiangc@outlook.com>
Co-authored-by: root <meizhiyu.mzy>
Co-authored-by: Zijian Zhang <futrime@outlook.com>
commit d48bf007cfb0821c35a18a673f3c2e8f51a8ee45
Merge: 42c717b b9dbd4a
Author: 博惟 <bowei.fw@antgroup.com>
Date: Thu Jul 10 12:53:30 2025 +0800
Merge branch 'main' of https://github.com/inclusionAI/AReaL into lite
commit 42c717b6e45b3b345f29cff5882ba5374b79faee
Merge: c38cffc a203c7c
Author: 博惟 <bowei.fw@antgroup.com>
Date: Thu Jul 10 11:15:01 2025 +0800
Merge branch 'lite' of https://github.com/inclusionAI/AReaL into lite
commit c38cffc02385acf382196ba6b720b2d8b91a5675
Author: 博惟 <bowei.fw@antgroup.com>
Date: Thu Jul 10 11:10:10 2025 +0800
PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods
Merge branch fw/lite-dev of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/340
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* fix
commit b9dbd4a2c18539648dca373c9c8c1347e4e9db68
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date: Wed Jul 9 10:50:19 2025 +0800
Update to persistent wechat QR code. (#159)
commit 17ea7fe94dc0bd9299082a069383d7372ed13ac9
Author: xssstory <33601810+xssstory@users.noreply.github.com>
Date: Mon Jul 7 15:49:13 2025 +0800
fix math reward verifier (#156)
* PullRequest: 293 fix get_param_realloc_path
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/293
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix get_param_realloc_path
* PullRequest: 297 bugfix: reward is always -5
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/297
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* bugfix: reward is always -5
* PullRequest: 321 fix checkpoint save dir
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/321
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix checkpoint save dir
* PullRequest: 328 [Doc] update installation
Merge branch sxj/doc of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/328
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* [Doc] update installation
* PullRequest: 329 bugfix: math verifier blocks the async training
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/329
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* bugfix: math verifier block the async training
* format
---------
Co-authored-by: 冰临 <shenxujie.sxj@antgroup.com>
Co-authored-by: garrett4wade <fuwth17@gmail.com>
* add autotp for hf
* refactor test
* fix bugs
* fix issues
* format files
* Squashed commit of the following:
commit 9ed043f6ab83cf9c9b38e0251ad08009a73243e8
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date: Tue Jul 15 10:24:48 2025 +0800
format (#174)
commit 8cc9b1feb5ed03cfa42f3ed588dd5f6ec16e5e55
Author: Night <32424487+PrinsYin@users.noreply.github.com>
Date: Mon Jul 14 19:22:00 2025 -0700
added LocalSGlangEngine and test (#170)
* added LocalSGLangEngine
* upload test file
* add build args
* fix sgl_local generate
* improved sgl local robustness
* test
* test updated
* added fallback when sgl engine isn't initialized
* finish test local engine
* added LocalSGlangEngine and test
* format and fix
format and fix, raise when generate missing field
format
* change cli_args.py
* add comment header
format
---------
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
---------
Co-authored-by: Jayon02 <12012211@mail..sustech.edu.cn>
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
* fix ci (#175)
* [Feature] Add pre-commit (#178)
* fix ci
* .
* .
* .
* [lite] [refactor] Add GSM8k GRPO example. (#179)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* fix ci
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* change doc (#180)
* [Fix] [lite] Merge from the internal repo to fix GRPO bugs and refactor the train engine (#181)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* Implement fsdp distributed update (#183)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* added remote nccl weight update
feat: implement update_weights_from_distributed in fsdp_engine.py
unfinishd test, raise PR first
coroutine for each server
chore: change uploads weights behavior, change test order
fix small bug
fixed test
* fix rebase
* add test.sh
* updated, test stil fails
* .
* .
* .
* fix: full_tensor() should happen in all rank (#187)
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
Co-authored-by: 博惟 <bowei.fw@antgroup.com>
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
Co-authored-by: ChangyiYang <112288487+ChangyiYang@users.noreply.github.com>
* [Feature] [lite] Merge from internal dev repo (#189)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* format
* format
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* [Refactor] Rename files in arealite before release. (#190)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* format
* format
* .
* rename
* rename
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* add quickstart (#194)
* [doc] [lite] Add customization docs for AReaLite. (#191)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* .
* .
* .
* .
* .
* .
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* [lite] [feature] Bump to SGLang v0.4.9.post2 and use NCCL to update weights (#196)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* .
* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .
* PullRequest: 423 [lite] Remove the boba example for github release.
Merge branch fw/remove-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* [Docs] [lite] Add example code walkthrough documentation. (#197)
* .
* .
* fix grammar and minor revise
* [lite] [doc] Add AReaLite design doc as README (#198)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* .
* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .
* PullRequest: 423 [lite] Remove the boba example for github release.
Merge branch fw/remove-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* update readme
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* Add self-hosted runner support (#199)
* ci: add testsz for self hosted runner
* ci: use self-hosted runner
* ci: run ci on push
* ci: remove cd to tmp dir
* ci: fix
* ci: add environment back
* ci: set concurrency
* ci: fix build env image
* Update .github/workflows/test-arealite.yml
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update ci/build_env_image.sh
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* ci: add some test code
* ci: simplify
* ci: try to fix
* ci: test
* ci: test2
* ci: test3
* ci: test3
* ci: tidy up
* ci: simplify
* ci: do not trigger on push
* ci: test arealite/tests/
* ci: use UUID as run id
* ci: use openssl rand to genreate run id
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* [WIP][feat] Initial support for VLMs, add Qwen2VL SFT test and Qwen2.5VL GRPO test (#188)
* vlm_sft_test
* vlm_sft_test
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139)
* .
* .
* efficient loading
* format
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139)
* .
* .
* efficient loading
* format
* .
* .
* image_process0701
* image_process0701
* image_process0701_2
* image_process0701_2
* image_process0701_3
* image_process0701_3
* .
* .
* .
* .
* .
* .
* imageprocess0702
* imageprocess0702
* image_process0702_2
* image_process0702_2
* image_process0702_3
* image_process0702_3
* image_process0702_4
* image_process0702_4
* image_process0702_5
* image_process0702_5
* image_process0703_1
* image_process0703_1
* 0703_2
* 0703_2
* 0703_3
* 0703_3
* 0703_4
* 0703_4
* 0703_4
* 0703_4
* 0703_5
* 0703_5
* 0703_6
* 0703_6
* 0703_7
* 0703_7
* 0703_8
* 0703_8
* 0703_9
* 0703_9
* 0703_11
* 0703_11
* 0703_12
* 0703_12
* 0703_13
* 0703_13
* 0703_14
* 0703_14
* 0703_15
* 0703_15
* 0703_16
* 0703_16
* 0703-17
* 0703-17
* 0703_18
* 0703_18
* 0703_18
* 0703_18
* 0703_19
* 0703_19
* 0704_1
* 0704_1
* 0704_2
* 0704_2
* 0704_3
* 0704_3
* .
* .
* 0707_1
* 0707_1
* 0707_2
* 0707_2
* 0703_3
* 0703_3
* r
* p
* fix
* fix
* refactor
* 0707_6
* 0707_7
* refactor1
* 0707_undone
* 0708_1
* 0708_2
* 0708_3
* 0708_7
* 0708_4
* 0709_1
* 0709_2
* 0709_3
* 0709_4
* 0709_5
* 0709_
* 0709_6
* 0709_7
* 0709_7
* 0709_8
* 0709_9
* 0710_1
* 0710_2
* 0710_2
* 0710_3
* 0710_3
* 0710_3
* 0710_5
* 0710_4
* merge_2
* merge_3
* 0711_1
* 0711_2
* 0711_3
* 0711_4
* 0711_6
* 0711_7
* 0711_8
* 0711_8
* 0711_9
* 0711_10
* 0711-11
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* 0714_1
* 0714_2
* 0714_3
* 0714_3
* 0714_5
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* 0715_1
* 0715_2
* 0715_2
* 0716_1
* 0716_2
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* 0716_3
* 0716_4
* 0716_4
* 0716_5
* 0717_1
* 0717_3
* 0717_3
* 0717_4
* 0717_5
* 0717_6
* 0717_6
* 0717_6
* 0718_2
* 0718_4
* 0718_5
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* merge_0721
* 0721_1
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* 0721_2
* 0721_3
* 0721_4
* .
* 0721_formal
* 0721_formal
* 0721_merge4
* 0721_merge5
* 0721_6
* 0721_merge6
* 0721_merge7
* 0721_8
* 0722_1
* 0722_2
* 0722_3
* 0722_4
* 0722_4
* 0722_5
* 0722_6
* 0722_7
* 0723_1
* reformatted
* clang-reformatted
* clang-reformatted2
* 0723_1
* 0723_1
* 0723_1
* 0723_merge3
* 0723_4
* 0723_reformatted_5
* 0724_1
* 0724_1
* 0724_merge1
* 0724_merge2
* 0724_merge3
* 0724_merge3
* 0724_merge4
* 0724_merge5
* 0724_merge6
* 0724_merge7
* 0724_4
* 0724-merge8
* 0724_merge8
* 0725_1
* 0725_6
* 0725_7
* 0725_4padded_image
* 0725_9padded_image
* 0725_10padded_image
* 0725
* 0725_12
* 0725_format
---------
Co-authored-by: bowei.fw <bowei.fw@antgroup.com>
Co-authored-by: nuzant <meizhiyu.mzy@antgroup.com>
Co-authored-by: 朱晗 <lichangye.lcy@antgroup.com>
* [lite] [fix] Fix a performance issue and several minor issues before release (#203)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* .
* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .
* PullRequest: 423 [lite] Remove the boba example for github release.
Merge branch fw/remove-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* update readme
* PullRequest: 431 [Fix] Fix environment of lite
Merge branch fw/lite-dev of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/431
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* change requirements
* .
* .
* .
* PullRequest: 440 [FIX] fix update weight from disk
Merge branch sxj/lite-fix-disk-update of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/440
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* [FIX] fix update weight from disk
* PullRequest: 442 [lite] Refactor `RemoteSGLangEngine` into two parts: `RemoteSGLangEngine` and `WorkflowExecutor`.
Merge branch mzy/workflow-executor of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/442
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* refactor workflow executor
* .
* fix tests and eval
* .
* .
* revert workflow executor into remote sglang engine
* .
* PullRequest: 456 [lite] [Bug] Use `ProcessPoolExecutor` to calculate reward to avoid rollout slow down
Merge branch mzy/lite/fix-reward of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/456?tab=comment
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix reward
* .
* .
* .
* PullRequest: 460 [lite][fix] add a warning when reward computation timeout
Merge branch fw/lite-fix of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/460
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* add a warning when reward computation timeout
* PullRequest: 465 [lite][fix] Fix issues raised by tsao
Merge branch fw/lite-fix of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/465
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* fix
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
Co-authored-by: 冰临 <shenxujie.sxj@antgroup.com>
* [lite] [refactor] Rename the `arealite` folder to `areal` (#205)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* .
* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .
* PullRequest: 423 [lite] Remove the boba example for github release.
Merge branch fw/remove-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* update readme
* PullRequest: 431 [Fix] Fix environment of lite
Merge branch fw/lite-dev of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/431
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* change requirements
* .
* .
* .
* PullRequest: 440 [FIX] fix update weight from disk
Merge branch sxj/lite-fix-disk-update of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/440
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* [FIX] fix update weight from disk
* PullRequest: 442 [lite] Refactor `RemoteSGLangEngine` into two parts: `RemoteSGLangEngine` and `WorkflowExecutor`.
Merge branch mzy/workflow-executor of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/442
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* refactor workflow executor
* .
* fix tests and eval
* .
* .
* revert workflow executor into remote sglang engine
* .
* PullRequest: 456 [lite] [Bug] Use `ProcessPoolExecutor` to calculate reward to avoid rollout slow down
Merge branch mzy/lite/fix-reward of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/456?tab=comment
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix reward
* .
* .
* .
* PullRequest: 460 [lite][fix] add a warning when reward computation timeout
Merge branch fw/lite-fix of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/460
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* add a warning when reward computation timeout
* PullRequest: 465 [lite][fix] Fix issues raised by tsao
Merge branch fw/lite-fix of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/465
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* fix
* PullRequest: 463 [FEAT] add config converter
Merge branch sxj/lite-add-converter of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/463
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* [FEAT] add config converter
* [FIX] fix bugs
* [FIX] fix typos
* [FIX] fix cpu allocation
* [FIX] del vllm
* PullRequest: 466 [lite] Add seeding in training scripts and pad to max_tokens_per_mb to avoid memory fragmentation
Merge branch fw/lite-fix of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/466
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* add seed
* .
* add option for pad to max len
* .
* PullRequest: 467 [lite][fix] remove logging functionality in stats_logger
Merge branch fw/lite-fix of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/467
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* remove logging functionality in stats_logger
* .
* rename arealite to areal
* rename arealite to areal
* .
* local experiment run
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
Co-authored-by: 冰临 <shenxujie.sxj@antgroup.com>
* [lite] [fix] Fix running the legacy experiment in the v0.4.9.post2 sglang environment (#207)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update we…
SathyaGnanakumar
pushed a commit
to danielkiely/AReaL
that referenced
this pull request
Apr 29, 2026
* 0707_1
* 0707_1
* 0707_2
* 0707_2
* 0703_3
* 0703_3
* r
* add api
* add directory structure
* add tests template
* p
* format
* fix
* fix
* checkout previous impl
* checkout previous implementations
* checkout prev impl
* refactor
* add remote sglang engine
* format
* 0707_6
* 0707_7
* add readme
* refactor1
* 0707_undone
* 0708_1
* 0708_2
* 0708_3
* 0708_7
* 0708_4
* 0709_1
* 0709_2
* 0709_3
* PullRequest: 331 [lite] Support remote sglang engine with corresponding testcases.
Merge branch fw/lite of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/331
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* add test for sglang remote engine
* fix
* 0709_4
* 0709_5
* 0709_
* 0709_6
* 0709_7
* PullRequest: 336 add wrapper
Merge branch lite-util-wrapper of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/336
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add wrapper
* 0709_7
* 0709_8
* 0709_9
* PullRequest: 332 [lite] Support FSDP engines
Merge branch mzy/lite/engines of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/332
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* fsdp2 engine
* fix utils
* add fsdp engine test
* .
* fsdp engine test passed
* unsqueeze immediately before model inputs and after model outputts
* add optimizer save/load, add position id calculation for input
* .
* format
* not to squeeze
* add train and eval api
* .
* .
* improve fsdp engine data preprocessing
* format
* PullRequest: 337 [lite] Add SFT trainer example.
* trainer log
* minor changes
* add update weights from disk
* fix type annotation
* PullRequest: 339 [Fix] Fix some minor issues to pass all tests.
Merge branch fw/lite of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/339
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* chore: empty commit
* ci: build images on demand
* ci: fix on demand condition
* ci: fix env sha
* PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods
Merge branch fw/lite-dev of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/340
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* fix
* 0710_1
* 0710_2
* [Fix] Merge previous contributions from fw/refactor to lite (#163)
* initial proposal
* add arealite
* .
* change api
* .
* remove LOG_ROOT
* remove MODEL_SAVE_PATH
* remove PARAM_REALLOC_PATH, DATASET_CACHE
* prepare for testing
* prepare for testing
* ready for run
* local run
* tests mainly pass
* format
* .
* amend cluster.py
* .
* .
* client test pass
* pass rollout test
* remove unused imports
* add arealite readme
* change api
* .
* .
* .
* .
* .
* .
* .
* .
* format
* .
* implement iteraptable generation (#112)
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
* .
* fix
* .
* .
* .
* pass controller generate batch test
* .
* refactor rollout controller into worker and controller
* .
* .
* .
* change to async rollout
* pass rollout controller test
* pass test
* .
* update readme
* .
* sft debug
* .
* add lisence
* remove unused files
* remove unsed args in ppo
* add hf engine wrapper (#116)
* add hf engine
* fix issues
* fix ppo bugs and add test
* add hf client interface and modify cli args
* fix bugs
* fix issues
* Merge fw/refactor
* Finish hf wrapper test
* add test
---------
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
* format
* format
* .
* refine hf engine
* .
* fix
* add fsdp engine and sft tests
* .
* .
* .
* pass ppo unittest
* pass ppo and rollout controller tests
* clear unused imports
* rename ppo to grpo
* change reward function organization
* reorganize code
* add dataset api
* .
* .
* .
* format
* chmod fix
* .
* rename workflow to collector
* refactor llm_client location
* .
* .
* fix llm server api
* refactor config structure
* .
* fix tests
* .
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139)
* .
* .
* efficient loading
* format
* .
* .
* .
* .
* .
* .
* Add CI for testing AReaLite (#150)
* ci: add test-arealite
* ci: add checkout before running test-arealite
* ci: add USERNAME
* ci: add test script
* ci: add GitHub mirror
* ci: fix typo
* ci: clone one commit
* ci: fix condition
* ci: set command timeout to 60m
* ci: enable pip cache
* ci: optimize container lifecycle
* ci: split into many stages
* ci(test-arealite): fix typo
* ci: fix wrong env
* ci: fix pytest
* ci: uninstall transformer-engine
* ci: uninstall transformer-engine
* ci: fix model paths
* ci: show stdout/stderr
* ci: fix not clean up
* ci: backup sglang
* ci: remove tmp repo dir when run
* ci: fix docker run exit 1 condition
* ci(test-arealite): limit the concurrency and extend command timeout
* .
* merge fw/refactor
* revert some changes
* fix
---------
Co-authored-by: meizhiyu.mzy <meizhiyu.mzy@antgroup.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
Co-authored-by: Jayon02 <qiujiangc@outlook.com>
Co-authored-by: root <meizhiyu.mzy>
Co-authored-by: Zijian Zhang <futrime@outlook.com>
* 0710_2
* 0710_3
* [Doc] Add an instruction about how to run the SFT example. (#164)
* 0710_3
* 0710_3
* 0710_5
* 0710_4
* merge_2
* merge_3
* 0711_1
* 0711_2
* 0711_3
* 0711_4
* 0711_6
* 0711_7
* 0711_8
* 0711_8
* 0711_9
* 0711_10
* 0711-11
* [Fix] Fix CI running condition for lite. (#172)
* .
* fix
* .
* .
* fix
* fix
* fix
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* 0714_1
* 0714_2
* 0714_3
* 0714_3
* 0714_5
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* added LocalSGlangEngine and test (#170)
* added LocalSGLangEngine
* upload test file
* add build args
* fix sgl_local generate
* improved sgl local robustness
* test
* test updated
* added fallback when sgl engine isn't initialized
* finish test local engine
* added LocalSGlangEngine and test
* format and fix
format and fix, raise when generate missing field
format
* change cli_args.py
* add comment header
format
---------
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
* format (#174)
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* 0715_1
* 0715_2
* 0715_2
* [Feat][Refactor]Support DeepSpeed AutoTP; Refactor hf_engine.py and unit test. (#161)
* refactor hf engine
* format file
* revert file format
* Squashed commit of the following:
commit 8d4b8dc90fceb144bf00f8125f938b57f696e166
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date: Thu Jul 10 13:14:10 2025 +0800
[Doc] Add an instruction about how to run the SFT example. (#164)
commit 3bf9c85e400a2fe1c6a1e488d27d458ede8cea45
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date: Thu Jul 10 12:56:24 2025 +0800
[Fix] Merge previous contributions from fw/refactor to lite (#163)
* initial proposal
* add arealite
* .
* change api
* .
* remove LOG_ROOT
* remove MODEL_SAVE_PATH
* remove PARAM_REALLOC_PATH, DATASET_CACHE
* prepare for testing
* prepare for testing
* ready for run
* local run
* tests mainly pass
* format
* .
* amend cluster.py
* .
* .
* client test pass
* pass rollout test
* remove unused imports
* add arealite readme
* change api
* .
* .
* .
* .
* .
* .
* .
* .
* format
* .
* implement iteraptable generation (#112)
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
* .
* fix
* .
* .
* .
* pass controller generate batch test
* .
* refactor rollout controller into worker and controller
* .
* .
* .
* change to async rollout
* pass rollout controller test
* pass test
* .
* update readme
* .
* sft debug
* .
* add lisence
* remove unused files
* remove unsed args in ppo
* add hf engine wrapper (#116)
* add hf engine
* fix issues
* fix ppo bugs and add test
* add hf client interface and modify cli args
* fix bugs
* fix issues
* Merge fw/refactor
* Finish hf wrapper test
* add test
---------
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
* format
* format
* .
* refine hf engine
* .
* fix
* add fsdp engine and sft tests
* .
* .
* .
* pass ppo unittest
* pass ppo and rollout controller tests
* clear unused imports
* rename ppo to grpo
* change reward function organization
* reorganize code
* add dataset api
* .
* .
* .
* format
* chmod fix
* .
* rename workflow to collector
* refactor llm_client location
* .
* .
* fix llm server api
* refactor config structure
* .
* fix tests
* .
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139)
* .
* .
* efficient loading
* format
* .
* .
* .
* .
* .
* .
* Add CI for testing AReaLite (#150)
* ci: add test-arealite
* ci: add checkout before running test-arealite
* ci: add USERNAME
* ci: add test script
* ci: add GitHub mirror
* ci: fix typo
* ci: clone one commit
* ci: fix condition
* ci: set command timeout to 60m
* ci: enable pip cache
* ci: optimize container lifecycle
* ci: split into many stages
* ci(test-arealite): fix typo
* ci: fix wrong env
* ci: fix pytest
* ci: uninstall transformer-engine
* ci: uninstall transformer-engine
* ci: fix model paths
* ci: show stdout/stderr
* ci: fix not clean up
* ci: backup sglang
* ci: remove tmp repo dir when run
* ci: fix docker run exit 1 condition
* ci(test-arealite): limit the concurrency and extend command timeout
* .
* merge fw/refactor
* revert some changes
* fix
---------
Co-authored-by: meizhiyu.mzy <meizhiyu.mzy@antgroup.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
Co-authored-by: Jayon02 <qiujiangc@outlook.com>
Co-authored-by: root <meizhiyu.mzy>
Co-authored-by: Zijian Zhang <futrime@outlook.com>
commit d48bf007cfb0821c35a18a673f3c2e8f51a8ee45
Merge: 42c717b b9dbd4a
Author: 博惟 <bowei.fw@antgroup.com>
Date: Thu Jul 10 12:53:30 2025 +0800
Merge branch 'main' of https://github.com/inclusionAI/AReaL into lite
commit 42c717b6e45b3b345f29cff5882ba5374b79faee
Merge: c38cffc a203c7c
Author: 博惟 <bowei.fw@antgroup.com>
Date: Thu Jul 10 11:15:01 2025 +0800
Merge branch 'lite' of https://github.com/inclusionAI/AReaL into lite
commit c38cffc02385acf382196ba6b720b2d8b91a5675
Author: 博惟 <bowei.fw@antgroup.com>
Date: Thu Jul 10 11:10:10 2025 +0800
PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods
Merge branch fw/lite-dev of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/340
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* fix
commit b9dbd4a2c18539648dca373c9c8c1347e4e9db68
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date: Wed Jul 9 10:50:19 2025 +0800
Update to persistent wechat QR code. (#159)
commit 17ea7fe94dc0bd9299082a069383d7372ed13ac9
Author: xssstory <33601810+xssstory@users.noreply.github.com>
Date: Mon Jul 7 15:49:13 2025 +0800
fix math reward verifier (#156)
* PullRequest: 293 fix get_param_realloc_path
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/293
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix get_param_realloc_path
* PullRequest: 297 bugfix: reward is always -5
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/297
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* bugfix: reward is always -5
* PullRequest: 321 fix checkpoint save dir
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/321
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix checkpoint save dir
* PullRequest: 328 [Doc] update installation
Merge branch sxj/doc of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/328
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* [Doc] update installation
* PullRequest: 329 bugfix: math verifier blocks the async training
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/329
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* bugfix: math verifier block the async training
* format
---------
Co-authored-by: 冰临 <shenxujie.sxj@antgroup.com>
Co-authored-by: garrett4wade <fuwth17@gmail.com>
* add autotp for hf
* refactor test
* fix bugs
* fix issues
* format files
* Squashed commit of the following:
commit 9ed043f6ab83cf9c9b38e0251ad08009a73243e8
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date: Tue Jul 15 10:24:48 2025 +0800
format (#174)
commit 8cc9b1feb5ed03cfa42f3ed588dd5f6ec16e5e55
Author: Night <32424487+PrinsYin@users.noreply.github.com>
Date: Mon Jul 14 19:22:00 2025 -0700
added LocalSGlangEngine and test (#170)
* added LocalSGLangEngine
* upload test file
* add build args
* fix sgl_local generate
* improved sgl local robustness
* test
* test updated
* added fallback when sgl engine isn't initialized
* finish test local engine
* added LocalSGlangEngine and test
* format and fix
format and fix, raise when generate missing field
format
* change cli_args.py
* add comment header
format
---------
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
---------
Co-authored-by: Jayon02 <12012211@mail..sustech.edu.cn>
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
* fix ci (#175)
* [Feature] Add pre-commit (#178)
* fix ci
* .
* .
* .
* [lite] [refactor] Add GSM8k GRPO example. (#179)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* fix ci
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* 0716_1
* 0716_2
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* change doc (#180)
* [Fix] [lite] Merge from the internal repo to fix GRPO bugs and refactor the train engine (#181)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* 0716_3
* 0716_4
* 0716_4
* 0716_5
* 0717_1
* 0717_3
* 0717_3
* 0717_4
* 0717_5
* 0717_6
* 0717_6
* 0717_6
* 0718_2
* 0718_4
* 0718_5
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* merge_0721
* 0721_1
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* 0721_2
* 0721_3
* Implement fsdp distributed update (#183)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* added remote nccl weight update
feat: implement update_weights_from_distributed in fsdp_engine.py
unfinishd test, raise PR first
coroutine for each server
chore: change uploads weights behavior, change test order
fix small bug
fixed test
* fix rebase
* add test.sh
* updated, test stil fails
* .
* .
* .
* fix: full_tensor() should happen in all rank (#187)
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
Co-authored-by: 博惟 <bowei.fw@antgroup.com>
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
Co-authored-by: ChangyiYang <112288487+ChangyiYang@users.noreply.github.com>
* [Feature] [lite] Merge from internal dev repo (#189)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* format
* format
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* [Refactor] Rename files in arealite before release. (#190)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* format
* format
* .
* rename
* rename
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* 0721_4
* .
* 0721_formal
* 0721_formal
* 0721_merge4
* 0721_merge5
* 0721_6
* 0721_merge6
* 0721_merge7
* 0721_8
* 0722_1
* 0722_2
* 0722_3
* add quickstart (#194)
* [doc] [lite] Add customization docs for AReaLite. (#191)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* .
* .
* .
* .
* .
* .
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* 0722_4
* 0722_4
* 0722_5
* 0722_6
* 0722_7
* 0723_1
* reformatted
* clang-reformatted
* clang-reformatted2
* 0723_1
* 0723_1
* 0723_1
* 0723_merge3
* 0723_4
* 0723_reformatted_5
* 0724_1
* 0724_1
* 0724_merge1
* 0724_merge2
* 0724_merge3
* 0724_merge3
* 0724_merge4
* [lite] [feature] Bump to SGLang v0.4.9.post2 and use NCCL to update weights (#196)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* .
* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .
* PullRequest: 423 [lite] Remove the boba example for github release.
Merge branch fw/remove-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* 0724_merge5
* 0724_merge6
* 0724_merge7
* [Docs] [lite] Add example code walkthrough documentation. (#197)
* .
* .
* fix grammar and minor revise
* [lite] [doc] Add AReaLite design doc as README (#198)
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* .
* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9
Merge branch fw/sgl049 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .
* PullRequest: 423 [lite] Remove the boba example for github release.
Merge branch fw/remove-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* update readme
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* 0724_4
* 0724-merge8
* 0724_merge8
* 0725_1
* 0725_6
* 0725_7
* 0725_4padded_image
* 0725_9padded_image
* 0725_10padded_image
* 0725
* 0725_12
* 0725_format
* Add self-hosted runner support (#199)
* ci: add testsz for self hosted runner
* ci: use self-hosted runner
* ci: run ci on push
* ci: remove cd to tmp dir
* ci: fix
* ci: add environment back
* ci: set concurrency
* ci: fix build env image
* Update .github/workflows/test-arealite.yml
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update ci/build_env_image.sh
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* ci: add some test code
* ci: simplify
* ci: try to fix
* ci: test
* ci: test2
* ci: test3
* ci: test3
* ci: tidy up
* ci: simplify
* ci: do not trigger on push
* ci: test arealite/tests/
* ci: use UUID as run id
* ci: use openssl rand to genreate run id
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* [WIP][feat] Initial support for VLMs, add Qwen2VL SFT test and Qwen2.5VL GRPO test (#188)
* vlm_sft_test
* vlm_sft_test
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139)
* .
* .
* efficient loading
* format
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139)
* .
* .
* efficient loading
* format
* .
* .
* image_process0701
* image_process0701
* image_process0701_2
* image_process0701_2
* image_process0701_3
* image_process0701_3
* .
* .
* .
* .
* .
* .
* imageprocess0702
* imageprocess0702
* image_process0702_2
* image_process0702_2
* image_process0702_3
* image_process0702_3
* image_process0702_4
* image_process0702_4
* image_process0702_5
* image_process0702_5
* image_process0703_1
* image_process0703_1
* 0703_2
* 0703_2
* 0703_3
* 0703_3
* 0703_4
* 0703_4
* 0703_4
* 0703_4
* 0703_5
* 0703_5
* 0703_6
* 0703_6
* 0703_7
* 0703_7
* 0703_8
* 0703_8
* 0703_9
* 0703_9
* 0703_11
* 0703_11
* 0703_12
* 0703_12
* 0703_13
* 0703_13
* 0703_14
* 0703_14
* 0703_15
* 0703_15
* 0703_16
* 0703_16
* 0703-17
* 0703-17
* 0703_18
* 0703_18
* 0703_18
* 0703_18
* 0703_19
* 0703_19
* 0704_1
* 0704_1
* 0704_2
* 0704_2
* 0704_3
* 0704_3
* .
* .
* 0707_1
* 0707_1
* 0707_2
* 0707_2
* 0703_3
* 0703_3
* r
* p
* fix
* fix
* refactor
* 0707_6
* 0707_7
* refactor1
* 0707_undone
* 0708_1
* 0708_2
* 0708_3
* 0708_7
* 0708_4
* 0709_1
* 0709_2
* 0709_3
* 0709_4
* 0709_5
* 0709_
* 0709_6
* 0709_7
* 0709_7
* 0709_8
* 0709_9
* 0710_1
* 0710_2
* 0710_2
* 0710_3
* 0710_3
* 0710_3
* 0710_5
* 0710_4
* merge_2
* merge_3
* 0711_1
* 0711_2
* 0711_3
* 0711_4
* 0711_6
* 0711_7
* 0711_8
* 0711_8
* 0711_9
* 0711_10
* 0711-11
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* 0714_1
* 0714_2
* 0714_3
* 0714_3
* 0714_5
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* 0715_1
* 0715_2
* 0715_2
* 0716_1
* 0716_2
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* 0716_3
* 0716_4
* 0716_4
* 0716_5
* 0717_1
* 0717_3
* 0717_3
* 0717_4
* 0717_5
* 0717_6
* 0717_6
* 0717_6
* 0718_2
* 0718_4
* 0718_5
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* merge_0721
* 0721_1
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* 0721_2
* 0721_3
* 0721_4
* .
* 0721_formal
* 0721_formal
* 0721_merge4
* 0721_merge5
* 0721_6
* 0721_merge6
* 0721_merge7
* 0721_8
* 0722_1
* 0722_2
* 0722_3
* 0722_4
* 0722_4
* 0722_5
* 0722_6
* 0722_7
* 0723_1
* reformatted
* clang-reformatted
* clang-reformatted2
* 0723_1
* 0723_1
* 0723_1
* 0723_merge3
* 0723_4
* 0723_reformatted_5
* 0724_1
* 0724_1
* 0724_merge1
* 0724_merge2
* 0724_merge3
* 0724_merge3
* 0724_merge4
* 0724_merge5
* 0724_merge6
* 0724_merge7
* 0724_4
* 0724-merge8
* 0724_merge8
* 0725_1
* 0725_6
* 0725_7
* 0725_4padded_image
* 0725_9padded_image
* 0725_10padded_image
* 0725
* 0725_12
* 0725_format
---------
Co-authored-by: bowei.fw <bowei.fw@antgroup.com>
Co-authored-by: nuzant <meizhiyu.mzy@antgroup.com>
Co-authored-by: 朱晗 <lichangye.lcy@antgroup.com>
* 0731
* 0731_1
* 0731_2
* 0731_2
* 0731_3
* 0731_4
* 0801_1
* 0801_2
* 0804_1
* 0804_2
* 0804_2
* 0804_5
* 0805_3
* 0805_2
* 0806
* 0806_merge1
* 0806_merge2
* 0806_format1
* 0806_merge3
* 0806_4
* 0806_6
* 0806_7
* 0806_formatted2
* fix
* revert examples
* .
* .
---------
Co-authored-by: 朱晗 <lichangye.lcy@antgroup.com>
Co-authored-by: 博惟 <bowei.fw@antgroup.com>
Co-authored-by: garrett4wade <fuwth17@gmail.com>
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.