Skip to content

fix math reward verifier #156

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
garrett4wade merged 7 commits into from
Jul 7, 2025
Merged

fix math reward verifier #156

garrett4wade merged 7 commits into from
Jul 7, 2025

Conversation

@xssstory
Copy link
Collaborator

@xssstory xssstory commented Jul 7, 2025 •
edited
Loading

During async training, having too many verifier processes can cause the entire training to hang (under slurm) or some processes to be dropped (under ray).
Now, within the async training, multiple rollouts in the same group (same prompt) will undergo verification sequentially.

xssstory and others added 5 commits June 26, 2025 21:26
@xssstory @garrett4wade
PullRequest: 293 fix get_param_realloc_path
a5cabdd 
Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/293

Reviewed-by: 博惟 <[email protected]>


* fix get_param_realloc_path
@xssstory @garrett4wade
PullRequest: 297 bugfix: reward is always -5
623f7c7 
Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/297

Reviewed-by: 博惟 <[email protected]>


* bugfix: reward is always -5
@xssstory @garrett4wade
PullRequest: 321 fix checkpoint save dir
ed6bcd6 
Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/321

Reviewed-by: 博惟 <[email protected]>


* fix checkpoint save dir
@fishcrap @garrett4wade
PullRequest: 328 [Doc] update installation
ff7beeb 
Merge branch sxj/doc of [email protected]:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/328

Reviewed-by: 博惟 <[email protected]>


* [Doc] update installation
@xssstory @garrett4wade
PullRequest: 329 bugfix: math verifier blocks the async training
f2c4828 
Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/329

Reviewed-by: 博惟 <[email protected]>


* bugfix: math verifier block the async training
@xssstory xssstory changed the title fix fix math reward verifier Jul 7, 2025
@xssstory xssstory closed this Jul 7, 2025
@xssstory xssstory reopened this Jul 7, 2025
@garrett4wade garrett4wade merged commit 17ea7fe into main Jul 7, 2025
1 check passed
@garrett4wade garrett4wade deleted the xss/debug branch July 7, 2025 07:49
Jayon02 pushed a commit to Jayon02/AReaL that referenced this pull request Jul 10, 2025
Squashed commit of the following:
b14f45f 
commit 8d4b8dc
Author: Wei Fu <[email protected]>
Date:   Thu Jul 10 13:14:10 2025 +0800

    [Doc] Add an instruction about how to run the SFT example. (inclusionAI#164)

commit 3bf9c85
Author: Wei Fu <[email protected]>
Date:   Thu Jul 10 12:56:24 2025 +0800

    [Fix] Merge previous contributions from fw/refactor to lite (inclusionAI#163)

    * initial proposal

    * add arealite

    * .

    * change api

    * .

    * remove LOG_ROOT

    * remove MODEL_SAVE_PATH

    * remove PARAM_REALLOC_PATH, DATASET_CACHE

    * prepare for testing

    * prepare for testing

    * ready for run

    * local run

    * tests mainly pass

    * format

    * .

    * amend cluster.py

    * .

    * .

    * client test pass

    * pass rollout test

    * remove unused imports

    * add arealite readme

    * change api

    * .

    * .

    * .

    * .

    * .

    * .

    * .

    * .

    * format

    * .

    * implement iteraptable generation (inclusionAI#112)

    Co-authored-by: zhaochenyang <[email protected]>

    * .

    * fix

    * .

    * .

    * .

    * pass controller generate batch test

    * .

    * refactor rollout controller into worker and controller

    * .

    * .

    * .

    * change to async rollout

    * pass rollout controller test

    * pass test

    * .

    * update readme

    * .

    * sft debug

    * .

    * add lisence

    * remove unused files

    * remove unsed args in ppo

    * add hf engine wrapper  (inclusionAI#116)

    * add hf engine

    * fix issues

    * fix ppo bugs and add test

    * add hf client interface and modify cli args

    * fix bugs

    * fix issues

    * Merge fw/refactor

    * Finish hf wrapper test

    * add test

    ---------

    Co-authored-by: Wei Fu <[email protected]>

    * format

    * format

    * .

    * refine hf engine

    * .

    * fix

    * add fsdp engine and sft tests

    * .

    * .

    * .

    * pass ppo unittest

    * pass ppo and rollout controller tests

    * clear unused imports

    * rename ppo to grpo

    * change reward function organization

    * reorganize code

    * add dataset api

    * .

    * .

    * .

    * format

    * chmod fix

    * .

    * rename workflow to collector

    * refactor llm_client location

    * .

    * .

    * fix llm server api

    * refactor config structure

    * .

    * fix tests

    * .

    * .

    * .

    * Fix unresolved issue in SFTTrainer PR (inclusionAI#139)

    * .

    * .

    * efficient loading

    * format

    * .

    * .

    * .

    * .

    * .

    * .

    * Add CI for testing AReaLite (inclusionAI#150)

    * ci: add test-arealite

    * ci: add checkout before running test-arealite

    * ci: add USERNAME

    * ci: add test script

    * ci: add GitHub mirror

    * ci: fix typo

    * ci: clone one commit

    * ci: fix condition

    * ci: set command timeout to 60m

    * ci: enable pip cache

    * ci: optimize container lifecycle

    * ci: split into many stages

    * ci(test-arealite): fix typo

    * ci: fix wrong env

    * ci: fix pytest

    * ci: uninstall transformer-engine

    * ci: uninstall transformer-engine

    * ci: fix model paths

    * ci: show stdout/stderr

    * ci: fix not clean up

    * ci: backup sglang

    * ci: remove tmp repo dir when run

    * ci: fix docker run exit 1 condition

    * ci(test-arealite): limit the concurrency and extend command timeout

    * .

    * merge fw/refactor

    * revert some changes

    * fix

    ---------

    Co-authored-by: meizhiyu.mzy <[email protected]>
    Co-authored-by: Chayenne <[email protected]>
    Co-authored-by: zhaochenyang <[email protected]>
    Co-authored-by: Jayon02 <[email protected]>
    Co-authored-by: root <meizhiyu.mzy>
    Co-authored-by: Zijian Zhang <[email protected]>

commit d48bf00
Merge: 42c717b b9dbd4a
Author: 博惟 <[email protected]>
Date:   Thu Jul 10 12:53:30 2025 +0800

    Merge branch 'main' of https://github.com/inclusionAI/AReaL into lite

commit 42c717b
Merge: c38cffc a203c7c
Author: 博惟 <[email protected]>
Date:   Thu Jul 10 11:15:01 2025 +0800

    Merge branch 'lite' of https://github.com/inclusionAI/AReaL into lite

commit c38cffc
Author: 博惟 <[email protected]>
Date:   Thu Jul 10 11:10:10 2025 +0800

    PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods

    Merge branch fw/lite-dev of [email protected]:inclusionAI/AReaL.git into lite
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/340

    Reviewed-by: 晓雷 <[email protected]>

    * support fsdp engine and sglang remote engine
    * minor fix
    * .
    * refactor trainer
    * add close
    * rm mb_spec
    * fix

commit b9dbd4a
Author: Wei Fu <[email protected]>
Date:   Wed Jul 9 10:50:19 2025 +0800

    Update to persistent wechat QR code. (inclusionAI#159)

commit 17ea7fe
Author: xssstory <[email protected]>
Date:   Mon Jul 7 15:49:13 2025 +0800

    fix math reward verifier (inclusionAI#156)

    * PullRequest: 293 fix get_param_realloc_path

    Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/293

    Reviewed-by: 博惟 <[email protected]>

    * fix get_param_realloc_path

    * PullRequest: 297 bugfix: reward is always -5

    Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/297

    Reviewed-by: 博惟 <[email protected]>

    * bugfix: reward is always -5

    * PullRequest: 321 fix checkpoint save dir

    Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/321

    Reviewed-by: 博惟 <[email protected]>

    * fix checkpoint save dir

    * PullRequest: 328 [Doc] update installation

    Merge branch sxj/doc of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/328

    Reviewed-by: 博惟 <[email protected]>

    * [Doc] update installation

    * PullRequest: 329 bugfix: math verifier blocks the async training

    Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/329

    Reviewed-by: 博惟 <[email protected]>

    * bugfix: math verifier block the async training

    * format

    ---------

    Co-authored-by: 冰临 <[email protected]>
    Co-authored-by: garrett4wade <[email protected]>
garrett4wade added a commit that referenced this pull request Jul 16, 2025
@Jayon02 @garrett4wade
[Feat][Refactor]Support DeepSpeed AutoTP; Refactor hf_engine.py and u…
ef4215d 
…nit test. (#161)

* refactor hf engine

* format file

* revert file format

* Squashed commit of the following:

commit 8d4b8dc
Author: Wei Fu <[email protected]>
Date:   Thu Jul 10 13:14:10 2025 +0800

    [Doc] Add an instruction about how to run the SFT example. (#164)

commit 3bf9c85
Author: Wei Fu <[email protected]>
Date:   Thu Jul 10 12:56:24 2025 +0800

    [Fix] Merge previous contributions from fw/refactor to lite (#163)

    * initial proposal

    * add arealite

    * .

    * change api

    * .

    * remove LOG_ROOT

    * remove MODEL_SAVE_PATH

    * remove PARAM_REALLOC_PATH, DATASET_CACHE

    * prepare for testing

    * prepare for testing

    * ready for run

    * local run

    * tests mainly pass

    * format

    * .

    * amend cluster.py

    * .

    * .

    * client test pass

    * pass rollout test

    * remove unused imports

    * add arealite readme

    * change api

    * .

    * .

    * .

    * .

    * .

    * .

    * .

    * .

    * format

    * .

    * implement iteraptable generation (#112)

    Co-authored-by: zhaochenyang <[email protected]>

    * .

    * fix

    * .

    * .

    * .

    * pass controller generate batch test

    * .

    * refactor rollout controller into worker and controller

    * .

    * .

    * .

    * change to async rollout

    * pass rollout controller test

    * pass test

    * .

    * update readme

    * .

    * sft debug

    * .

    * add lisence

    * remove unused files

    * remove unsed args in ppo

    * add hf engine wrapper  (#116)

    * add hf engine

    * fix issues

    * fix ppo bugs and add test

    * add hf client interface and modify cli args

    * fix bugs

    * fix issues

    * Merge fw/refactor

    * Finish hf wrapper test

    * add test

    ---------

    Co-authored-by: Wei Fu <[email protected]>

    * format

    * format

    * .

    * refine hf engine

    * .

    * fix

    * add fsdp engine and sft tests

    * .

    * .

    * .

    * pass ppo unittest

    * pass ppo and rollout controller tests

    * clear unused imports

    * rename ppo to grpo

    * change reward function organization

    * reorganize code

    * add dataset api

    * .

    * .

    * .

    * format

    * chmod fix

    * .

    * rename workflow to collector

    * refactor llm_client location

    * .

    * .

    * fix llm server api

    * refactor config structure

    * .

    * fix tests

    * .

    * .

    * .

    * Fix unresolved issue in SFTTrainer PR (#139)

    * .

    * .

    * efficient loading

    * format

    * .

    * .

    * .

    * .

    * .

    * .

    * Add CI for testing AReaLite (#150)

    * ci: add test-arealite

    * ci: add checkout before running test-arealite

    * ci: add USERNAME

    * ci: add test script

    * ci: add GitHub mirror

    * ci: fix typo

    * ci: clone one commit

    * ci: fix condition

    * ci: set command timeout to 60m

    * ci: enable pip cache

    * ci: optimize container lifecycle

    * ci: split into many stages

    * ci(test-arealite): fix typo

    * ci: fix wrong env

    * ci: fix pytest

    * ci: uninstall transformer-engine

    * ci: uninstall transformer-engine

    * ci: fix model paths

    * ci: show stdout/stderr

    * ci: fix not clean up

    * ci: backup sglang

    * ci: remove tmp repo dir when run

    * ci: fix docker run exit 1 condition

    * ci(test-arealite): limit the concurrency and extend command timeout

    * .

    * merge fw/refactor

    * revert some changes

    * fix

    ---------

    Co-authored-by: meizhiyu.mzy <[email protected]>
    Co-authored-by: Chayenne <[email protected]>
    Co-authored-by: zhaochenyang <[email protected]>
    Co-authored-by: Jayon02 <[email protected]>
    Co-authored-by: root <meizhiyu.mzy>
    Co-authored-by: Zijian Zhang <[email protected]>

commit d48bf00
Merge: 42c717b b9dbd4a
Author: 博惟 <[email protected]>
Date:   Thu Jul 10 12:53:30 2025 +0800

    Merge branch 'main' of https://github.com/inclusionAI/AReaL into lite

commit 42c717b
Merge: c38cffc a203c7c
Author: 博惟 <[email protected]>
Date:   Thu Jul 10 11:15:01 2025 +0800

    Merge branch 'lite' of https://github.com/inclusionAI/AReaL into lite

commit c38cffc
Author: 博惟 <[email protected]>
Date:   Thu Jul 10 11:10:10 2025 +0800

    PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods

    Merge branch fw/lite-dev of [email protected]:inclusionAI/AReaL.git into lite
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/340

    Reviewed-by: 晓雷 <[email protected]>

    * support fsdp engine and sglang remote engine
    * minor fix
    * .
    * refactor trainer
    * add close
    * rm mb_spec
    * fix

commit b9dbd4a
Author: Wei Fu <[email protected]>
Date:   Wed Jul 9 10:50:19 2025 +0800

    Update to persistent wechat QR code. (#159)

commit 17ea7fe
Author: xssstory <[email protected]>
Date:   Mon Jul 7 15:49:13 2025 +0800

    fix math reward verifier (#156)

    * PullRequest: 293 fix get_param_realloc_path

    Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/293

    Reviewed-by: 博惟 <[email protected]>

    * fix get_param_realloc_path

    * PullRequest: 297 bugfix: reward is always -5

    Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/297

    Reviewed-by: 博惟 <[email protected]>

    * bugfix: reward is always -5

    * PullRequest: 321 fix checkpoint save dir

    Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/321

    Reviewed-by: 博惟 <[email protected]>

    * fix checkpoint save dir

    * PullRequest: 328 [Doc] update installation

    Merge branch sxj/doc of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/328

    Reviewed-by: 博惟 <[email protected]>

    * [Doc] update installation

    * PullRequest: 329 bugfix: math verifier blocks the async training

    Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/329

    Reviewed-by: 博惟 <[email protected]>

    * bugfix: math verifier block the async training

    * format

    ---------

    Co-authored-by: 冰临 <[email protected]>
    Co-authored-by: garrett4wade <[email protected]>

* add autotp for hf

* refactor test

* fix bugs

* fix issues

* format files

* Squashed commit of the following:

commit 9ed043f
Author: Wei Fu <[email protected]>
Date:   Tue Jul 15 10:24:48 2025 +0800

    format (#174)

commit 8cc9b1f
Author: Night <[email protected]>
Date:   Mon Jul 14 19:22:00 2025 -0700

    added LocalSGlangEngine and test (#170)

    * added LocalSGLangEngine

    * upload test file

    * add build args

    * fix sgl_local generate

    * improved sgl local robustness

    * test

    * test updated

    * added fallback when sgl engine isn't initialized

    * finish test local engine

    * added LocalSGlangEngine and test

    * format and fix

    format and fix, raise when generate missing field

    format

    * change cli_args.py

    * add comment header

    format

    ---------

    Co-authored-by: ChangyiYang <[email protected]>

---------

Co-authored-by: Jayon02 <[email protected]>
Co-authored-by: Wei Fu <[email protected]>
garrett4wade added a commit that referenced this pull request Aug 1, 2025
@garrett4wade @xssstory
@kdada @futrime @nuzant @zhaochenyang20 @zhaochen20 @Jayon02 @PrinsYin @ChangyiYang @antoinegg1 @fishcrap @richardbsk
Refactor stage #1: AReaL-lite (#154)
094adbb 
* PullRequest: 293 fix get_param_realloc_path

Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/293

Reviewed-by: 博惟 <[email protected]>


* fix get_param_realloc_path

* PullRequest: 297 bugfix: reward is always -5

Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/297

Reviewed-by: 博惟 <[email protected]>


* bugfix: reward is always -5

* PullRequest: 321 fix checkpoint save dir

Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/321

Reviewed-by: 博惟 <[email protected]>


* fix checkpoint save dir

* add api

* add directory structure

* add tests template

* format

* checkout previous impl

* checkout previous implementations

* checkout prev impl

* add remote sglang engine

* format

* add readme

* PullRequest: 331 [lite] Support remote sglang engine with corresponding testcases.

Merge branch fw/lite of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/331

Reviewed-by: 晓雷 <[email protected]>


* .
* add test for sglang remote engine
* fix

* PullRequest: 336 add wrapper

Merge branch lite-util-wrapper of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/336

Reviewed-by: 博惟 <[email protected]>


* add wrapper

* PullRequest: 332 [lite] Support FSDP engines

Merge branch mzy/lite/engines of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/332

Reviewed-by: 晓雷 <[email protected]>


* fsdp2 engine
* fix utils
* add fsdp engine test
* .
* fsdp engine test passed
* unsqueeze immediately before model inputs and after model outputts
* add optimizer save/load, add position id calculation for input
* .
* format
* not to squeeze
* add train and eval api
* .
* .
* improve fsdp engine data preprocessing
* format
* PullRequest: 337 [lite] Add SFT trainer example.
* trainer log
* minor changes
* add update weights from disk
* fix type annotation

* PullRequest: 339 [Fix] Fix some minor issues to pass all tests.

Merge branch fw/lite of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/339

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .

* chore: empty commit

* ci: build images on demand

* ci: fix on demand condition

* ci: fix env sha

* PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods

Merge branch fw/lite-dev of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/340

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* fix

* [Fix] Merge previous contributions from fw/refactor to lite (#163)

* initial proposal

* add arealite

* .

* change api

* .

* remove LOG_ROOT

* remove MODEL_SAVE_PATH

* remove PARAM_REALLOC_PATH, DATASET_CACHE

* prepare for testing

* prepare for testing

* ready for run

* local run

* tests mainly pass

* format

* .

* amend cluster.py

* .

* .

* client test pass

* pass rollout test

* remove unused imports

* add arealite readme

* change api

* .

* .

* .

* .

* .

* .

* .

* .

* format

* .

* implement iteraptable generation (#112)

Co-authored-by: zhaochenyang <[email protected]>

* .

* fix

* .

* .

* .

* pass controller generate batch test

* .

* refactor rollout controller into worker and controller

* .

* .

* .

* change to async rollout

* pass rollout controller test

* pass test

* .

* update readme

* .

* sft debug

* .

* add lisence

* remove unused files

* remove unsed args in ppo

* add hf engine wrapper  (#116)

* add hf engine

* fix issues

* fix ppo bugs and add test

* add hf client interface and modify cli args

* fix bugs

* fix issues

* Merge fw/refactor

* Finish hf wrapper test

* add test

---------

Co-authored-by: Wei Fu <[email protected]>

* format

* format

* .

* refine hf engine

* .

* fix

* add fsdp engine and sft tests

* .

* .

* .

* pass ppo unittest

* pass ppo and rollout controller tests

* clear unused imports

* rename ppo to grpo

* change reward function organization

* reorganize code

* add dataset api

* .

* .

* .

* format

* chmod fix

* .

* rename workflow to collector

* refactor llm_client location

* .

* .

* fix llm server api

* refactor config structure

* .

* fix tests

* .

* .

* .

* Fix unresolved issue in SFTTrainer PR (#139)

* .

* .

* efficient loading

* format

* .

* .

* .

* .

* .

* .

* Add CI for testing AReaLite (#150)

* ci: add test-arealite

* ci: add checkout before running test-arealite

* ci: add USERNAME

* ci: add test script

* ci: add GitHub mirror

* ci: fix typo

* ci: clone one commit

* ci: fix condition

* ci: set command timeout to 60m

* ci: enable pip cache

* ci: optimize container lifecycle

* ci: split into many stages

* ci(test-arealite): fix typo

* ci: fix wrong env

* ci: fix pytest

* ci: uninstall transformer-engine

* ci: uninstall transformer-engine

* ci: fix model paths

* ci: show stdout/stderr

* ci: fix not clean up

* ci: backup sglang

* ci: remove tmp repo dir when run

* ci: fix docker run exit 1 condition

* ci(test-arealite): limit the concurrency and extend command timeout

* .

* merge fw/refactor

* revert some changes

* fix

---------

Co-authored-by: meizhiyu.mzy <[email protected]>
Co-authored-by: Chayenne <[email protected]>
Co-authored-by: zhaochenyang <[email protected]>
Co-authored-by: Jayon02 <[email protected]>
Co-authored-by: root <meizhiyu.mzy>
Co-authored-by: Zijian Zhang <[email protected]>

* [Doc] Add an instruction about how to run the SFT example. (#164)

* [Fix] Fix CI running condition for lite. (#172)

* .

* fix

* .

* .

* fix

* fix

* fix

* added LocalSGlangEngine and test (#170)

* added LocalSGLangEngine

* upload test file

* add build args

* fix sgl_local generate

* improved sgl local robustness

* test

* test updated

* added fallback when sgl engine isn't initialized

* finish test local engine

* added LocalSGlangEngine and test

* format and fix

format and fix, raise when generate missing field

format

* change cli_args.py

* add comment header

format

---------

Co-authored-by: ChangyiYang <[email protected]>

* format (#174)

* [Feat][Refactor]Support DeepSpeed AutoTP; Refactor hf_engine.py and unit test. (#161)

* refactor hf engine

* format file

* revert file format

* Squashed commit of the following:

commit 8d4b8dc90fceb144bf00f8125f938b57f696e166
Author: Wei Fu <[email protected]>
Date:   Thu Jul 10 13:14:10 2025 +0800

    [Doc] Add an instruction about how to run the SFT example. (#164)

commit 3bf9c85e400a2fe1c6a1e488d27d458ede8cea45
Author: Wei Fu <[email protected]>
Date:   Thu Jul 10 12:56:24 2025 +0800

    [Fix] Merge previous contributions from fw/refactor to lite (#163)

    * initial proposal

    * add arealite

    * .

    * change api

    * .

    * remove LOG_ROOT

    * remove MODEL_SAVE_PATH

    * remove PARAM_REALLOC_PATH, DATASET_CACHE

    * prepare for testing

    * prepare for testing

    * ready for run

    * local run

    * tests mainly pass

    * format

    * .

    * amend cluster.py

    * .

    * .

    * client test pass

    * pass rollout test

    * remove unused imports

    * add arealite readme

    * change api

    * .

    * .

    * .

    * .

    * .

    * .

    * .

    * .

    * format

    * .

    * implement iteraptable generation (#112)

    Co-authored-by: zhaochenyang <[email protected]>

    * .

    * fix

    * .

    * .

    * .

    * pass controller generate batch test

    * .

    * refactor rollout controller into worker and controller

    * .

    * .

    * .

    * change to async rollout

    * pass rollout controller test

    * pass test

    * .

    * update readme

    * .

    * sft debug

    * .

    * add lisence

    * remove unused files

    * remove unsed args in ppo

    * add hf engine wrapper  (#116)

    * add hf engine

    * fix issues

    * fix ppo bugs and add test

    * add hf client interface and modify cli args

    * fix bugs

    * fix issues

    * Merge fw/refactor

    * Finish hf wrapper test

    * add test

    ---------

    Co-authored-by: Wei Fu <[email protected]>

    * format

    * format

    * .

    * refine hf engine

    * .

    * fix

    * add fsdp engine and sft tests

    * .

    * .

    * .

    * pass ppo unittest

    * pass ppo and rollout controller tests

    * clear unused imports

    * rename ppo to grpo

    * change reward function organization

    * reorganize code

    * add dataset api

    * .

    * .

    * .

    * format

    * chmod fix

    * .

    * rename workflow to collector

    * refactor llm_client location

    * .

    * .

    * fix llm server api

    * refactor config structure

    * .

    * fix tests

    * .

    * .

    * .

    * Fix unresolved issue in SFTTrainer PR (#139)

    * .

    * .

    * efficient loading

    * format

    * .

    * .

    * .

    * .

    * .

    * .

    * Add CI for testing AReaLite (#150)

    * ci: add test-arealite

    * ci: add checkout before running test-arealite

    * ci: add USERNAME

    * ci: add test script

    * ci: add GitHub mirror

    * ci: fix typo

    * ci: clone one commit

    * ci: fix condition

    * ci: set command timeout to 60m

    * ci: enable pip cache

    * ci: optimize container lifecycle

    * ci: split into many stages

    * ci(test-arealite): fix typo

    * ci: fix wrong env

    * ci: fix pytest

    * ci: uninstall transformer-engine

    * ci: uninstall transformer-engine

    * ci: fix model paths

    * ci: show stdout/stderr

    * ci: fix not clean up

    * ci: backup sglang

    * ci: remove tmp repo dir when run

    * ci: fix docker run exit 1 condition

    * ci(test-arealite): limit the concurrency and extend command timeout

    * .

    * merge fw/refactor

    * revert some changes

    * fix

    ---------

    Co-authored-by: meizhiyu.mzy <[email protected]>
    Co-authored-by: Chayenne <[email protected]>
    Co-authored-by: zhaochenyang <[email protected]>
    Co-authored-by: Jayon02 <[email protected]>
    Co-authored-by: root <meizhiyu.mzy>
    Co-authored-by: Zijian Zhang <[email protected]>

commit d48bf007cfb0821c35a18a673f3c2e8f51a8ee45
Merge: 42c717b b9dbd4a
Author: 博惟 <[email protected]>
Date:   Thu Jul 10 12:53:30 2025 +0800

    Merge branch 'main' of https://github.com/inclusionAI/AReaL into lite

commit 42c717b6e45b3b345f29cff5882ba5374b79faee
Merge: c38cffc a203c7c
Author: 博惟 <[email protected]>
Date:   Thu Jul 10 11:15:01 2025 +0800

    Merge branch 'lite' of https://github.com/inclusionAI/AReaL into lite

commit c38cffc02385acf382196ba6b720b2d8b91a5675
Author: 博惟 <[email protected]>
Date:   Thu Jul 10 11:10:10 2025 +0800

    PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods

    Merge branch fw/lite-dev of [email protected]:inclusionAI/AReaL.git into lite
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/340

    Reviewed-by: 晓雷 <[email protected]>

    * support fsdp engine and sglang remote engine
    * minor fix
    * .
    * refactor trainer
    * add close
    * rm mb_spec
    * fix

commit b9dbd4a2c18539648dca373c9c8c1347e4e9db68
Author: Wei Fu <[email protected]>
Date:   Wed Jul 9 10:50:19 2025 +0800

    Update to persistent wechat QR code. (#159)

commit 17ea7fe94dc0bd9299082a069383d7372ed13ac9
Author: xssstory <[email protected]>
Date:   Mon Jul 7 15:49:13 2025 +0800

    fix math reward verifier (#156)

    * PullRequest: 293 fix get_param_realloc_path

    Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/293

    Reviewed-by: 博惟 <[email protected]>

    * fix get_param_realloc_path

    * PullRequest: 297 bugfix: reward is always -5

    Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/297

    Reviewed-by: 博惟 <[email protected]>

    * bugfix: reward is always -5

    * PullRequest: 321 fix checkpoint save dir

    Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/321

    Reviewed-by: 博惟 <[email protected]>

    * fix checkpoint save dir

    * PullRequest: 328 [Doc] update installation

    Merge branch sxj/doc of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/328

    Reviewed-by: 博惟 <[email protected]>

    * [Doc] update installation

    * PullRequest: 329 bugfix: math verifier blocks the async training

    Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/329

    Reviewed-by: 博惟 <[email protected]>

    * bugfix: math verifier block the async training

    * format

    ---------

    Co-authored-by: 冰临 <[email protected]>
    Co-authored-by: garrett4wade <[email protected]>

* add autotp for hf

* refactor test

* fix bugs

* fix issues

* format files

* Squashed commit of the following:

commit 9ed043f6ab83cf9c9b38e0251ad08009a73243e8
Author: Wei Fu <[email protected]>
Date:   Tue Jul 15 10:24:48 2025 +0800

    format (#174)

commit 8cc9b1feb5ed03cfa42f3ed588dd5f6ec16e5e55
Author: Night <[email protected]>
Date:   Mon Jul 14 19:22:00 2025 -0700

    added LocalSGlangEngine and test (#170)

    * added LocalSGLangEngine

    * upload test file

    * add build args

    * fix sgl_local generate

    * improved sgl local robustness

    * test

    * test updated

    * added fallback when sgl engine isn't initialized

    * finish test local engine

    * added LocalSGlangEngine and test

    * format and fix

    format and fix, raise when generate missing field

    format

    * change cli_args.py

    * add comment header

    format

    ---------

    Co-authored-by: ChangyiYang <[email protected]>

---------

Co-authored-by: Jayon02 <[email protected]>
Co-authored-by: Wei Fu <[email protected]>

* fix ci (#175)

* [Feature] Add pre-commit (#178)

* fix ci

* .

* .

* .

* [lite] [refactor] Add GSM8k GRPO example. (#179)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* fix ci

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

---------

Co-authored-by: 晓雷 <[email protected]>

* change doc (#180)

* [Fix] [lite] Merge from the internal repo to fix GRPO bugs and refactor the train engine (#181)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

---------

Co-authored-by: 晓雷 <[email protected]>

* Implement fsdp distributed update (#183)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* added remote nccl weight update

feat: implement update_weights_from_distributed in fsdp_engine.py

unfinishd test, raise PR first

coroutine for each server

chore: change uploads weights behavior, change test order

fix small bug

fixed test

* fix rebase

* add test.sh

* updated, test stil fails

* .

* .

* .

* fix: full_tensor() should happen in all rank (#187)

Co-authored-by: ChangyiYang <[email protected]>

---------

Co-authored-by: 晓雷 <[email protected]>
Co-authored-by: 博惟 <[email protected]>
Co-authored-by: ChangyiYang <[email protected]>
Co-authored-by: ChangyiYang <[email protected]>

* [Feature] [lite] Merge from internal dev repo (#189)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* format

* format

* .

---------

Co-authored-by: 晓雷 <[email protected]>

* [Refactor] Rename files in arealite before release. (#190)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* format

* format

* .

* rename

* rename

---------

Co-authored-by: 晓雷 <[email protected]>

* add quickstart (#194)

* [doc] [lite] Add customization docs for AReaLite. (#191)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* .

* .

* .

* .

* .

* .

* .

---------

Co-authored-by: 晓雷 <[email protected]>

* [lite] [feature] Bump to SGLang v0.4.9.post2 and use NCCL to update weights (#196)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* .

* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2

Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408

Reviewed-by: 晓雷 <[email protected]>


* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`

* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9

Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422

Reviewed-by: 晓雷 <[email protected]>


* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .

* PullRequest: 423 [lite] Remove the boba example for github release.

Merge branch fw/remove-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423

Reviewed-by: 晓雷 <[email protected]>


* .
* .

---------

Co-authored-by: 晓雷 <[email protected]>

* [Docs] [lite] Add example code walkthrough documentation. (#197)

* .

* .

* fix grammar and minor revise

* [lite] [doc] Add AReaLite design doc as README (#198)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* .

* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2

Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408

Reviewed-by: 晓雷 <[email protected]>


* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`

* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9

Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422

Reviewed-by: 晓雷 <[email protected]>


* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .

* PullRequest: 423 [lite] Remove the boba example for github release.

Merge branch fw/remove-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* update readme

* .

---------

Co-authored-by: 晓雷 <[email protected]>

* Add self-hosted runner support (#199)

* ci: add testsz for self hosted runner

* ci: use self-hosted runner

* ci: run ci on push

* ci: remove cd to tmp dir

* ci: fix

* ci: add environment back

* ci: set concurrency

* ci: fix build env image

* Update .github/workflows/test-arealite.yml

Co-authored-by: Copilot <[email protected]>

* Update ci/build_env_image.sh

Co-authored-by: Copilot <[email protected]>

* ci: add some test code

* ci: simplify

* ci: try to fix

* ci: test

* ci: test2

* ci: test3

* ci: test3

* ci: tidy up

* ci: simplify

* ci: do not trigger on push

* ci: test arealite/tests/

* ci: use UUID as run id

* ci: use openssl rand to genreate run id

---------

Co-authored-by: Copilot <[email protected]>

* [WIP][feat] Initial support for VLMs, add Qwen2VL SFT test and Qwen2.5VL GRPO test (#188)

* vlm_sft_test

* vlm_sft_test

* .

* .

* Fix unresolved issue in SFTTrainer PR (#139)

* .

* .

* efficient loading

* format

* .

* .

* Fix unresolved issue in SFTTrainer PR (#139)

* .

* .

* efficient loading

* format

* .

* .

* image_process0701

* image_process0701

* image_process0701_2

* image_process0701_2

* image_process0701_3

* image_process0701_3

* .

* .

* .

* .

* .

* .

* imageprocess0702

* imageprocess0702

* image_process0702_2

* image_process0702_2

* image_process0702_3

* image_process0702_3

* image_process0702_4

* image_process0702_4

* image_process0702_5

* image_process0702_5

* image_process0703_1

* image_process0703_1

* 0703_2

* 0703_2

* 0703_3

* 0703_3

* 0703_4

* 0703_4

* 0703_4

* 0703_4

* 0703_5

* 0703_5

* 0703_6

* 0703_6

* 0703_7

* 0703_7

* 0703_8

* 0703_8

* 0703_9

* 0703_9

* 0703_11

* 0703_11

* 0703_12

* 0703_12

* 0703_13

* 0703_13

* 0703_14

* 0703_14

* 0703_15

* 0703_15

* 0703_16

* 0703_16

* 0703-17

* 0703-17

* 0703_18

* 0703_18

* 0703_18

* 0703_18

* 0703_19

* 0703_19

* 0704_1

* 0704_1

* 0704_2

* 0704_2

* 0704_3

* 0704_3

* .

* .

* 0707_1

* 0707_1

* 0707_2

* 0707_2

* 0703_3

* 0703_3

* r

* p

* fix

* fix

* refactor

* 0707_6

* 0707_7

* refactor1

* 0707_undone

* 0708_1

* 0708_2

* 0708_3

* 0708_7

* 0708_4

* 0709_1

* 0709_2

* 0709_3

* 0709_4

* 0709_5

* 0709_

* 0709_6

* 0709_7

* 0709_7

* 0709_8

* 0709_9

* 0710_1

* 0710_2

* 0710_2

* 0710_3

* 0710_3

* 0710_3

* 0710_5

* 0710_4

* merge_2

* merge_3

* 0711_1

* 0711_2

* 0711_3

* 0711_4

* 0711_6

* 0711_7

* 0711_8

* 0711_8

* 0711_9

* 0711_10

* 0711-11

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* 0714_1

* 0714_2

* 0714_3

* 0714_3

* 0714_5

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* 0715_1

* 0715_2

* 0715_2

* 0716_1

* 0716_2

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* 0716_3

* 0716_4

* 0716_4

* 0716_5

* 0717_1

* 0717_3

* 0717_3

* 0717_4

* 0717_5

* 0717_6

* 0717_6

* 0717_6

* 0718_2

* 0718_4

* 0718_5

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* merge_0721

* 0721_1

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* 0721_2

* 0721_3

* 0721_4

* .

* 0721_formal

* 0721_formal

* 0721_merge4

* 0721_merge5

* 0721_6

* 0721_merge6

* 0721_merge7

* 0721_8

* 0722_1

* 0722_2

* 0722_3

* 0722_4

* 0722_4

* 0722_5

* 0722_6

* 0722_7

* 0723_1

* reformatted

* clang-reformatted

* clang-reformatted2

* 0723_1

* 0723_1

* 0723_1

* 0723_merge3

* 0723_4

* 0723_reformatted_5

* 0724_1

* 0724_1

* 0724_merge1

* 0724_merge2

* 0724_merge3

* 0724_merge3

* 0724_merge4

* 0724_merge5

* 0724_merge6

* 0724_merge7

* 0724_4

* 0724-merge8

* 0724_merge8

* 0725_1

* 0725_6

* 0725_7

* 0725_4padded_image

* 0725_9padded_image

* 0725_10padded_image

* 0725

* 0725_12

* 0725_format

---------

Co-authored-by: bowei.fw <[email protected]>
Co-authored-by: nuzant <[email protected]>
Co-authored-by: 朱晗 <[email protected]>

* [lite] [fix] Fix a performance issue and several minor issues before release (#203)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* .

* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2

Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408

Reviewed-by: 晓雷 <[email protected]>


* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`

* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9

Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422

Reviewed-by: 晓雷 <[email protected]>


* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .

* PullRequest: 423 [lite] Remove the boba example for github release.

Merge branch fw/remove-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* update readme

* PullRequest: 431 [Fix] Fix environment of lite

Merge branch fw/lite-dev of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/431

Reviewed-by: 晓雷 <[email protected]>


* change requirements
* .
* .
* .

* PullRequest: 440 [FIX] fix update weight from disk

Merge branch sxj/lite-fix-disk-update of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/440

Reviewed-by: 博惟 <[email protected]>


* [FIX] fix update weight from disk

* PullRequest: 442 [lite] Refactor `RemoteSGLangEngine` into two parts: `RemoteSGLangEngine` and `WorkflowExecutor`.

Merge branch mzy/workflow-executor of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/442

Reviewed-by: 博惟 <[email protected]>


* refactor workflow executor
* .
* fix tests and eval
* .
* .
* revert workflow executor into remote sglang engine
* .

* PullRequest: 456 [lite] [Bug] Use `ProcessPoolExecutor` to calculate reward to avoid rollout slow down

Merge branch mzy/lite/fix-reward of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/456?tab=comment

Reviewed-by: 博惟 <[email protected]>


* fix reward
* .
* .
* .

* PullRequest: 460 [lite][fix] add a warning when reward computation timeout

Merge branch fw/lite-fix of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/460

Reviewed-by: 晓雷 <[email protected]>


* add a warning when reward computation timeout

* PullRequest: 465 [lite][fix] Fix issues raised by tsao

Merge branch fw/lite-fix of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/465

Reviewed-by: 晓雷 <[email protected]>


* fix

---------

Co-authored-by: 晓雷 <[email protected]>
Co-authored-by: 冰临 <[email protected]>

* [lite] [refactor] Rename the `arealite` folder to `areal` (#205)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* .

* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2

Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408

Reviewed-by: 晓雷 <[email protected]>


* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`

* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9

Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422

Reviewed-by: 晓雷 <[email protected]>


* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .

* PullRequest: 423 [lite] Remove the boba example for github release.

Merge branch fw/remove-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* update readme

* PullRequest: 431 [Fix] Fix environment of lite

Merge branch fw/lite-dev of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/431

Reviewed-by: 晓雷 <[email protected]>


* change requirements
* .
* .
* .

* PullRequest: 440 [FIX] fix update weight from disk

Merge branch sxj/lite-fix-disk-update of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/440

Reviewed-by: 博惟 <[email protected]>


* [FIX] fix update weight from disk

* PullRequest: 442 [lite] Refactor `RemoteSGLangEngine` into two parts: `RemoteSGLangEngine` and `WorkflowExecutor`.

Merge branch mzy/workflow-executor of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/442

Reviewed-by: 博惟 <[email protected]>


* refactor workflow executor
* .
* fix tests and eval
* .
* .
* revert workflow executor into remote sglang engine
* .

* PullRequest: 456 [lite] [Bug] Use `ProcessPoolExecutor` to calculate reward to avoid rollout slow down

Merge branch mzy/lite/fix-reward of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/456?tab=comment

Reviewed-by: 博惟 <[email protected]>


* fix reward
* .
* .
* .

* PullRequest: 460 [lite][fix] add a warning when reward computation timeout

Merge branch fw/lite-fix of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/460

Reviewed-by: 晓雷 <[email protected]>


* add a warning when reward computation timeout

* PullRequest: 465 [lite][fix] Fix issues raised by tsao

Merge branch fw/lite-fix of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/465

Reviewed-by: 晓雷 <[email protected]>


* fix

* PullRequest: 463 [FEAT] add config converter

Merge branch sxj/lite-add-converter of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/463

Reviewed-by: 博惟 <[email protected]>


* [FEAT] add config converter
* [FIX] fix bugs
* [FIX] fix typos
* [FIX] fix cpu allocation
* [FIX] del vllm

* PullRequest: 466 [lite] Add seeding in training scripts and pad to max_tokens_per_mb to avoid memory fragmentation

Merge branch fw/lite-fix of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/466

Reviewed-by: 晓雷 <[email protected]>


* add seed
* .
* add option for pad to max len
* .

* PullRequest: 467 [lite][fix] remove logging functionality in stats_logger

Merge branch fw/lite-fix of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/467

Reviewed-by: 晓雷 <[email protected]>


* remove logging functionality in stats_logger
* .

* rename arealite to areal

* rename arealite to areal

* .

* local experiment run

* .

---------

Co-authored-by: 晓雷 <[email protected]>
Co-authored-by: 冰临 <[email protected]>

* [lite] [fix] Fix running the legacy experiment in the v0.4.9.post2 sglang environment (#207)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update we…
garrett4wade added a commit that referenced this pull request Aug 7, 2025
@antoinegg1 @garrett4wade
[WIP][BugFix] Fix forward bug of LLM and VLM (#218)
89b9935 
* 0707_1

* 0707_1

* 0707_2

* 0707_2

* 0703_3

* 0703_3

* r

* add api

* add directory structure

* add tests template

* p

* format

* fix

* fix

* checkout previous impl

* checkout previous implementations

* checkout prev impl

* refactor

* add remote sglang engine

* format

* 0707_6

* 0707_7

* add readme

* refactor1

* 0707_undone

* 0708_1

* 0708_2

* 0708_3

* 0708_7

* 0708_4

* 0709_1

* 0709_2

* 0709_3

* PullRequest: 331 [lite] Support remote sglang engine with corresponding testcases.

Merge branch fw/lite of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/331

Reviewed-by: 晓雷 <[email protected]>


* .
* add test for sglang remote engine
* fix

* 0709_4

* 0709_5

* 0709_

* 0709_6

* 0709_7

* PullRequest: 336 add wrapper

Merge branch lite-util-wrapper of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/336

Reviewed-by: 博惟 <[email protected]>


* add wrapper

* 0709_7

* 0709_8

* 0709_9

* PullRequest: 332 [lite] Support FSDP engines

Merge branch mzy/lite/engines of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/332

Reviewed-by: 晓雷 <[email protected]>


* fsdp2 engine
* fix utils
* add fsdp engine test
* .
* fsdp engine test passed
* unsqueeze immediately before model inputs and after model outputts
* add optimizer save/load, add position id calculation for input
* .
* format
* not to squeeze
* add train and eval api
* .
* .
* improve fsdp engine data preprocessing
* format
* PullRequest: 337 [lite] Add SFT trainer example.
* trainer log
* minor changes
* add update weights from disk
* fix type annotation

* PullRequest: 339 [Fix] Fix some minor issues to pass all tests.

Merge branch fw/lite of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/339

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .

* chore: empty commit

* ci: build images on demand

* ci: fix on demand condition

* ci: fix env sha

* PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods

Merge branch fw/lite-dev of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/340

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* fix

* 0710_1

* 0710_2

* [Fix] Merge previous contributions from fw/refactor to lite (#163)

* initial proposal

* add arealite

* .

* change api

* .

* remove LOG_ROOT

* remove MODEL_SAVE_PATH

* remove PARAM_REALLOC_PATH, DATASET_CACHE

* prepare for testing

* prepare for testing

* ready for run

* local run

* tests mainly pass

* format

* .

* amend cluster.py

* .

* .

* client test pass

* pass rollout test

* remove unused imports

* add arealite readme

* change api

* .

* .

* .

* .

* .

* .

* .

* .

* format

* .

* implement iteraptable generation (#112)

Co-authored-by: zhaochenyang <[email protected]>

* .

* fix

* .

* .

* .

* pass controller generate batch test

* .

* refactor rollout controller into worker and controller

* .

* .

* .

* change to async rollout

* pass rollout controller test

* pass test

* .

* update readme

* .

* sft debug

* .

* add lisence

* remove unused files

* remove unsed args in ppo

* add hf engine wrapper  (#116)

* add hf engine

* fix issues

* fix ppo bugs and add test

* add hf client interface and modify cli args

* fix bugs

* fix issues

* Merge fw/refactor

* Finish hf wrapper test

* add test

---------

Co-authored-by: Wei Fu <[email protected]>

* format

* format

* .

* refine hf engine

* .

* fix

* add fsdp engine and sft tests

* .

* .

* .

* pass ppo unittest

* pass ppo and rollout controller tests

* clear unused imports

* rename ppo to grpo

* change reward function organization

* reorganize code

* add dataset api

* .

* .

* .

* format

* chmod fix

* .

* rename workflow to collector

* refactor llm_client location

* .

* .

* fix llm server api

* refactor config structure

* .

* fix tests

* .

* .

* .

* Fix unresolved issue in SFTTrainer PR (#139)

* .

* .

* efficient loading

* format

* .

* .

* .

* .

* .

* .

* Add CI for testing AReaLite (#150)

* ci: add test-arealite

* ci: add checkout before running test-arealite

* ci: add USERNAME

* ci: add test script

* ci: add GitHub mirror

* ci: fix typo

* ci: clone one commit

* ci: fix condition

* ci: set command timeout to 60m

* ci: enable pip cache

* ci: optimize container lifecycle

* ci: split into many stages

* ci(test-arealite): fix typo

* ci: fix wrong env

* ci: fix pytest

* ci: uninstall transformer-engine

* ci: uninstall transformer-engine

* ci: fix model paths

* ci: show stdout/stderr

* ci: fix not clean up

* ci: backup sglang

* ci: remove tmp repo dir when run

* ci: fix docker run exit 1 condition

* ci(test-arealite): limit the concurrency and extend command timeout

* .

* merge fw/refactor

* revert some changes

* fix

---------

Co-authored-by: meizhiyu.mzy <[email protected]>
Co-authored-by: Chayenne <[email protected]>
Co-authored-by: zhaochenyang <[email protected]>
Co-authored-by: Jayon02 <[email protected]>
Co-authored-by: root <meizhiyu.mzy>
Co-authored-by: Zijian Zhang <[email protected]>

* 0710_2

* 0710_3

* [Doc] Add an instruction about how to run the SFT example. (#164)

* 0710_3

* 0710_3

* 0710_5

* 0710_4

* merge_2

* merge_3

* 0711_1

* 0711_2

* 0711_3

* 0711_4

* 0711_6

* 0711_7

* 0711_8

* 0711_8

* 0711_9

* 0711_10

* 0711-11

* [Fix] Fix CI running condition for lite. (#172)

* .

* fix

* .

* .

* fix

* fix

* fix

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* 0714_1

* 0714_2

* 0714_3

* 0714_3

* 0714_5

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* added LocalSGlangEngine and test (#170)

* added LocalSGLangEngine

* upload test file

* add build args

* fix sgl_local generate

* improved sgl local robustness

* test

* test updated

* added fallback when sgl engine isn't initialized

* finish test local engine

* added LocalSGlangEngine and test

* format and fix

format and fix, raise when generate missing field

format

* change cli_args.py

* add comment header

format

---------

Co-authored-by: ChangyiYang <[email protected]>

* format (#174)

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* 0715_1

* 0715_2

* 0715_2

* [Feat][Refactor]Support DeepSpeed AutoTP; Refactor hf_engine.py and unit test. (#161)

* refactor hf engine

* format file

* revert file format

* Squashed commit of the following:

commit 8d4b8dc90fceb144bf00f8125f938b57f696e166
Author: Wei Fu <[email protected]>
Date:   Thu Jul 10 13:14:10 2025 +0800

    [Doc] Add an instruction about how to run the SFT example. (#164)

commit 3bf9c85e400a2fe1c6a1e488d27d458ede8cea45
Author: Wei Fu <[email protected]>
Date:   Thu Jul 10 12:56:24 2025 +0800

    [Fix] Merge previous contributions from fw/refactor to lite (#163)

    * initial proposal

    * add arealite

    * .

    * change api

    * .

    * remove LOG_ROOT

    * remove MODEL_SAVE_PATH

    * remove PARAM_REALLOC_PATH, DATASET_CACHE

    * prepare for testing

    * prepare for testing

    * ready for run

    * local run

    * tests mainly pass

    * format

    * .

    * amend cluster.py

    * .

    * .

    * client test pass

    * pass rollout test

    * remove unused imports

    * add arealite readme

    * change api

    * .

    * .

    * .

    * .

    * .

    * .

    * .

    * .

    * format

    * .

    * implement iteraptable generation (#112)

    Co-authored-by: zhaochenyang <[email protected]>

    * .

    * fix

    * .

    * .

    * .

    * pass controller generate batch test

    * .

    * refactor rollout controller into worker and controller

    * .

    * .

    * .

    * change to async rollout

    * pass rollout controller test

    * pass test

    * .

    * update readme

    * .

    * sft debug

    * .

    * add lisence

    * remove unused files

    * remove unsed args in ppo

    * add hf engine wrapper  (#116)

    * add hf engine

    * fix issues

    * fix ppo bugs and add test

    * add hf client interface and modify cli args

    * fix bugs

    * fix issues

    * Merge fw/refactor

    * Finish hf wrapper test

    * add test

    ---------

    Co-authored-by: Wei Fu <[email protected]>

    * format

    * format

    * .

    * refine hf engine

    * .

    * fix

    * add fsdp engine and sft tests

    * .

    * .

    * .

    * pass ppo unittest

    * pass ppo and rollout controller tests

    * clear unused imports

    * rename ppo to grpo

    * change reward function organization

    * reorganize code

    * add dataset api

    * .

    * .

    * .

    * format

    * chmod fix

    * .

    * rename workflow to collector

    * refactor llm_client location

    * .

    * .

    * fix llm server api

    * refactor config structure

    * .

    * fix tests

    * .

    * .

    * .

    * Fix unresolved issue in SFTTrainer PR (#139)

    * .

    * .

    * efficient loading

    * format

    * .

    * .

    * .

    * .

    * .

    * .

    * Add CI for testing AReaLite (#150)

    * ci: add test-arealite

    * ci: add checkout before running test-arealite

    * ci: add USERNAME

    * ci: add test script

    * ci: add GitHub mirror

    * ci: fix typo

    * ci: clone one commit

    * ci: fix condition

    * ci: set command timeout to 60m

    * ci: enable pip cache

    * ci: optimize container lifecycle

    * ci: split into many stages

    * ci(test-arealite): fix typo

    * ci: fix wrong env

    * ci: fix pytest

    * ci: uninstall transformer-engine

    * ci: uninstall transformer-engine

    * ci: fix model paths

    * ci: show stdout/stderr

    * ci: fix not clean up

    * ci: backup sglang

    * ci: remove tmp repo dir when run

    * ci: fix docker run exit 1 condition

    * ci(test-arealite): limit the concurrency and extend command timeout

    * .

    * merge fw/refactor

    * revert some changes

    * fix

    ---------

    Co-authored-by: meizhiyu.mzy <[email protected]>
    Co-authored-by: Chayenne <[email protected]>
    Co-authored-by: zhaochenyang <[email protected]>
    Co-authored-by: Jayon02 <[email protected]>
    Co-authored-by: root <meizhiyu.mzy>
    Co-authored-by: Zijian Zhang <[email protected]>

commit d48bf007cfb0821c35a18a673f3c2e8f51a8ee45
Merge: 42c717b b9dbd4a
Author: 博惟 <[email protected]>
Date:   Thu Jul 10 12:53:30 2025 +0800

    Merge branch 'main' of https://github.com/inclusionAI/AReaL into lite

commit 42c717b6e45b3b345f29cff5882ba5374b79faee
Merge: c38cffc a203c7c
Author: 博惟 <[email protected]>
Date:   Thu Jul 10 11:15:01 2025 +0800

    Merge branch 'lite' of https://github.com/inclusionAI/AReaL into lite

commit c38cffc02385acf382196ba6b720b2d8b91a5675
Author: 博惟 <[email protected]>
Date:   Thu Jul 10 11:10:10 2025 +0800

    PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods

    Merge branch fw/lite-dev of [email protected]:inclusionAI/AReaL.git into lite
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/340

    Reviewed-by: 晓雷 <[email protected]>

    * support fsdp engine and sglang remote engine
    * minor fix
    * .
    * refactor trainer
    * add close
    * rm mb_spec
    * fix

commit b9dbd4a2c18539648dca373c9c8c1347e4e9db68
Author: Wei Fu <[email protected]>
Date:   Wed Jul 9 10:50:19 2025 +0800

    Update to persistent wechat QR code. (#159)

commit 17ea7fe94dc0bd9299082a069383d7372ed13ac9
Author: xssstory <[email protected]>
Date:   Mon Jul 7 15:49:13 2025 +0800

    fix math reward verifier (#156)

    * PullRequest: 293 fix get_param_realloc_path

    Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/293

    Reviewed-by: 博惟 <[email protected]>

    * fix get_param_realloc_path

    * PullRequest: 297 bugfix: reward is always -5

    Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/297

    Reviewed-by: 博惟 <[email protected]>

    * bugfix: reward is always -5

    * PullRequest: 321 fix checkpoint save dir

    Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/321

    Reviewed-by: 博惟 <[email protected]>

    * fix checkpoint save dir

    * PullRequest: 328 [Doc] update installation

    Merge branch sxj/doc of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/328

    Reviewed-by: 博惟 <[email protected]>

    * [Doc] update installation

    * PullRequest: 329 bugfix: math verifier blocks the async training

    Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/329

    Reviewed-by: 博惟 <[email protected]>

    * bugfix: math verifier block the async training

    * format

    ---------

    Co-authored-by: 冰临 <[email protected]>
    Co-authored-by: garrett4wade <[email protected]>

* add autotp for hf

* refactor test

* fix bugs

* fix issues

* format files

* Squashed commit of the following:

commit 9ed043f6ab83cf9c9b38e0251ad08009a73243e8
Author: Wei Fu <[email protected]>
Date:   Tue Jul 15 10:24:48 2025 +0800

    format (#174)

commit 8cc9b1feb5ed03cfa42f3ed588dd5f6ec16e5e55
Author: Night <[email protected]>
Date:   Mon Jul 14 19:22:00 2025 -0700

    added LocalSGlangEngine and test (#170)

    * added LocalSGLangEngine

    * upload test file

    * add build args

    * fix sgl_local generate

    * improved sgl local robustness

    * test

    * test updated

    * added fallback when sgl engine isn't initialized

    * finish test local engine

    * added LocalSGlangEngine and test

    * format and fix

    format and fix, raise when generate missing field

    format

    * change cli_args.py

    * add comment header

    format

    ---------

    Co-authored-by: ChangyiYang <[email protected]>

---------

Co-authored-by: Jayon02 <[email protected]>
Co-authored-by: Wei Fu <[email protected]>

* fix ci (#175)

* [Feature] Add pre-commit (#178)

* fix ci

* .

* .

* .

* [lite] [refactor] Add GSM8k GRPO example. (#179)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* fix ci

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

---------

Co-authored-by: 晓雷 <[email protected]>

* 0716_1

* 0716_2

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* change doc (#180)

* [Fix] [lite] Merge from the internal repo to fix GRPO bugs and refactor the train engine (#181)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

---------

Co-authored-by: 晓雷 <[email protected]>

* 0716_3

* 0716_4

* 0716_4

* 0716_5

* 0717_1

* 0717_3

* 0717_3

* 0717_4

* 0717_5

* 0717_6

* 0717_6

* 0717_6

* 0718_2

* 0718_4

* 0718_5

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* merge_0721

* 0721_1

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* 0721_2

* 0721_3

* Implement fsdp distributed update (#183)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* added remote nccl weight update

feat: implement update_weights_from_distributed in fsdp_engine.py

unfinishd test, raise PR first

coroutine for each server

chore: change uploads weights behavior, change test order

fix small bug

fixed test

* fix rebase

* add test.sh

* updated, test stil fails

* .

* .

* .

* fix: full_tensor() should happen in all rank (#187)

Co-authored-by: ChangyiYang <[email protected]>

---------

Co-authored-by: 晓雷 <[email protected]>
Co-authored-by: 博惟 <[email protected]>
Co-authored-by: ChangyiYang <[email protected]>
Co-authored-by: ChangyiYang <[email protected]>

* [Feature] [lite] Merge from internal dev repo (#189)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* format

* format

* .

---------

Co-authored-by: 晓雷 <[email protected]>

* [Refactor] Rename files in arealite before release. (#190)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* format

* format

* .

* rename

* rename

---------

Co-authored-by: 晓雷 <[email protected]>

* 0721_4

* .

* 0721_formal

* 0721_formal

* 0721_merge4

* 0721_merge5

* 0721_6

* 0721_merge6

* 0721_merge7

* 0721_8

* 0722_1

* 0722_2

* 0722_3

* add quickstart (#194)

* [doc] [lite] Add customization docs for AReaLite. (#191)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* .

* .

* .

* .

* .

* .

* .

---------

Co-authored-by: 晓雷 <[email protected]>

* 0722_4

* 0722_4

* 0722_5

* 0722_6

* 0722_7

* 0723_1

* reformatted

* clang-reformatted

* clang-reformatted2

* 0723_1

* 0723_1

* 0723_1

* 0723_merge3

* 0723_4

* 0723_reformatted_5

* 0724_1

* 0724_1

* 0724_merge1

* 0724_merge2

* 0724_merge3

* 0724_merge3

* 0724_merge4

* [lite] [feature] Bump to SGLang v0.4.9.post2 and use NCCL to update weights (#196)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* .

* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2

Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408

Reviewed-by: 晓雷 <[email protected]>


* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`

* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9

Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422

Reviewed-by: 晓雷 <[email protected]>


* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .

* PullRequest: 423 [lite] Remove the boba example for github release.

Merge branch fw/remove-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423

Reviewed-by: 晓雷 <[email protected]>


* .
* .

---------

Co-authored-by: 晓雷 <[email protected]>

* 0724_merge5

* 0724_merge6

* 0724_merge7

* [Docs] [lite] Add example code walkthrough documentation. (#197)

* .

* .

* fix grammar and minor revise

* [lite] [doc] Add AReaLite design doc as README (#198)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* .

* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2

Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408

Reviewed-by: 晓雷 <[email protected]>


* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`

* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9

Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422

Reviewed-by: 晓雷 <[email protected]>


* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .

* PullRequest: 423 [lite] Remove the boba example for github release.

Merge branch fw/remove-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* update readme

* .

---------

Co-authored-by: 晓雷 <[email protected]>

* 0724_4

* 0724-merge8

* 0724_merge8

* 0725_1

* 0725_6

* 0725_7

* 0725_4padded_image

* 0725_9padded_image

* 0725_10padded_image

* 0725

* 0725_12

* 0725_format

* Add self-hosted runner support (#199)

* ci: add testsz for self hosted runner

* ci: use self-hosted runner

* ci: run ci on push

* ci: remove cd to tmp dir

* ci: fix

* ci: add environment back

* ci: set concurrency

* ci: fix build env image

* Update .github/workflows/test-arealite.yml

Co-authored-by: Copilot <[email protected]>

* Update ci/build_env_image.sh

Co-authored-by: Copilot <[email protected]>

* ci: add some test code

* ci: simplify

* ci: try to fix

* ci: test

* ci: test2

* ci: test3

* ci: test3

* ci: tidy up

* ci: simplify

* ci: do not trigger on push

* ci: test arealite/tests/

* ci: use UUID as run id

* ci: use openssl rand to genreate run id

---------

Co-authored-by: Copilot <[email protected]>

* [WIP][feat] Initial support for VLMs, add Qwen2VL SFT test and Qwen2.5VL GRPO test (#188)

* vlm_sft_test

* vlm_sft_test

* .

* .

* Fix unresolved issue in SFTTrainer PR (#139)

* .

* .

* efficient loading

* format

* .

* .

* Fix unresolved issue in SFTTrainer PR (#139)

* .

* .

* efficient loading

* format

* .

* .

* image_process0701

* image_process0701

* image_process0701_2

* image_process0701_2

* image_process0701_3

* image_process0701_3

* .

* .

* .

* .

* .

* .

* imageprocess0702

* imageprocess0702

* image_process0702_2

* image_process0702_2

* image_process0702_3

* image_process0702_3

* image_process0702_4

* image_process0702_4

* image_process0702_5

* image_process0702_5

* image_process0703_1

* image_process0703_1

* 0703_2

* 0703_2

* 0703_3

* 0703_3

* 0703_4

* 0703_4

* 0703_4

* 0703_4

* 0703_5

* 0703_5

* 0703_6

* 0703_6

* 0703_7

* 0703_7

* 0703_8

* 0703_8

* 0703_9

* 0703_9

* 0703_11

* 0703_11

* 0703_12

* 0703_12

* 0703_13

* 0703_13

* 0703_14

* 0703_14

* 0703_15

* 0703_15

* 0703_16

* 0703_16

* 0703-17

* 0703-17

* 0703_18

* 0703_18

* 0703_18

* 0703_18

* 0703_19

* 0703_19

* 0704_1

* 0704_1

* 0704_2

* 0704_2

* 0704_3

* 0704_3

* .

* .

* 0707_1

* 0707_1

* 0707_2

* 0707_2

* 0703_3

* 0703_3

* r

* p

* fix

* fix

* refactor

* 0707_6

* 0707_7

* refactor1

* 0707_undone

* 0708_1

* 0708_2

* 0708_3

* 0708_7

* 0708_4

* 0709_1

* 0709_2

* 0709_3

* 0709_4

* 0709_5

* 0709_

* 0709_6

* 0709_7

* 0709_7

* 0709_8

* 0709_9

* 0710_1

* 0710_2

* 0710_2

* 0710_3

* 0710_3

* 0710_3

* 0710_5

* 0710_4

* merge_2

* merge_3

* 0711_1

* 0711_2

* 0711_3

* 0711_4

* 0711_6

* 0711_7

* 0711_8

* 0711_8

* 0711_9

* 0711_10

* 0711-11

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* 0714_1

* 0714_2

* 0714_3

* 0714_3

* 0714_5

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* 0715_1

* 0715_2

* 0715_2

* 0716_1

* 0716_2

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* 0716_3

* 0716_4

* 0716_4

* 0716_5

* 0717_1

* 0717_3

* 0717_3

* 0717_4

* 0717_5

* 0717_6

* 0717_6

* 0717_6

* 0718_2

* 0718_4

* 0718_5

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* merge_0721

* 0721_1

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* 0721_2

* 0721_3

* 0721_4

* .

* 0721_formal

* 0721_formal

* 0721_merge4

* 0721_merge5

* 0721_6

* 0721_merge6

* 0721_merge7

* 0721_8

* 0722_1

* 0722_2

* 0722_3

* 0722_4

* 0722_4

* 0722_5

* 0722_6

* 0722_7

* 0723_1

* reformatted

* clang-reformatted

* clang-reformatted2

* 0723_1

* 0723_1

* 0723_1

* 0723_merge3

* 0723_4

* 0723_reformatted_5

* 0724_1

* 0724_1

* 0724_merge1

* 0724_merge2

* 0724_merge3

* 0724_merge3

* 0724_merge4

* 0724_merge5

* 0724_merge6

* 0724_merge7

* 0724_4

* 0724-merge8

* 0724_merge8

* 0725_1

* 0725_6

* 0725_7

* 0725_4padded_image

* 0725_9padded_image

* 0725_10padded_image

* 0725

* 0725_12

* 0725_format

---------

Co-authored-by: bowei.fw <[email protected]>
Co-authored-by: nuzant <[email protected]>
Co-authored-by: 朱晗 <[email protected]>

* 0731

* 0731_1

* 0731_2

* 0731_2

* 0731_3

* 0731_4

* 0801_1

* 0801_2

* 0804_1

* 0804_2

* 0804_2

* 0804_5

* 0805_3

* 0805_2

* 0806

* 0806_merge1

* 0806_merge2

* 0806_format1

* 0806_merge3

* 0806_4

* 0806_6

* 0806_7

* 0806_formatted2

* fix

* revert examples

* .

* .

---------

Co-authored-by: 朱晗 <[email protected]>
Co-authored-by: 博惟 <[email protected]>
Co-authored-by: garrett4wade <[email protected]>
Co-authored-by: Wei Fu <[email protected]>
mjbmjb pushed a commit to mjbmjb/AReaL that referenced this pull request Sep 22, 2025
@xssstory @fishcrap @garrett4wade
fix math reward verifier (inclusionAI#156)
80e1637 
* PullRequest: 293 fix get_param_realloc_path

Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/293

Reviewed-by: 博惟 <[email protected]>


* fix get_param_realloc_path

* PullRequest: 297 bugfix: reward is always -5

Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/297

Reviewed-by: 博惟 <[email protected]>


* bugfix: reward is always -5

* PullRequest: 321 fix checkpoint save dir

Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/321

Reviewed-by: 博惟 <[email protected]>


* fix checkpoint save dir

* PullRequest: 328 [Doc] update installation

Merge branch sxj/doc of [email protected]:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/328

Reviewed-by: 博惟 <[email protected]>


* [Doc] update installation

* PullRequest: 329 bugfix: math verifier blocks the async training

Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/329

Reviewed-by: 博惟 <[email protected]>


* bugfix: math verifier block the async training

* format

---------

Co-authored-by: 冰临 <[email protected]>
Co-authored-by: garrett4wade <[email protected]>
mjbmjb pushed a commit to mjbmjb/AReaL that referenced this pull request Sep 22, 2025
@garrett4wade @xssstory
@kdada @futrime @nuzant @zhaochenyang20 @zhaochen20 @Jayon02 @PrinsYin @ChangyiYang @antoinegg1 @fishcrap @richardbsk
Refactor stage inclusionAI#1: AReaL-lite (inclusionAI#154)
4270b62 
* PullRequest: 293 fix get_param_realloc_path

Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/293

Reviewed-by: 博惟 <[email protected]>


* fix get_param_realloc_path

* PullRequest: 297 bugfix: reward is always -5

Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/297

Reviewed-by: 博惟 <[email protected]>


* bugfix: reward is always -5

* PullRequest: 321 fix checkpoint save dir

Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/321

Reviewed-by: 博惟 <[email protected]>


* fix checkpoint save dir

* add api

* add directory structure

* add tests template

* format

* checkout previous impl

* checkout previous implementations

* checkout prev impl

* add remote sglang engine

* format

* add readme

* PullRequest: 331 [lite] Support remote sglang engine with corresponding testcases.

Merge branch fw/lite of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/331

Reviewed-by: 晓雷 <[email protected]>


* .
* add test for sglang remote engine
* fix

* PullRequest: 336 add wrapper

Merge branch lite-util-wrapper of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/336

Reviewed-by: 博惟 <[email protected]>


* add wrapper

* PullRequest: 332 [lite] Support FSDP engines

Merge branch mzy/lite/engines of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/332

Reviewed-by: 晓雷 <[email protected]>


* fsdp2 engine
* fix utils
* add fsdp engine test
* .
* fsdp engine test passed
* unsqueeze immediately before model inputs and after model outputts
* add optimizer save/load, add position id calculation for input
* .
* format
* not to squeeze
* add train and eval api
* .
* .
* improve fsdp engine data preprocessing
* format
* PullRequest: 337 [lite] Add SFT trainer example.
* trainer log
* minor changes
* add update weights from disk
* fix type annotation

* PullRequest: 339 [Fix] Fix some minor issues to pass all tests.

Merge branch fw/lite of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/339

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .

* chore: empty commit

* ci: build images on demand

* ci: fix on demand condition

* ci: fix env sha

* PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods

Merge branch fw/lite-dev of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/340

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* fix

* [Fix] Merge previous contributions from fw/refactor to lite (#163)

* initial proposal

* add arealite

* .

* change api

* .

* remove LOG_ROOT

* remove MODEL_SAVE_PATH

* remove PARAM_REALLOC_PATH, DATASET_CACHE

* prepare for testing

* prepare for testing

* ready for run

* local run

* tests mainly pass

* format

* .

* amend cluster.py

* .

* .

* client test pass

* pass rollout test

* remove unused imports

* add arealite readme

* change api

* .

* .

* .

* .

* .

* .

* .

* .

* format

* .

* implement iteraptable generation (#112)

Co-authored-by: zhaochenyang <[email protected]>

* .

* fix

* .

* .

* .

* pass controller generate batch test

* .

* refactor rollout controller into worker and controller

* .

* .

* .

* change to async rollout

* pass rollout controller test

* pass test

* .

* update readme

* .

* sft debug

* .

* add lisence

* remove unused files

* remove unsed args in ppo

* add hf engine wrapper  (#116)

* add hf engine

* fix issues

* fix ppo bugs and add test

* add hf client interface and modify cli args

* fix bugs

* fix issues

* Merge fw/refactor

* Finish hf wrapper test

* add test

---------

Co-authored-by: Wei Fu <[email protected]>

* format

* format

* .

* refine hf engine

* .

* fix

* add fsdp engine and sft tests

* .

* .

* .

* pass ppo unittest

* pass ppo and rollout controller tests

* clear unused imports

* rename ppo to grpo

* change reward function organization

* reorganize code

* add dataset api

* .

* .

* .

* format

* chmod fix

* .

* rename workflow to collector

* refactor llm_client location

* .

* .

* fix llm server api

* refactor config structure

* .

* fix tests

* .

* .

* .

* Fix unresolved issue in SFTTrainer PR (#139)

* .

* .

* efficient loading

* format

* .

* .

* .

* .

* .

* .

* Add CI for testing AReaLite (#150)

* ci: add test-arealite

* ci: add checkout before running test-arealite

* ci: add USERNAME

* ci: add test script

* ci: add GitHub mirror

* ci: fix typo

* ci: clone one commit

* ci: fix condition

* ci: set command timeout to 60m

* ci: enable pip cache

* ci: optimize container lifecycle

* ci: split into many stages

* ci(test-arealite): fix typo

* ci: fix wrong env

* ci: fix pytest

* ci: uninstall transformer-engine

* ci: uninstall transformer-engine

* ci: fix model paths

* ci: show stdout/stderr

* ci: fix not clean up

* ci: backup sglang

* ci: remove tmp repo dir when run

* ci: fix docker run exit 1 condition

* ci(test-arealite): limit the concurrency and extend command timeout

* .

* merge fw/refactor

* revert some changes

* fix

---------

Co-authored-by: meizhiyu.mzy <[email protected]>
Co-authored-by: Chayenne <[email protected]>
Co-authored-by: zhaochenyang <[email protected]>
Co-authored-by: Jayon02 <[email protected]>
Co-authored-by: root <meizhiyu.mzy>
Co-authored-by: Zijian Zhang <[email protected]>

* [Doc] Add an instruction about how to run the SFT example. (#164)

* [Fix] Fix CI running condition for lite. (#172)

* .

* fix

* .

* .

* fix

* fix

* fix

* added LocalSGlangEngine and test (#170)

* added LocalSGLangEngine

* upload test file

* add build args

* fix sgl_local generate

* improved sgl local robustness

* test

* test updated

* added fallback when sgl engine isn't initialized

* finish test local engine

* added LocalSGlangEngine and test

* format and fix

format and fix, raise when generate missing field

format

* change cli_args.py

* add comment header

format

---------

Co-authored-by: ChangyiYang <[email protected]>

* format (#174)

* [Feat][Refactor]Support DeepSpeed AutoTP; Refactor hf_engine.py and unit test. (#161)

* refactor hf engine

* format file

* revert file format

* Squashed commit of the following:

commit 8d4b8dc90fceb144bf00f8125f938b57f696e166
Author: Wei Fu <[email protected]>
Date:   Thu Jul 10 13:14:10 2025 +0800

    [Doc] Add an instruction about how to run the SFT example. (#164)

commit 3bf9c85e400a2fe1c6a1e488d27d458ede8cea45
Author: Wei Fu <[email protected]>
Date:   Thu Jul 10 12:56:24 2025 +0800

    [Fix] Merge previous contributions from fw/refactor to lite (#163)

    * initial proposal

    * add arealite

    * .

    * change api

    * .

    * remove LOG_ROOT

    * remove MODEL_SAVE_PATH

    * remove PARAM_REALLOC_PATH, DATASET_CACHE

    * prepare for testing

    * prepare for testing

    * ready for run

    * local run

    * tests mainly pass

    * format

    * .

    * amend cluster.py

    * .

    * .

    * client test pass

    * pass rollout test

    * remove unused imports

    * add arealite readme

    * change api

    * .

    * .

    * .

    * .

    * .

    * .

    * .

    * .

    * format

    * .

    * implement iteraptable generation (#112)

    Co-authored-by: zhaochenyang <[email protected]>

    * .

    * fix

    * .

    * .

    * .

    * pass controller generate batch test

    * .

    * refactor rollout controller into worker and controller

    * .

    * .

    * .

    * change to async rollout

    * pass rollout controller test

    * pass test

    * .

    * update readme

    * .

    * sft debug

    * .

    * add lisence

    * remove unused files

    * remove unsed args in ppo

    * add hf engine wrapper  (#116)

    * add hf engine

    * fix issues

    * fix ppo bugs and add test

    * add hf client interface and modify cli args

    * fix bugs

    * fix issues

    * Merge fw/refactor

    * Finish hf wrapper test

    * add test

    ---------

    Co-authored-by: Wei Fu <[email protected]>

    * format

    * format

    * .

    * refine hf engine

    * .

    * fix

    * add fsdp engine and sft tests

    * .

    * .

    * .

    * pass ppo unittest

    * pass ppo and rollout controller tests

    * clear unused imports

    * rename ppo to grpo

    * change reward function organization

    * reorganize code

    * add dataset api

    * .

    * .

    * .

    * format

    * chmod fix

    * .

    * rename workflow to collector

    * refactor llm_client location

    * .

    * .

    * fix llm server api

    * refactor config structure

    * .

    * fix tests

    * .

    * .

    * .

    * Fix unresolved issue in SFTTrainer PR (#139)

    * .

    * .

    * efficient loading

    * format

    * .

    * .

    * .

    * .

    * .

    * .

    * Add CI for testing AReaLite (#150)

    * ci: add test-arealite

    * ci: add checkout before running test-arealite

    * ci: add USERNAME

    * ci: add test script

    * ci: add GitHub mirror

    * ci: fix typo

    * ci: clone one commit

    * ci: fix condition

    * ci: set command timeout to 60m

    * ci: enable pip cache

    * ci: optimize container lifecycle

    * ci: split into many stages

    * ci(test-arealite): fix typo

    * ci: fix wrong env

    * ci: fix pytest

    * ci: uninstall transformer-engine

    * ci: uninstall transformer-engine

    * ci: fix model paths

    * ci: show stdout/stderr

    * ci: fix not clean up

    * ci: backup sglang

    * ci: remove tmp repo dir when run

    * ci: fix docker run exit 1 condition

    * ci(test-arealite): limit the concurrency and extend command timeout

    * .

    * merge fw/refactor

    * revert some changes

    * fix

    ---------

    Co-authored-by: meizhiyu.mzy <[email protected]>
    Co-authored-by: Chayenne <[email protected]>
    Co-authored-by: zhaochenyang <[email protected]>
    Co-authored-by: Jayon02 <[email protected]>
    Co-authored-by: root <meizhiyu.mzy>
    Co-authored-by: Zijian Zhang <[email protected]>

commit d48bf007cfb0821c35a18a673f3c2e8f51a8ee45
Merge: 42c717b 743896d
Author: 博惟 <[email protected]>
Date:   Thu Jul 10 12:53:30 2025 +0800

    Merge branch 'main' of https://github.com/inclusionAI/AReaL into lite

commit 42c717b6e45b3b345f29cff5882ba5374b79faee
Merge: c38cffc a203c7c
Author: 博惟 <[email protected]>
Date:   Thu Jul 10 11:15:01 2025 +0800

    Merge branch 'lite' of https://github.com/inclusionAI/AReaL into lite

commit c38cffc02385acf382196ba6b720b2d8b91a5675
Author: 博惟 <[email protected]>
Date:   Thu Jul 10 11:10:10 2025 +0800

    PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods

    Merge branch fw/lite-dev of [email protected]:inclusionAI/AReaL.git into lite
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/340

    Reviewed-by: 晓雷 <[email protected]>

    * support fsdp engine and sglang remote engine
    * minor fix
    * .
    * refactor trainer
    * add close
    * rm mb_spec
    * fix

commit 743896dbd3938fb879dc67cb9990462236778193
Author: Wei Fu <[email protected]>
Date:   Wed Jul 9 10:50:19 2025 +0800

    Update to persistent wechat QR code. (#159)

commit 80e16378237dd89878bdf79e5992932550df9493
Author: xssstory <[email protected]>
Date:   Mon Jul 7 15:49:13 2025 +0800

    fix math reward verifier (#156)

    * PullRequest: 293 fix get_param_realloc_path

    Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/293

    Reviewed-by: 博惟 <[email protected]>

    * fix get_param_realloc_path

    * PullRequest: 297 bugfix: reward is always -5

    Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/297

    Reviewed-by: 博惟 <[email protected]>

    * bugfix: reward is always -5

    * PullRequest: 321 fix checkpoint save dir

    Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/321

    Reviewed-by: 博惟 <[email protected]>

    * fix checkpoint save dir

    * PullRequest: 328 [Doc] update installation

    Merge branch sxj/doc of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/328

    Reviewed-by: 博惟 <[email protected]>

    * [Doc] update installation

    * PullRequest: 329 bugfix: math verifier blocks the async training

    Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/329

    Reviewed-by: 博惟 <[email protected]>

    * bugfix: math verifier block the async training

    * format

    ---------

    Co-authored-by: 冰临 <[email protected]>
    Co-authored-by: garrett4wade <[email protected]>

* add autotp for hf

* refactor test

* fix bugs

* fix issues

* format files

* Squashed commit of the following:

commit 9ed043f6ab83cf9c9b38e0251ad08009a73243e8
Author: Wei Fu <[email protected]>
Date:   Tue Jul 15 10:24:48 2025 +0800

    format (#174)

commit 8cc9b1feb5ed03cfa42f3ed588dd5f6ec16e5e55
Author: Night <[email protected]>
Date:   Mon Jul 14 19:22:00 2025 -0700

    added LocalSGlangEngine and test (#170)

    * added LocalSGLangEngine

    * upload test file

    * add build args

    * fix sgl_local generate

    * improved sgl local robustness

    * test

    * test updated

    * added fallback when sgl engine isn't initialized

    * finish test local engine

    * added LocalSGlangEngine and test

    * format and fix

    format and fix, raise when generate missing field

    format

    * change cli_args.py

    * add comment header

    format

    ---------

    Co-authored-by: ChangyiYang <[email protected]>

---------

Co-authored-by: Jayon02 <[email protected]>
Co-authored-by: Wei Fu <[email protected]>

* fix ci (#175)

* [Feature] Add pre-commit (#178)

* fix ci

* .

* .

* .

* [lite] [refactor] Add GSM8k GRPO example. (#179)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* fix ci

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

---------

Co-authored-by: 晓雷 <[email protected]>

* change doc (#180)

* [Fix] [lite] Merge from the internal repo to fix GRPO bugs and refactor the train engine (#181)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

---------

Co-authored-by: 晓雷 <[email protected]>

* Implement fsdp distributed update (#183)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* added remote nccl weight update

feat: implement update_weights_from_distributed in fsdp_engine.py

unfinishd test, raise PR first

coroutine for each server

chore: change uploads weights behavior, change test order

fix small bug

fixed test

* fix rebase

* add test.sh

* updated, test stil fails

* .

* .

* .

* fix: full_tensor() should happen in all rank (#187)

Co-authored-by: ChangyiYang <[email protected]>

---------

Co-authored-by: 晓雷 <[email protected]>
Co-authored-by: 博惟 <[email protected]>
Co-authored-by: ChangyiYang <[email protected]>
Co-authored-by: ChangyiYang <[email protected]>

* [Feature] [lite] Merge from internal dev repo (#189)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* format

* format

* .

---------

Co-authored-by: 晓雷 <[email protected]>

* [Refactor] Rename files in arealite before release. (#190)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* format

* format

* .

* rename

* rename

---------

Co-authored-by: 晓雷 <[email protected]>

* add quickstart (#194)

* [doc] [lite] Add customization docs for AReaLite. (#191)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* .

* .

* .

* .

* .

* .

* .

---------

Co-authored-by: 晓雷 <[email protected]>

* [lite] [feature] Bump to SGLang v0.4.9.post2 and use NCCL to update weights (#196)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* .

* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2

Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408

Reviewed-by: 晓雷 <[email protected]>


* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`

* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9

Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422

Reviewed-by: 晓雷 <[email protected]>


* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .

* PullRequest: 423 [lite] Remove the boba example for github release.

Merge branch fw/remove-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423

Reviewed-by: 晓雷 <[email protected]>


* .
* .

---------

Co-authored-by: 晓雷 <[email protected]>

* [Docs] [lite] Add example code walkthrough documentation. (#197)

* .

* .

* fix grammar and minor revise

* [lite] [doc] Add AReaLite design doc as README (#198)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* .

* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2

Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408

Reviewed-by: 晓雷 <[email protected]>


* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`

* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9

Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422

Reviewed-by: 晓雷 <[email protected]>


* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .

* PullRequest: 423 [lite] Remove the boba example for github release.

Merge branch fw/remove-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* update readme

* .

---------

Co-authored-by: 晓雷 <[email protected]>

* Add self-hosted runner support (#199)

* ci: add testsz for self hosted runner

* ci: use self-hosted runner

* ci: run ci on push

* ci: remove cd to tmp dir

* ci: fix

* ci: add environment back

* ci: set concurrency

* ci: fix build env image

* Update .github/workflows/test-arealite.yml

Co-authored-by: Copilot <[email protected]>

* Update ci/build_env_image.sh

Co-authored-by: Copilot <[email protected]>

* ci: add some test code

* ci: simplify

* ci: try to fix

* ci: test

* ci: test2

* ci: test3

* ci: test3

* ci: tidy up

* ci: simplify

* ci: do not trigger on push

* ci: test arealite/tests/

* ci: use UUID as run id

* ci: use openssl rand to genreate run id

---------

Co-authored-by: Copilot <[email protected]>

* [WIP][feat] Initial support for VLMs, add Qwen2VL SFT test and Qwen2.5VL GRPO test (#188)

* vlm_sft_test

* vlm_sft_test

* .

* .

* Fix unresolved issue in SFTTrainer PR (#139)

* .

* .

* efficient loading

* format

* .

* .

* Fix unresolved issue in SFTTrainer PR (#139)

* .

* .

* efficient loading

* format

* .

* .

* image_process0701

* image_process0701

* image_process0701_2

* image_process0701_2

* image_process0701_3

* image_process0701_3

* .

* .

* .

* .

* .

* .

* imageprocess0702

* imageprocess0702

* image_process0702_2

* image_process0702_2

* image_process0702_3

* image_process0702_3

* image_process0702_4

* image_process0702_4

* image_process0702_5

* image_process0702_5

* image_process0703_1

* image_process0703_1

* 0703_2

* 0703_2

* 0703_3

* 0703_3

* 0703_4

* 0703_4

* 0703_4

* 0703_4

* 0703_5

* 0703_5

* 0703_6

* 0703_6

* 0703_7

* 0703_7

* 0703_8

* 0703_8

* 0703_9

* 0703_9

* 0703_11

* 0703_11

* 0703_12

* 0703_12

* 0703_13

* 0703_13

* 0703_14

* 0703_14

* 0703_15

* 0703_15

* 0703_16

* 0703_16

* 0703-17

* 0703-17

* 0703_18

* 0703_18

* 0703_18

* 0703_18

* 0703_19

* 0703_19

* 0704_1

* 0704_1

* 0704_2

* 0704_2

* 0704_3

* 0704_3

* .

* .

* 0707_1

* 0707_1

* 0707_2

* 0707_2

* 0703_3

* 0703_3

* r

* p

* fix

* fix

* refactor

* 0707_6

* 0707_7

* refactor1

* 0707_undone

* 0708_1

* 0708_2

* 0708_3

* 0708_7

* 0708_4

* 0709_1

* 0709_2

* 0709_3

* 0709_4

* 0709_5

* 0709_

* 0709_6

* 0709_7

* 0709_7

* 0709_8

* 0709_9

* 0710_1

* 0710_2

* 0710_2

* 0710_3

* 0710_3

* 0710_3

* 0710_5

* 0710_4

* merge_2

* merge_3

* 0711_1

* 0711_2

* 0711_3

* 0711_4

* 0711_6

* 0711_7

* 0711_8

* 0711_8

* 0711_9

* 0711_10

* 0711-11

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* 0714_1

* 0714_2

* 0714_3

* 0714_3

* 0714_5

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* 0715_1

* 0715_2

* 0715_2

* 0716_1

* 0716_2

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* 0716_3

* 0716_4

* 0716_4

* 0716_5

* 0717_1

* 0717_3

* 0717_3

* 0717_4

* 0717_5

* 0717_6

* 0717_6

* 0717_6

* 0718_2

* 0718_4

* 0718_5

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* merge_0721

* 0721_1

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* 0721_2

* 0721_3

* 0721_4

* .

* 0721_formal

* 0721_formal

* 0721_merge4

* 0721_merge5

* 0721_6

* 0721_merge6

* 0721_merge7

* 0721_8

* 0722_1

* 0722_2

* 0722_3

* 0722_4

* 0722_4

* 0722_5

* 0722_6

* 0722_7

* 0723_1

* reformatted

* clang-reformatted

* clang-reformatted2

* 0723_1

* 0723_1

* 0723_1

* 0723_merge3

* 0723_4

* 0723_reformatted_5

* 0724_1

* 0724_1

* 0724_merge1

* 0724_merge2

* 0724_merge3

* 0724_merge3

* 0724_merge4

* 0724_merge5

* 0724_merge6

* 0724_merge7

* 0724_4

* 0724-merge8

* 0724_merge8

* 0725_1

* 0725_6

* 0725_7

* 0725_4padded_image

* 0725_9padded_image

* 0725_10padded_image

* 0725

* 0725_12

* 0725_format

---------

Co-authored-by: bowei.fw <[email protected]>
Co-authored-by: nuzant <[email protected]>
Co-authored-by: 朱晗 <[email protected]>

* [lite] [fix] Fix a performance issue and several minor issues before release (#203)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* .

* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2

Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408

Reviewed-by: 晓雷 <[email protected]>


* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`

* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9

Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422

Reviewed-by: 晓雷 <[email protected]>


* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .

* PullRequest: 423 [lite] Remove the boba example for github release.

Merge branch fw/remove-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* update readme

* PullRequest: 431 [Fix] Fix environment of lite

Merge branch fw/lite-dev of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/431

Reviewed-by: 晓雷 <[email protected]>


* change requirements
* .
* .
* .

* PullRequest: 440 [FIX] fix update weight from disk

Merge branch sxj/lite-fix-disk-update of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/440

Reviewed-by: 博惟 <[email protected]>


* [FIX] fix update weight from disk

* PullRequest: 442 [lite] Refactor `RemoteSGLangEngine` into two parts: `RemoteSGLangEngine` and `WorkflowExecutor`.

Merge branch mzy/workflow-executor of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/442

Reviewed-by: 博惟 <[email protected]>


* refactor workflow executor
* .
* fix tests and eval
* .
* .
* revert workflow executor into remote sglang engine
* .

* PullRequest: 456 [lite] [Bug] Use `ProcessPoolExecutor` to calculate reward to avoid rollout slow down

Merge branch mzy/lite/fix-reward of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/456?tab=comment

Reviewed-by: 博惟 <[email protected]>


* fix reward
* .
* .
* .

* PullRequest: 460 [lite][fix] add a warning when reward computation timeout

Merge branch fw/lite-fix of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/460

Reviewed-by: 晓雷 <[email protected]>


* add a warning when reward computation timeout

* PullRequest: 465 [lite][fix] Fix issues raised by tsao

Merge branch fw/lite-fix of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/465

Reviewed-by: 晓雷 <[email protected]>


* fix

---------

Co-authored-by: 晓雷 <[email protected]>
Co-authored-by: 冰临 <[email protected]>

* [lite] [refactor] Rename the `arealite` folder to `areal` (#205)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* .

* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2

Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408

Reviewed-by: 晓雷 <[email protected]>


* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`

* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9

Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422

Reviewed-by: 晓雷 <[email protected]>


* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .

* PullRequest: 423 [lite] Remove the boba example for github release.

Merge branch fw/remove-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* update readme

* PullRequest: 431 [Fix] Fix environment of lite

Merge branch fw/lite-dev of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/431

Reviewed-by: 晓雷 <[email protected]>


* change requirements
* .
* .
* .

* PullRequest: 440 [FIX] fix update weight from disk

Merge branch sxj/lite-fix-disk-update of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/440

Reviewed-by: 博惟 <[email protected]>


* [FIX] fix update weight from disk

* PullRequest: 442 [lite] Refactor `RemoteSGLangEngine` into two parts: `RemoteSGLangEngine` and `WorkflowExecutor`.

Merge branch mzy/workflow-executor of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/442

Reviewed-by: 博惟 <[email protected]>


* refactor workflow executor
* .
* fix tests and eval
* .
* .
* revert workflow executor into remote sglang engine
* .

* PullRequest: 456 [lite] [Bug] Use `ProcessPoolExecutor` to calculate reward to avoid rollout slow down

Merge branch mzy/lite/fix-reward of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/456?tab=comment

Reviewed-by: 博惟 <[email protected]>


* fix reward
* .
* .
* .

* PullRequest: 460 [lite][fix] add a warning when reward computation timeout

Merge branch fw/lite-fix of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/460

Reviewed-by: 晓雷 <[email protected]>


* add a warning when reward computation timeout

* PullRequest: 465 [lite][fix] Fix issues raised by tsao

Merge branch fw/lite-fix of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/465

Reviewed-by: 晓雷 <[email protected]>


* fix

* PullRequest: 463 [FEAT] add config converter

Merge branch sxj/lite-add-converter of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/463

Reviewed-by: 博惟 <[email protected]>


* [FEAT] add config converter
* [FIX] fix bugs
* [FIX] fix typos
* [FIX] fix cpu allocation
* [FIX] del vllm

* PullRequest: 466 [lite] Add seeding in training scripts and pad to max_tokens_per_mb to avoid memory fragmentation

Merge branch fw/lite-fix of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/466

Reviewed-by: 晓雷 <[email protected]>


* add seed
* .
* add option for pad to max len
* .

* PullRequest: 467 [lite][fix] remove logging functionality in stats_logger

Merge branch fw/lite-fix of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/467

Reviewed-by: 晓雷 <[email protected]>


* remove logging functionality in stats_logger
* .

* rename arealite to areal

* rename arealite to areal

* .

* local experiment run

* .

---------

Co-authored-by: 晓雷 <[email protected]>
Co-authored-by: 冰临 <[email protected]>

* [lite] [fix] Fix running the legacy experiment in the v0.4.9.post2 sglang environment (#207)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update we…
mjbmjb pushed a commit to mjbmjb/AReaL that referenced this pull request Sep 22, 2025
@antoinegg1 @garrett4wade
[WIP][BugFix] Fix forward bug of LLM and VLM (inclusionAI#218)
5b9e67b 
* 0707_1

* 0707_1

* 0707_2

* 0707_2

* 0703_3

* 0703_3

* r

* add api

* add directory structure

* add tests template

* p

* format

* fix

* fix

* checkout previous impl

* checkout previous implementations

* checkout prev impl

* refactor

* add remote sglang engine

* format

* 0707_6

* 0707_7

* add readme

* refactor1

* 0707_undone

* 0708_1

* 0708_2

* 0708_3

* 0708_7

* 0708_4

* 0709_1

* 0709_2

* 0709_3

* PullRequest: 331 [lite] Support remote sglang engine with corresponding testcases.

Merge branch fw/lite of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/331

Reviewed-by: 晓雷 <[email protected]>


* .
* add test for sglang remote engine
* fix

* 0709_4

* 0709_5

* 0709_

* 0709_6

* 0709_7

* PullRequest: 336 add wrapper

Merge branch lite-util-wrapper of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/336

Reviewed-by: 博惟 <[email protected]>


* add wrapper

* 0709_7

* 0709_8

* 0709_9

* PullRequest: 332 [lite] Support FSDP engines

Merge branch mzy/lite/engines of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/332

Reviewed-by: 晓雷 <[email protected]>


* fsdp2 engine
* fix utils
* add fsdp engine test
* .
* fsdp engine test passed
* unsqueeze immediately before model inputs and after model outputts
* add optimizer save/load, add position id calculation for input
* .
* format
* not to squeeze
* add train and eval api
* .
* .
* improve fsdp engine data preprocessing
* format
* PullRequest: 337 [lite] Add SFT trainer example.
* trainer log
* minor changes
* add update weights from disk
* fix type annotation

* PullRequest: 339 [Fix] Fix some minor issues to pass all tests.

Merge branch fw/lite of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/339

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .

* chore: empty commit

* ci: build images on demand

* ci: fix on demand condition

* ci: fix env sha

* PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods

Merge branch fw/lite-dev of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/340

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* fix

* 0710_1

* 0710_2

* [Fix] Merge previous contributions from fw/refactor to lite (#163)

* initial proposal

* add arealite

* .

* change api

* .

* remove LOG_ROOT

* remove MODEL_SAVE_PATH

* remove PARAM_REALLOC_PATH, DATASET_CACHE

* prepare for testing

* prepare for testing

* ready for run

* local run

* tests mainly pass

* format

* .

* amend cluster.py

* .

* .

* client test pass

* pass rollout test

* remove unused imports

* add arealite readme

* change api

* .

* .

* .

* .

* .

* .

* .

* .

* format

* .

* implement iteraptable generation (#112)

Co-authored-by: zhaochenyang <[email protected]>

* .

* fix

* .

* .

* .

* pass controller generate batch test

* .

* refactor rollout controller into worker and controller

* .

* .

* .

* change to async rollout

* pass rollout controller test

* pass test

* .

* update readme

* .

* sft debug

* .

* add lisence

* remove unused files

* remove unsed args in ppo

* add hf engine wrapper  (#116)

* add hf engine

* fix issues

* fix ppo bugs and add test

* add hf client interface and modify cli args

* fix bugs

* fix issues

* Merge fw/refactor

* Finish hf wrapper test

* add test

---------

Co-authored-by: Wei Fu <[email protected]>

* format

* format

* .

* refine hf engine

* .

* fix

* add fsdp engine and sft tests

* .

* .

* .

* pass ppo unittest

* pass ppo and rollout controller tests

* clear unused imports

* rename ppo to grpo

* change reward function organization

* reorganize code

* add dataset api

* .

* .

* .

* format

* chmod fix

* .

* rename workflow to collector

* refactor llm_client location

* .

* .

* fix llm server api

* refactor config structure

* .

* fix tests

* .

* .

* .

* Fix unresolved issue in SFTTrainer PR (#139)

* .

* .

* efficient loading

* format

* .

* .

* .

* .

* .

* .

* Add CI for testing AReaLite (#150)

* ci: add test-arealite

* ci: add checkout before running test-arealite

* ci: add USERNAME

* ci: add test script

* ci: add GitHub mirror

* ci: fix typo

* ci: clone one commit

* ci: fix condition

* ci: set command timeout to 60m

* ci: enable pip cache

* ci: optimize container lifecycle

* ci: split into many stages

* ci(test-arealite): fix typo

* ci: fix wrong env

* ci: fix pytest

* ci: uninstall transformer-engine

* ci: uninstall transformer-engine

* ci: fix model paths

* ci: show stdout/stderr

* ci: fix not clean up

* ci: backup sglang

* ci: remove tmp repo dir when run

* ci: fix docker run exit 1 condition

* ci(test-arealite): limit the concurrency and extend command timeout

* .

* merge fw/refactor

* revert some changes

* fix

---------

Co-authored-by: meizhiyu.mzy <[email protected]>
Co-authored-by: Chayenne <[email protected]>
Co-authored-by: zhaochenyang <[email protected]>
Co-authored-by: Jayon02 <[email protected]>
Co-authored-by: root <meizhiyu.mzy>
Co-authored-by: Zijian Zhang <[email protected]>

* 0710_2

* 0710_3

* [Doc] Add an instruction about how to run the SFT example. (#164)

* 0710_3

* 0710_3

* 0710_5

* 0710_4

* merge_2

* merge_3

* 0711_1

* 0711_2

* 0711_3

* 0711_4

* 0711_6

* 0711_7

* 0711_8

* 0711_8

* 0711_9

* 0711_10

* 0711-11

* [Fix] Fix CI running condition for lite. (#172)

* .

* fix

* .

* .

* fix

* fix

* fix

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* 0714_1

* 0714_2

* 0714_3

* 0714_3

* 0714_5

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* added LocalSGlangEngine and test (#170)

* added LocalSGLangEngine

* upload test file

* add build args

* fix sgl_local generate

* improved sgl local robustness

* test

* test updated

* added fallback when sgl engine isn't initialized

* finish test local engine

* added LocalSGlangEngine and test

* format and fix

format and fix, raise when generate missing field

format

* change cli_args.py

* add comment header

format

---------

Co-authored-by: ChangyiYang <[email protected]>

* format (#174)

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* 0715_1

* 0715_2

* 0715_2

* [Feat][Refactor]Support DeepSpeed AutoTP; Refactor hf_engine.py and unit test. (#161)

* refactor hf engine

* format file

* revert file format

* Squashed commit of the following:

commit a45164235353ea77babfe4d9ccf34b7b8549dd0e
Author: Wei Fu <[email protected]>
Date:   Thu Jul 10 13:14:10 2025 +0800

    [Doc] Add an instruction about how to run the SFT example. (#164)

commit 1690c4471f0f8e93fa69665bb72f42ee080f0590
Author: Wei Fu <[email protected]>
Date:   Thu Jul 10 12:56:24 2025 +0800

    [Fix] Merge previous contributions from fw/refactor to lite (#163)

    * initial proposal

    * add arealite

    * .

    * change api

    * .

    * remove LOG_ROOT

    * remove MODEL_SAVE_PATH

    * remove PARAM_REALLOC_PATH, DATASET_CACHE

    * prepare for testing

    * prepare for testing

    * ready for run

    * local run

    * tests mainly pass

    * format

    * .

    * amend cluster.py

    * .

    * .

    * client test pass

    * pass rollout test

    * remove unused imports

    * add arealite readme

    * change api

    * .

    * .

    * .

    * .

    * .

    * .

    * .

    * .

    * format

    * .

    * implement iteraptable generation (#112)

    Co-authored-by: zhaochenyang <[email protected]>

    * .

    * fix

    * .

    * .

    * .

    * pass controller generate batch test

    * .

    * refactor rollout controller into worker and controller

    * .

    * .

    * .

    * change to async rollout

    * pass rollout controller test

    * pass test

    * .

    * update readme

    * .

    * sft debug

    * .

    * add lisence

    * remove unused files

    * remove unsed args in ppo

    * add hf engine wrapper  (#116)

    * add hf engine

    * fix issues

    * fix ppo bugs and add test

    * add hf client interface and modify cli args

    * fix bugs

    * fix issues

    * Merge fw/refactor

    * Finish hf wrapper test

    * add test

    ---------

    Co-authored-by: Wei Fu <[email protected]>

    * format

    * format

    * .

    * refine hf engine

    * .

    * fix

    * add fsdp engine and sft tests

    * .

    * .

    * .

    * pass ppo unittest

    * pass ppo and rollout controller tests

    * clear unused imports

    * rename ppo to grpo

    * change reward function organization

    * reorganize code

    * add dataset api

    * .

    * .

    * .

    * format

    * chmod fix

    * .

    * rename workflow to collector

    * refactor llm_client location

    * .

    * .

    * fix llm server api

    * refactor config structure

    * .

    * fix tests

    * .

    * .

    * .

    * Fix unresolved issue in SFTTrainer PR (#139)

    * .

    * .

    * efficient loading

    * format

    * .

    * .

    * .

    * .

    * .

    * .

    * Add CI for testing AReaLite (#150)

    * ci: add test-arealite

    * ci: add checkout before running test-arealite

    * ci: add USERNAME

    * ci: add test script

    * ci: add GitHub mirror

    * ci: fix typo

    * ci: clone one commit

    * ci: fix condition

    * ci: set command timeout to 60m

    * ci: enable pip cache

    * ci: optimize container lifecycle

    * ci: split into many stages

    * ci(test-arealite): fix typo

    * ci: fix wrong env

    * ci: fix pytest

    * ci: uninstall transformer-engine

    * ci: uninstall transformer-engine

    * ci: fix model paths

    * ci: show stdout/stderr

    * ci: fix not clean up

    * ci: backup sglang

    * ci: remove tmp repo dir when run

    * ci: fix docker run exit 1 condition

    * ci(test-arealite): limit the concurrency and extend command timeout

    * .

    * merge fw/refactor

    * revert some changes

    * fix

    ---------

    Co-authored-by: meizhiyu.mzy <[email protected]>
    Co-authored-by: Chayenne <[email protected]>
    Co-authored-by: zhaochenyang <[email protected]>
    Co-authored-by: Jayon02 <[email protected]>
    Co-authored-by: root <meizhiyu.mzy>
    Co-authored-by: Zijian Zhang <[email protected]>

commit d5696a0056eb5622f5cc4c43857d2f423a3efe2c
Merge: 8fc5aa1 743896d
Author: 博惟 <[email protected]>
Date:   Thu Jul 10 12:53:30 2025 +0800

    Merge branch 'main' of https://github.com/inclusionAI/AReaL into lite

commit 8fc5aa11b154ea31fba5a1600cf10dc74ee6f42d
Merge: aa2f819 b588dc3
Author: 博惟 <[email protected]>
Date:   Thu Jul 10 11:15:01 2025 +0800

    Merge branch 'lite' of https://github.com/inclusionAI/AReaL into lite

commit aa2f8196989673556bc25bdf8228c1d8e31834c5
Author: 博惟 <[email protected]>
Date:   Thu Jul 10 11:10:10 2025 +0800

    PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods

    Merge branch fw/lite-dev of [email protected]:inclusionAI/AReaL.git into lite
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/340

    Reviewed-by: 晓雷 <[email protected]>

    * support fsdp engine and sglang remote engine
    * minor fix
    * .
    * refactor trainer
    * add close
    * rm mb_spec
    * fix

commit 743896dbd3938fb879dc67cb9990462236778193
Author: Wei Fu <[email protected]>
Date:   Wed Jul 9 10:50:19 2025 +0800

    Update to persistent wechat QR code. (#159)

commit 80e16378237dd89878bdf79e5992932550df9493
Author: xssstory <[email protected]>
Date:   Mon Jul 7 15:49:13 2025 +0800

    fix math reward verifier (#156)

    * PullRequest: 293 fix get_param_realloc_path

    Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/293

    Reviewed-by: 博惟 <[email protected]>

    * fix get_param_realloc_path

    * PullRequest: 297 bugfix: reward is always -5

    Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/297

    Reviewed-by: 博惟 <[email protected]>

    * bugfix: reward is always -5

    * PullRequest: 321 fix checkpoint save dir

    Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/321

    Reviewed-by: 博惟 <[email protected]>

    * fix checkpoint save dir

    * PullRequest: 328 [Doc] update installation

    Merge branch sxj/doc of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/328

    Reviewed-by: 博惟 <[email protected]>

    * [Doc] update installation

    * PullRequest: 329 bugfix: math verifier blocks the async training

    Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/329

    Reviewed-by: 博惟 <[email protected]>

    * bugfix: math verifier block the async training

    * format

    ---------

    Co-authored-by: 冰临 <[email protected]>
    Co-authored-by: garrett4wade <[email protected]>

* add autotp for hf

* refactor test

* fix bugs

* fix issues

* format files

* Squashed commit of the following:

commit 93f024b2d173b180a291cc58f73b04bec493a7fb
Author: Wei Fu <[email protected]>
Date:   Tue Jul 15 10:24:48 2025 +0800

    format (#174)

commit 4bac0a556eca458c0b2f080453408f242bb79b3c
Author: Night <[email protected]>
Date:   Mon Jul 14 19:22:00 2025 -0700

    added LocalSGlangEngine and test (#170)

    * added LocalSGLangEngine

    * upload test file

    * add build args

    * fix sgl_local generate

    * improved sgl local robustness

    * test

    * test updated

    * added fallback when sgl engine isn't initialized

    * finish test local engine

    * added LocalSGlangEngine and test

    * format and fix

    format and fix, raise when generate missing field

    format

    * change cli_args.py

    * add comment header

    format

    ---------

    Co-authored-by: ChangyiYang <[email protected]>

---------

Co-authored-by: Jayon02 <[email protected]>
Co-authored-by: Wei Fu <[email protected]>

* fix ci (#175)

* [Feature] Add pre-commit (#178)

* fix ci

* .

* .

* .

* [lite] [refactor] Add GSM8k GRPO example. (#179)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* fix ci

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

---------

Co-authored-by: 晓雷 <[email protected]>

* 0716_1

* 0716_2

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* change doc (#180)

* [Fix] [lite] Merge from the internal repo to fix GRPO bugs and refactor the train engine (#181)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

---------

Co-authored-by: 晓雷 <[email protected]>

* 0716_3

* 0716_4

* 0716_4

* 0716_5

* 0717_1

* 0717_3

* 0717_3

* 0717_4

* 0717_5

* 0717_6

* 0717_6

* 0717_6

* 0718_2

* 0718_4

* 0718_5

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* merge_0721

* 0721_1

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* 0721_2

* 0721_3

* Implement fsdp distributed update (#183)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* added remote nccl weight update

feat: implement update_weights_from_distributed in fsdp_engine.py

unfinishd test, raise PR first

coroutine for each server

chore: change uploads weights behavior, change test order

fix small bug

fixed test

* fix rebase

* add test.sh

* updated, test stil fails

* .

* .

* .

* fix: full_tensor() should happen in all rank (#187)

Co-authored-by: ChangyiYang <[email protected]>

---------

Co-authored-by: 晓雷 <[email protected]>
Co-authored-by: 博惟 <[email protected]>
Co-authored-by: ChangyiYang <[email protected]>
Co-authored-by: ChangyiYang <[email protected]>

* [Feature] [lite] Merge from internal dev repo (#189)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* format

* format

* .

---------

Co-authored-by: 晓雷 <[email protected]>

* [Refactor] Rename files in arealite before release. (#190)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* format

* format

* .

* rename

* rename

---------

Co-authored-by: 晓雷 <[email protected]>

* 0721_4

* .

* 0721_formal

* 0721_formal

* 0721_merge4

* 0721_merge5

* 0721_6

* 0721_merge6

* 0721_merge7

* 0721_8

* 0722_1

* 0722_2

* 0722_3

* add quickstart (#194)

* [doc] [lite] Add customization docs for AReaLite. (#191)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* .

* .

* .

* .

* .

* .

* .

---------

Co-authored-by: 晓雷 <[email protected]>

* 0722_4

* 0722_4

* 0722_5

* 0722_6

* 0722_7

* 0723_1

* reformatted

* clang-reformatted

* clang-reformatted2

* 0723_1

* 0723_1

* 0723_1

* 0723_merge3

* 0723_4

* 0723_reformatted_5

* 0724_1

* 0724_1

* 0724_merge1

* 0724_merge2

* 0724_merge3

* 0724_merge3

* 0724_merge4

* [lite] [feature] Bump to SGLang v0.4.9.post2 and use NCCL to update weights (#196)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* .

* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2

Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408

Reviewed-by: 晓雷 <[email protected]>


* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`

* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9

Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422

Reviewed-by: 晓雷 <[email protected]>


* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .

* PullRequest: 423 [lite] Remove the boba example for github release.

Merge branch fw/remove-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423

Reviewed-by: 晓雷 <[email protected]>


* .
* .

---------

Co-authored-by: 晓雷 <[email protected]>

* 0724_merge5

* 0724_merge6

* 0724_merge7

* [Docs] [lite] Add example code walkthrough documentation. (#197)

* .

* .

* fix grammar and minor revise

* [lite] [doc] Add AReaLite design doc as README (#198)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* .

* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2

Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408

Reviewed-by: 晓雷 <[email protected]>


* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`

* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9

Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422

Reviewed-by: 晓雷 <[email protected]>


* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .

* PullRequest: 423 [lite] Remove the boba example for github release.

Merge branch fw/remove-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* update readme

* .

---------

Co-authored-by: 晓雷 <[email protected]>

* 0724_4

* 0724-merge8

* 0724_merge8

* 0725_1

* 0725_6

* 0725_7

* 0725_4padded_image

* 0725_9padded_image

* 0725_10padded_image

* 0725

* 0725_12

* 0725_format

* Add self-hosted runner support (#199)

* ci: add testsz for self hosted runner

* ci: use self-hosted runner

* ci: run ci on push

* ci: remove cd to tmp dir

* ci: fix

* ci: add environment back

* ci: set concurrency

* ci: fix build env image

* Update .github/workflows/test-arealite.yml

Co-authored-by: Copilot <[email protected]>

* Update ci/build_env_image.sh

Co-authored-by: Copilot <[email protected]>

* ci: add some test code

* ci: simplify

* ci: try to fix

* ci: test

* ci: test2

* ci: test3

* ci: test3

* ci: tidy up

* ci: simplify

* ci: do not trigger on push

* ci: test arealite/tests/

* ci: use UUID as run id

* ci: use openssl rand to genreate run id

---------

Co-authored-by: Copilot <[email protected]>

* [WIP][feat] Initial support for VLMs, add Qwen2VL SFT test and Qwen2.5VL GRPO test (#188)

* vlm_sft_test

* vlm_sft_test

* .

* .

* Fix unresolved issue in SFTTrainer PR (#139)

* .

* .

* efficient loading

* format

* .

* .

* Fix unresolved issue in SFTTrainer PR (#139)

* .

* .

* efficient loading

* format

* .

* .

* image_process0701

* image_process0701

* image_process0701_2

* image_process0701_2

* image_process0701_3

* image_process0701_3

* .

* .

* .

* .

* .

* .

* imageprocess0702

* imageprocess0702

* image_process0702_2

* image_process0702_2

* image_process0702_3

* image_process0702_3

* image_process0702_4

* image_process0702_4

* image_process0702_5

* image_process0702_5

* image_process0703_1

* image_process0703_1

* 0703_2

* 0703_2

* 0703_3

* 0703_3

* 0703_4

* 0703_4

* 0703_4

* 0703_4

* 0703_5

* 0703_5

* 0703_6

* 0703_6

* 0703_7

* 0703_7

* 0703_8

* 0703_8

* 0703_9

* 0703_9

* 0703_11

* 0703_11

* 0703_12

* 0703_12

* 0703_13

* 0703_13

* 0703_14

* 0703_14

* 0703_15

* 0703_15

* 0703_16

* 0703_16

* 0703-17

* 0703-17

* 0703_18

* 0703_18

* 0703_18

* 0703_18

* 0703_19

* 0703_19

* 0704_1

* 0704_1

* 0704_2

* 0704_2

* 0704_3

* 0704_3

* .

* .

* 0707_1

* 0707_1

* 0707_2

* 0707_2

* 0703_3

* 0703_3

* r

* p

* fix

* fix

* refactor

* 0707_6

* 0707_7

* refactor1

* 0707_undone

* 0708_1

* 0708_2

* 0708_3

* 0708_7

* 0708_4

* 0709_1

* 0709_2

* 0709_3

* 0709_4

* 0709_5

* 0709_

* 0709_6

* 0709_7

* 0709_7

* 0709_8

* 0709_9

* 0710_1

* 0710_2

* 0710_2

* 0710_3

* 0710_3

* 0710_3

* 0710_5

* 0710_4

* merge_2

* merge_3

* 0711_1

* 0711_2

* 0711_3

* 0711_4

* 0711_6

* 0711_7

* 0711_8

* 0711_8

* 0711_9

* 0711_10

* 0711-11

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* 0714_1

* 0714_2

* 0714_3

* 0714_3

* 0714_5

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* 0715_1

* 0715_2

* 0715_2

* 0716_1

* 0716_2

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* 0716_3

* 0716_4

* 0716_4

* 0716_5

* 0717_1

* 0717_3

* 0717_3

* 0717_4

* 0717_5

* 0717_6

* 0717_6

* 0717_6

* 0718_2

* 0718_4

* 0718_5

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* merge_0721

* 0721_1

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* 0721_2

* 0721_3

* 0721_4

* .

* 0721_formal

* 0721_formal

* 0721_merge4

* 0721_merge5

* 0721_6

* 0721_merge6

* 0721_merge7

* 0721_8

* 0722_1

* 0722_2

* 0722_3

* 0722_4

* 0722_4

* 0722_5

* 0722_6

* 0722_7

* 0723_1

* reformatted

* clang-reformatted

* clang-reformatted2

* 0723_1

* 0723_1

* 0723_1

* 0723_merge3

* 0723_4

* 0723_reformatted_5

* 0724_1

* 0724_1

* 0724_merge1

* 0724_merge2

* 0724_merge3

* 0724_merge3

* 0724_merge4

* 0724_merge5

* 0724_merge6

* 0724_merge7

* 0724_4

* 0724-merge8

* 0724_merge8

* 0725_1

* 0725_6

* 0725_7

* 0725_4padded_image

* 0725_9padded_image

* 0725_10padded_image

* 0725

* 0725_12

* 0725_format

---------

Co-authored-by: bowei.fw <[email protected]>
Co-authored-by: nuzant <[email protected]>
Co-authored-by: 朱晗 <[email protected]>

* 0731

* 0731_1

* 0731_2

* 0731_2

* 0731_3

* 0731_4

* 0801_1

* 0801_2

* 0804_1

* 0804_2

* 0804_2

* 0804_5

* 0805_3

* 0805_2

* 0806

* 0806_merge1

* 0806_merge2

* 0806_format1

* 0806_merge3

* 0806_4

* 0806_6

* 0806_7

* 0806_formatted2

* fix

* revert examples

* .

* .

---------

Co-authored-by: 朱晗 <[email protected]>
Co-authored-by: 博惟 <[email protected]>
Co-authored-by: garrett4wade <[email protected]>
Co-authored-by: Wei Fu <[email protected]>
nv8300 pushed a commit to nv8300/AReaL that referenced this pull request Oct 13, 2025
@antoinegg1 @garrett4wade
[WIP][BugFix] Fix forward bug of LLM and VLM (inclusionAI#218)
11062ab 
* 0707_1

* 0707_1

* 0707_2

* 0707_2

* 0703_3

* 0703_3

* r

* add api

* add directory structure

* add tests template

* p

* format

* fix

* fix

* checkout previous impl

* checkout previous implementations

* checkout prev impl

* refactor

* add remote sglang engine

* format

* 0707_6

* 0707_7

* add readme

* refactor1

* 0707_undone

* 0708_1

* 0708_2

* 0708_3

* 0708_7

* 0708_4

* 0709_1

* 0709_2

* 0709_3

* PullRequest: 331 [lite] Support remote sglang engine with corresponding testcases.

Merge branch fw/lite of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/331

Reviewed-by: 晓雷 <[email protected]>


* .
* add test for sglang remote engine
* fix

* 0709_4

* 0709_5

* 0709_

* 0709_6

* 0709_7

* PullRequest: 336 add wrapper

Merge branch lite-util-wrapper of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/336

Reviewed-by: 博惟 <[email protected]>


* add wrapper

* 0709_7

* 0709_8

* 0709_9

* PullRequest: 332 [lite] Support FSDP engines

Merge branch mzy/lite/engines of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/332

Reviewed-by: 晓雷 <[email protected]>


* fsdp2 engine
* fix utils
* add fsdp engine test
* .
* fsdp engine test passed
* unsqueeze immediately before model inputs and after model outputts
* add optimizer save/load, add position id calculation for input
* .
* format
* not to squeeze
* add train and eval api
* .
* .
* improve fsdp engine data preprocessing
* format
* PullRequest: 337 [lite] Add SFT trainer example.
* trainer log
* minor changes
* add update weights from disk
* fix type annotation

* PullRequest: 339 [Fix] Fix some minor issues to pass all tests.

Merge branch fw/lite of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/339

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .

* chore: empty commit

* ci: build images on demand

* ci: fix on demand condition

* ci: fix env sha

* PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods

Merge branch fw/lite-dev of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/340

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* fix

* 0710_1

* 0710_2

* [Fix] Merge previous contributions from fw/refactor to lite (#163)

* initial proposal

* add arealite

* .

* change api

* .

* remove LOG_ROOT

* remove MODEL_SAVE_PATH

* remove PARAM_REALLOC_PATH, DATASET_CACHE

* prepare for testing

* prepare for testing

* ready for run

* local run

* tests mainly pass

* format

* .

* amend cluster.py

* .

* .

* client test pass

* pass rollout test

* remove unused imports

* add arealite readme

* change api

* .

* .

* .

* .

* .

* .

* .

* .

* format

* .

* implement iteraptable generation (#112)

Co-authored-by: zhaochenyang <[email protected]>

* .

* fix

* .

* .

* .

* pass controller generate batch test

* .

* refactor rollout controller into worker and controller

* .

* .

* .

* change to async rollout

* pass rollout controller test

* pass test

* .

* update readme

* .

* sft debug

* .

* add lisence

* remove unused files

* remove unsed args in ppo

* add hf engine wrapper  (#116)

* add hf engine

* fix issues

* fix ppo bugs and add test

* add hf client interface and modify cli args

* fix bugs

* fix issues

* Merge fw/refactor

* Finish hf wrapper test

* add test

---------

Co-authored-by: Wei Fu <[email protected]>

* format

* format

* .

* refine hf engine

* .

* fix

* add fsdp engine and sft tests

* .

* .

* .

* pass ppo unittest

* pass ppo and rollout controller tests

* clear unused imports

* rename ppo to grpo

* change reward function organization

* reorganize code

* add dataset api

* .

* .

* .

* format

* chmod fix

* .

* rename workflow to collector

* refactor llm_client location

* .

* .

* fix llm server api

* refactor config structure

* .

* fix tests

* .

* .

* .

* Fix unresolved issue in SFTTrainer PR (#139)

* .

* .

* efficient loading

* format

* .

* .

* .

* .

* .

* .

* Add CI for testing AReaLite (#150)

* ci: add test-arealite

* ci: add checkout before running test-arealite

* ci: add USERNAME

* ci: add test script

* ci: add GitHub mirror

* ci: fix typo

* ci: clone one commit

* ci: fix condition

* ci: set command timeout to 60m

* ci: enable pip cache

* ci: optimize container lifecycle

* ci: split into many stages

* ci(test-arealite): fix typo

* ci: fix wrong env

* ci: fix pytest

* ci: uninstall transformer-engine

* ci: uninstall transformer-engine

* ci: fix model paths

* ci: show stdout/stderr

* ci: fix not clean up

* ci: backup sglang

* ci: remove tmp repo dir when run

* ci: fix docker run exit 1 condition

* ci(test-arealite): limit the concurrency and extend command timeout

* .

* merge fw/refactor

* revert some changes

* fix

---------

Co-authored-by: meizhiyu.mzy <[email protected]>
Co-authored-by: Chayenne <[email protected]>
Co-authored-by: zhaochenyang <[email protected]>
Co-authored-by: Jayon02 <[email protected]>
Co-authored-by: root <meizhiyu.mzy>
Co-authored-by: Zijian Zhang <[email protected]>

* 0710_2

* 0710_3

* [Doc] Add an instruction about how to run the SFT example. (#164)

* 0710_3

* 0710_3

* 0710_5

* 0710_4

* merge_2

* merge_3

* 0711_1

* 0711_2

* 0711_3

* 0711_4

* 0711_6

* 0711_7

* 0711_8

* 0711_8

* 0711_9

* 0711_10

* 0711-11

* [Fix] Fix CI running condition for lite. (#172)

* .

* fix

* .

* .

* fix

* fix

* fix

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* 0714_1

* 0714_2

* 0714_3

* 0714_3

* 0714_5

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* added LocalSGlangEngine and test (#170)

* added LocalSGLangEngine

* upload test file

* add build args

* fix sgl_local generate

* improved sgl local robustness

* test

* test updated

* added fallback when sgl engine isn't initialized

* finish test local engine

* added LocalSGlangEngine and test

* format and fix

format and fix, raise when generate missing field

format

* change cli_args.py

* add comment header

format

---------

Co-authored-by: ChangyiYang <[email protected]>

* format (#174)

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* 0715_1

* 0715_2

* 0715_2

* [Feat][Refactor]Support DeepSpeed AutoTP; Refactor hf_engine.py and unit test. (#161)

* refactor hf engine

* format file

* revert file format

* Squashed commit of the following:

commit 8d4b8dc90fceb144bf00f8125f938b57f696e166
Author: Wei Fu <[email protected]>
Date:   Thu Jul 10 13:14:10 2025 +0800

    [Doc] Add an instruction about how to run the SFT example. (#164)

commit 3bf9c85e400a2fe1c6a1e488d27d458ede8cea45
Author: Wei Fu <[email protected]>
Date:   Thu Jul 10 12:56:24 2025 +0800

    [Fix] Merge previous contributions from fw/refactor to lite (#163)

    * initial proposal

    * add arealite

    * .

    * change api

    * .

    * remove LOG_ROOT

    * remove MODEL_SAVE_PATH

    * remove PARAM_REALLOC_PATH, DATASET_CACHE

    * prepare for testing

    * prepare for testing

    * ready for run

    * local run

    * tests mainly pass

    * format

    * .

    * amend cluster.py

    * .

    * .

    * client test pass

    * pass rollout test

    * remove unused imports

    * add arealite readme

    * change api

    * .

    * .

    * .

    * .

    * .

    * .

    * .

    * .

    * format

    * .

    * implement iteraptable generation (#112)

    Co-authored-by: zhaochenyang <[email protected]>

    * .

    * fix

    * .

    * .

    * .

    * pass controller generate batch test

    * .

    * refactor rollout controller into worker and controller

    * .

    * .

    * .

    * change to async rollout

    * pass rollout controller test

    * pass test

    * .

    * update readme

    * .

    * sft debug

    * .

    * add lisence

    * remove unused files

    * remove unsed args in ppo

    * add hf engine wrapper  (#116)

    * add hf engine

    * fix issues

    * fix ppo bugs and add test

    * add hf client interface and modify cli args

    * fix bugs

    * fix issues

    * Merge fw/refactor

    * Finish hf wrapper test

    * add test

    ---------

    Co-authored-by: Wei Fu <[email protected]>

    * format

    * format

    * .

    * refine hf engine

    * .

    * fix

    * add fsdp engine and sft tests

    * .

    * .

    * .

    * pass ppo unittest

    * pass ppo and rollout controller tests

    * clear unused imports

    * rename ppo to grpo

    * change reward function organization

    * reorganize code

    * add dataset api

    * .

    * .

    * .

    * format

    * chmod fix

    * .

    * rename workflow to collector

    * refactor llm_client location

    * .

    * .

    * fix llm server api

    * refactor config structure

    * .

    * fix tests

    * .

    * .

    * .

    * Fix unresolved issue in SFTTrainer PR (#139)

    * .

    * .

    * efficient loading

    * format

    * .

    * .

    * .

    * .

    * .

    * .

    * Add CI for testing AReaLite (#150)

    * ci: add test-arealite

    * ci: add checkout before running test-arealite

    * ci: add USERNAME

    * ci: add test script

    * ci: add GitHub mirror

    * ci: fix typo

    * ci: clone one commit

    * ci: fix condition

    * ci: set command timeout to 60m

    * ci: enable pip cache

    * ci: optimize container lifecycle

    * ci: split into many stages

    * ci(test-arealite): fix typo

    * ci: fix wrong env

    * ci: fix pytest

    * ci: uninstall transformer-engine

    * ci: uninstall transformer-engine

    * ci: fix model paths

    * ci: show stdout/stderr

    * ci: fix not clean up

    * ci: backup sglang

    * ci: remove tmp repo dir when run

    * ci: fix docker run exit 1 condition

    * ci(test-arealite): limit the concurrency and extend command timeout

    * .

    * merge fw/refactor

    * revert some changes

    * fix

    ---------

    Co-authored-by: meizhiyu.mzy <[email protected]>
    Co-authored-by: Chayenne <[email protected]>
    Co-authored-by: zhaochenyang <[email protected]>
    Co-authored-by: Jayon02 <[email protected]>
    Co-authored-by: root <meizhiyu.mzy>
    Co-authored-by: Zijian Zhang <[email protected]>

commit d48bf007cfb0821c35a18a673f3c2e8f51a8ee45
Merge: 42c717b 743896d
Author: 博惟 <[email protected]>
Date:   Thu Jul 10 12:53:30 2025 +0800

    Merge branch 'main' of https://github.com/inclusionAI/AReaL into lite

commit 42c717b6e45b3b345f29cff5882ba5374b79faee
Merge: c38cffc a203c7c
Author: 博惟 <[email protected]>
Date:   Thu Jul 10 11:15:01 2025 +0800

    Merge branch 'lite' of https://github.com/inclusionAI/AReaL into lite

commit c38cffc02385acf382196ba6b720b2d8b91a5675
Author: 博惟 <[email protected]>
Date:   Thu Jul 10 11:10:10 2025 +0800

    PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods

    Merge branch fw/lite-dev of [email protected]:inclusionAI/AReaL.git into lite
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/340

    Reviewed-by: 晓雷 <[email protected]>

    * support fsdp engine and sglang remote engine
    * minor fix
    * .
    * refactor trainer
    * add close
    * rm mb_spec
    * fix

commit 743896dbd3938fb879dc67cb9990462236778193
Author: Wei Fu <[email protected]>
Date:   Wed Jul 9 10:50:19 2025 +0800

    Update to persistent wechat QR code. (#159)

commit 80e16378237dd89878bdf79e5992932550df9493
Author: xssstory <[email protected]>
Date:   Mon Jul 7 15:49:13 2025 +0800

    fix math reward verifier (#156)

    * PullRequest: 293 fix get_param_realloc_path

    Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/293

    Reviewed-by: 博惟 <[email protected]>

    * fix get_param_realloc_path

    * PullRequest: 297 bugfix: reward is always -5

    Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/297

    Reviewed-by: 博惟 <[email protected]>

    * bugfix: reward is always -5

    * PullRequest: 321 fix checkpoint save dir

    Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/321

    Reviewed-by: 博惟 <[email protected]>

    * fix checkpoint save dir

    * PullRequest: 328 [Doc] update installation

    Merge branch sxj/doc of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/328

    Reviewed-by: 博惟 <[email protected]>

    * [Doc] update installation

    * PullRequest: 329 bugfix: math verifier blocks the async training

    Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/329

    Reviewed-by: 博惟 <[email protected]>

    * bugfix: math verifier block the async training

    * format

    ---------

    Co-authored-by: 冰临 <[email protected]>
    Co-authored-by: garrett4wade <[email protected]>

* add autotp for hf

* refactor test

* fix bugs

* fix issues

* format files

* Squashed commit of the following:

commit 9ed043f6ab83cf9c9b38e0251ad08009a73243e8
Author: Wei Fu <[email protected]>
Date:   Tue Jul 15 10:24:48 2025 +0800

    format (#174)

commit 8cc9b1feb5ed03cfa42f3ed588dd5f6ec16e5e55
Author: Night <[email protected]>
Date:   Mon Jul 14 19:22:00 2025 -0700

    added LocalSGlangEngine and test (#170)

    * added LocalSGLangEngine

    * upload test file

    * add build args

    * fix sgl_local generate

    * improved sgl local robustness

    * test

    * test updated

    * added fallback when sgl engine isn't initialized

    * finish test local engine

    * added LocalSGlangEngine and test

    * format and fix

    format and fix, raise when generate missing field

    format

    * change cli_args.py

    * add comment header

    format

    ---------

    Co-authored-by: ChangyiYang <[email protected]>

---------

Co-authored-by: Jayon02 <[email protected]>
Co-authored-by: Wei Fu <[email protected]>

* fix ci (#175)

* [Feature] Add pre-commit (#178)

* fix ci

* .

* .

* .

* [lite] [refactor] Add GSM8k GRPO example. (#179)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* fix ci

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

---------

Co-authored-by: 晓雷 <[email protected]>

* 0716_1

* 0716_2

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* change doc (#180)

* [Fix] [lite] Merge from the internal repo to fix GRPO bugs and refactor the train engine (#181)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

---------

Co-authored-by: 晓雷 <[email protected]>

* 0716_3

* 0716_4

* 0716_4

* 0716_5

* 0717_1

* 0717_3

* 0717_3

* 0717_4

* 0717_5

* 0717_6

* 0717_6

* 0717_6

* 0718_2

* 0718_4

* 0718_5

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* merge_0721

* 0721_1

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* 0721_2

* 0721_3

* Implement fsdp distributed update (#183)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* added remote nccl weight update

feat: implement update_weights_from_distributed in fsdp_engine.py

unfinishd test, raise PR first

coroutine for each server

chore: change uploads weights behavior, change test order

fix small bug

fixed test

* fix rebase

* add test.sh

* updated, test stil fails

* .

* .

* .

* fix: full_tensor() should happen in all rank (#187)

Co-authored-by: ChangyiYang <[email protected]>

---------

Co-authored-by: 晓雷 <[email protected]>
Co-authored-by: 博惟 <[email protected]>
Co-authored-by: ChangyiYang <[email protected]>
Co-authored-by: ChangyiYang <[email protected]>

* [Feature] [lite] Merge from internal dev repo (#189)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* format

* format

* .

---------

Co-authored-by: 晓雷 <[email protected]>

* [Refactor] Rename files in arealite before release. (#190)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* format

* format

* .

* rename

* rename

---------

Co-authored-by: 晓雷 <[email protected]>

* 0721_4

* .

* 0721_formal

* 0721_formal

* 0721_merge4

* 0721_merge5

* 0721_6

* 0721_merge6

* 0721_merge7

* 0721_8

* 0722_1

* 0722_2

* 0722_3

* add quickstart (#194)

* [doc] [lite] Add customization docs for AReaLite. (#191)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* .

* .

* .

* .

* .

* .

* .

---------

Co-authored-by: 晓雷 <[email protected]>

* 0722_4

* 0722_4

* 0722_5

* 0722_6

* 0722_7

* 0723_1

* reformatted

* clang-reformatted

* clang-reformatted2

* 0723_1

* 0723_1

* 0723_1

* 0723_merge3

* 0723_4

* 0723_reformatted_5

* 0724_1

* 0724_1

* 0724_merge1

* 0724_merge2

* 0724_merge3

* 0724_merge3

* 0724_merge4

* [lite] [feature] Bump to SGLang v0.4.9.post2 and use NCCL to update weights (#196)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* .

* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2

Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408

Reviewed-by: 晓雷 <[email protected]>


* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`

* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9

Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422

Reviewed-by: 晓雷 <[email protected]>


* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .

* PullRequest: 423 [lite] Remove the boba example for github release.

Merge branch fw/remove-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423

Reviewed-by: 晓雷 <[email protected]>


* .
* .

---------

Co-authored-by: 晓雷 <[email protected]>

* 0724_merge5

* 0724_merge6

* 0724_merge7

* [Docs] [lite] Add example code walkthrough documentation. (#197)

* .

* .

* fix grammar and minor revise

* [lite] [doc] Add AReaLite design doc as README (#198)

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* .

* PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2

Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/408

Reviewed-by: 晓雷 <[email protected]>


* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`

* PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9

Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/422

Reviewed-by: 晓雷 <[email protected]>


* .
* bump arealite to sglang 0.4.9.post2
* .
* PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta`
* .

* PullRequest: 423 [lite] Remove the boba example for github release.

Merge branch fw/remove-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/423

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* update readme

* .

---------

Co-authored-by: 晓雷 <[email protected]>

* 0724_4

* 0724-merge8

* 0724_merge8

* 0725_1

* 0725_6

* 0725_7

* 0725_4padded_image

* 0725_9padded_image

* 0725_10padded_image

* 0725

* 0725_12

* 0725_format

* Add self-hosted runner support (#199)

* ci: add testsz for self hosted runner

* ci: use self-hosted runner

* ci: run ci on push

* ci: remove cd to tmp dir

* ci: fix

* ci: add environment back

* ci: set concurrency

* ci: fix build env image

* Update .github/workflows/test-arealite.yml

Co-authored-by: Copilot <[email protected]>

* Update ci/build_env_image.sh

Co-authored-by: Copilot <[email protected]>

* ci: add some test code

* ci: simplify

* ci: try to fix

* ci: test

* ci: test2

* ci: test3

* ci: test3

* ci: tidy up

* ci: simplify

* ci: do not trigger on push

* ci: test arealite/tests/

* ci: use UUID as run id

* ci: use openssl rand to genreate run id

---------

Co-authored-by: Copilot <[email protected]>

* [WIP][feat] Initial support for VLMs, add Qwen2VL SFT test and Qwen2.5VL GRPO test (#188)

* vlm_sft_test

* vlm_sft_test

* .

* .

* Fix unresolved issue in SFTTrainer PR (#139)

* .

* .

* efficient loading

* format

* .

* .

* Fix unresolved issue in SFTTrainer PR (#139)

* .

* .

* efficient loading

* format

* .

* .

* image_process0701

* image_process0701

* image_process0701_2

* image_process0701_2

* image_process0701_3

* image_process0701_3

* .

* .

* .

* .

* .

* .

* imageprocess0702

* imageprocess0702

* image_process0702_2

* image_process0702_2

* image_process0702_3

* image_process0702_3

* image_process0702_4

* image_process0702_4

* image_process0702_5

* image_process0702_5

* image_process0703_1

* image_process0703_1

* 0703_2

* 0703_2

* 0703_3

* 0703_3

* 0703_4

* 0703_4

* 0703_4

* 0703_4

* 0703_5

* 0703_5

* 0703_6

* 0703_6

* 0703_7

* 0703_7

* 0703_8

* 0703_8

* 0703_9

* 0703_9

* 0703_11

* 0703_11

* 0703_12

* 0703_12

* 0703_13

* 0703_13

* 0703_14

* 0703_14

* 0703_15

* 0703_15

* 0703_16

* 0703_16

* 0703-17

* 0703-17

* 0703_18

* 0703_18

* 0703_18

* 0703_18

* 0703_19

* 0703_19

* 0704_1

* 0704_1

* 0704_2

* 0704_2

* 0704_3

* 0704_3

* .

* .

* 0707_1

* 0707_1

* 0707_2

* 0707_2

* 0703_3

* 0703_3

* r

* p

* fix

* fix

* refactor

* 0707_6

* 0707_7

* refactor1

* 0707_undone

* 0708_1

* 0708_2

* 0708_3

* 0708_7

* 0708_4

* 0709_1

* 0709_2

* 0709_3

* 0709_4

* 0709_5

* 0709_

* 0709_6

* 0709_7

* 0709_7

* 0709_8

* 0709_9

* 0710_1

* 0710_2

* 0710_2

* 0710_3

* 0710_3

* 0710_3

* 0710_5

* 0710_4

* merge_2

* merge_3

* 0711_1

* 0711_2

* 0711_3

* 0711_4

* 0711_6

* 0711_7

* 0711_8

* 0711_8

* 0711_9

* 0711_10

* 0711-11

* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine

Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353

Reviewed-by: 博惟 <[email protected]>


* add gradient checkpointing

* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP  engine

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .

* 0714_1

* 0714_2

* 0714_3

* 0714_3

* 0714_5

* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration

Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* .
* fix
* .

* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities

Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* .
* fix destroy process group

* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset

Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358

Reviewed-by: 晓雷 <[email protected]>


* .
* .
* .
* .
* fix loss mask
* fix
* .

* 0715_1

* 0715_2

* 0715_2

* 0716_1

* 0716_2

* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub

Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368

Reviewed-by: 晓雷 <[email protected]>


* .
* .

* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation

Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371

Reviewed-by: 晓雷 <[email protected]>


* .

* 0716_3

* 0716_4

* 0716_4

* 0716_5

* 0717_1

* 0717_3

* 0717_3

* 0717_4

* 0717_5

* 0717_6

* 0717_6

* 0717_6

* 0718_2

* 0718_4

* 0718_5

* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher

Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370

Reviewed-by: 博惟 <[email protected]>


* .
* .
* .
* fix

* merge_0721

* 0721_1

* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.

Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392

Reviewed-by: 晓雷 <[email protected]>


* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .

* 0721_2

* 0721_3

* 0721_4

* .

* 0721_formal

* 0721_formal

* 0721_merge4

* 0721_merge5

* 0721_6

* 0721_merge6

* 0721_merge7

* 0721_8

* 0722_1

* 0722_2

* 0722_3

* 0722_4

* 0722_4

* 0722_5

* 0722_6

* 0722_7

* 0723_1

* reformatted

* clang-reformatted

* clang-reformatted2

* 0723_1

* 0723_1

* 0723_1

* 0723_merge3

* 0723_4

* 0723_reformatted_5

* 0724_1

* 0724_1

* 0724_merge1

* 0724_merge2

* 0724_merge3

* 0724_merge3

* 0724_merge4

* 0724_merge5

* 0724_merge6

* 0724_merge7

* 0724_4

* 0724-merge8

* 0724_merge8

* 0725_1

* 0725_6

* 0725_7

* 0725_4padded_image

* 0725_9padded_image

* 0725_10padded_image

* 0725

* 0725_12

* 0725_format

---------

Co-authored-by: bowei.fw <[email protected]>
Co-authored-by: nuzant <[email protected]>
Co-authored-by: 朱晗 <[email protected]>

* 0731

* 0731_1

* 0731_2

* 0731_2

* 0731_3

* 0731_4

* 0801_1

* 0801_2

* 0804_1

* 0804_2

* 0804_2

* 0804_5

* 0805_3

* 0805_2

* 0806

* 0806_merge1

* 0806_merge2

* 0806_format1

* 0806_merge3

* 0806_4

* 0806_6

* 0806_7

* 0806_formatted2

* fix

* revert examples

* .

* .

---------

Co-authored-by: 朱晗 <[email protected]>
Co-authored-by: 博惟 <[email protected]>
Co-authored-by: garrett4wade <[email protected]>
Co-authored-by: Wei Fu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@xssstory @garrett4wade @fishcrap