[data] feat: TransferQueue - An asynchronous streaming data management system#3649

Merged

wuxibin89 merged 52 commits intoverl-project:mainfrom

TransferQueue:main_tq_submodule

Oct 21, 2025

Commits on Sep 23, 2025

Commits on Sep 28, 2025

Support controller in TransferQueue (#2)

authored and
0oshowero0
committed
expose TransferQueueClient (#3)

ji-huazhong
authored and
0oshowero0
committed
Add copyright and license information
0oshowero0
committed
update client docstring (#5)
0oshowero0
committed
merge TransferQueue utils (#4)

zhabuye
authored and
0oshowero0
committed
[fix] Fix n_sample related problems (#8)
0oshowero0
committed
expose TransferQueue client/controller UT (#6)

zhabuye
authored and
0oshowero0
committed
Add metadata.py and test_simple_storage_unit.py (#9)

committed
Add reorder function to BatchMeta (#13)

authored and
0oshowero0
committed
Merge remote-tracking branch 'upstream/main' into main_tq_submodule
0oshowero0
committed
Add TransferQueue as submodule under experimental/transfer_queue
0oshowero0
committed
update requirements for transfer_queue
0oshowero0
committed
update doc
0oshowero0
committed

Commits on Sep 29, 2025

Commits on Sep 30, 2025

Commits on Oct 9, 2025

fix (#26)

baymax591
and
ji-huazhong
authored

Commits on Oct 10, 2025

Commits on Oct 11, 2025

fix: correct batch size assignment in _dataproto_to_tensordict function (#28)
jianjunzhong
authored

Commits on Oct 13, 2025

Commits on Oct 15, 2025

add TODOs for TransferQueue (#37)

LLLLxmmm
and
liuximeng
authored
apply review suggestions (#38)
ji-huazhong
authored

Commits on Oct 16, 2025

Commits on Oct 17, 2025

Adapt for multimodal data format (#42)

LLLLxmmm
and
liuximeng
authored

Commits on Oct 18, 2025

Commits on Oct 20, 2025

simpify the implementation of main_ppo in recipe/transfer_queue
ji-huazhong
committed