Skip to content

Commit

Permalink
Update ReleaseNote3.0.md
Browse files Browse the repository at this point in the history
  • Loading branch information
tastelikefeet authored Dec 4, 2024
1 parent b40f31d commit bd64084
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion docs/source/Instruction/ReleaseNote3.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
## 新功能

1. 数据集模块重构。数据集加载速度提升2~20倍,encode速度提升2~4倍,支持streaming模式
1. 数据集模块重构。数据集加载速度提升2-20倍,encode速度提升2-4倍,支持streaming模式
- 移除了dataset_name机制,采用dataset_id、dataset_dir、dataset_path方式指定数据集
- 使用`--dataset_num_proc`支持多进程加速处理、使用`--load_from_cache_file true`支持使用数据前处理缓存
- 使用`--streaming`支持流式加载hub端和本地数据集
Expand Down

0 comments on commit bd64084

Please sign in to comment.