Skip to content

Commit f4f27ca

Browse files
rentainherentainhe
authored
Add README Template (Oneflow-Inc#98)
* initialize README * add highlights and projects * update README template * Refine README * delete supported project in liBai * refine README * refine README * refine changelog * update README_ch-CN * add citation * refine readme * refine README * refine README * refine word
1 parent e83f05e commit f4f27ca

File tree

4 files changed

+191
-2
lines changed

4 files changed

+191
-2
lines changed

‎CONTRIBUTING.md

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
## Contributing to LiBai

‎README.md

+92-2
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,92 @@
1-
# LiBai
2-
Li Bai : components for large pretrain language models.
1+
<!-- 配图 -->
2+
3+
<h2 align="center">LiBai: Toolbox for Large Scale Pretraining</h2>
4+
<p align="center">
5+
<a href="https://pypi.org/project/LiBai/">
6+
<img alt="PyPI" src="https://img.shields.io/pypi/v/libai">
7+
</a>
8+
<a href="https://libai.readthedocs.io/en/latest/index.html">
9+
<img alt="docs" src="https://img.shields.io/badge/docs-latest-blue">
10+
</a>
11+
<a href="https://github.com/Oneflow-Inc/libai/blob/master/LICENSE">
12+
<img alt="GitHub" src="https://img.shields.io/github/license/Oneflow-Inc/libai.svg?color=blue">
13+
</a>
14+
<a href="https://github.com/Oneflow-Inc/libai/releases">
15+
<img alt="GitHub release" src="https://img.shields.io/github/release/Oneflow-Inc/libai.svg">
16+
</a>
17+
<a href="https://github.com/Oneflow-Inc/libai/issues">
18+
<img alt="PRs Welcome" src="https://img.shields.io/badge/PRs-welcome-pink.svg">
19+
</a>
20+
</p>
21+
22+
<h4 align="center">
23+
<p>
24+
<b>English</b> |
25+
<a href="https://github.com/Oneflow-Inc/libai/blob/main/README_zh-CN.md">简体中文</a>
26+
<p>
27+
</h4>
28+
29+
30+
## Introduction
31+
LiBai is an open-source large-scale model pretraining toolbox with efficient parallelization techniques based on OneFlow. The main branch works with OneFlow 0.7.0.
32+
33+
<details open>
34+
<summary> <b> Highlights </b> </summary>
35+
36+
- **Support a collection of parallel training components**
37+
38+
LiBai provides multiple parallelisms, e.g. Data Parallelism, Tensor Parallelism, and Pipeline Parallelism. Besides, it's extensible for new parallelism.
39+
40+
- **Varied training techniques**
41+
42+
LiBai provides some out-of-the-box training techniques, e.g. Distributed Training, Mixed Precision Training, Activation Checkpointing, Recomputation, Gradient Accumulation, and Zero Redundancy Optimizer(ZeRO).
43+
44+
- **Support for both CV and NLP tasks**
45+
46+
Predifined data process for both CV and NLP datasets, e.g. CIFAR, ImageNet, BERT Dataset.
47+
48+
- **Easy Usage**
49+
50+
Components are designed to be modular in LiBai for better and easier usage as follows:
51+
- LazyConfig system for more flexible syntax and no predefined structures
52+
- Friendly trainer and engine
53+
- Used as a library to support building research projects on top of it. Please see [projects/](/projects) for some projects that are built on top of LiBai.
54+
55+
- **High Efficience**
56+
57+
</details>
58+
59+
## Installation
60+
Please refer to [Getting Started.md](https://libai.readthedocs.io/en/latest/tutorials/Getting%20Started.html) for installation
61+
62+
## Getting Started
63+
Please refer to [Getting Started.md](https://libai.readthedocs.io/en/latest/tutorials/Getting%20Started.html) for basic usage of LiBai
64+
65+
## Documentation
66+
Please refer to [docs](https://libai.readthedocs.io/en/latest/index.html) for full API documentation and tutorials
67+
68+
## ChangeLog
69+
70+
**Beta 0.1.0** was released in 15/2/2022:
71+
- Support 3D parallelism [BERT](https://arxiv.org/abs/1810.04805) models for pretraining.
72+
- Support 2D Parallelism [ViT](https://arxiv.org/abs/2010.11929) models for Image Classification.
73+
74+
Please refer to [changelog.md](./changelog.md) for details and release history.
75+
76+
## Contributing
77+
We appreciate all contributions to improve LiBai. Please refer to [CONTRIBUTING.md](./CONTRIBUTING.md) for the contributing guideline.
78+
79+
## License
80+
This project is released under the [Apache 2.0 license](LICENSE).
81+
82+
## Citation
83+
If you find this project useful for your research, please consider cite:
84+
```BibTeX
85+
@misc{of2021libai,
86+
author = {Xingyu Liao and Peng Cheng and Tianhe Ren and Depeng Liang and
87+
Kai Dang and Yi Wang and Xiaoyu Xu},
88+
title = {LiBai},
89+
howpublished = {\url{https://github.com/Oneflow-Inc/libai}},
90+
year = {2021}
91+
}
92+
```

‎README_zh-CN.md

+92
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
<!-- 配图 -->
2+
3+
<h2 align="center">LiBai: 大规模预训练开源工具箱</h2>
4+
<p align="center">
5+
<a href="https://pypi.org/project/LiBai/">
6+
<img alt="PyPI" src="https://img.shields.io/pypi/v/libai">
7+
</a>
8+
<a href="https://libai.readthedocs.io/en/latest/index.html">
9+
<img alt="docs" src="https://img.shields.io/badge/docs-latest-blue">
10+
</a>
11+
<a href="https://github.com/Oneflow-Inc/libai/blob/master/LICENSE">
12+
<img alt="GitHub" src="https://img.shields.io/github/license/Oneflow-Inc/libai.svg?color=blue">
13+
</a>
14+
<a href="https://github.com/Oneflow-Inc/libai/releases">
15+
<img alt="GitHub release" src="https://img.shields.io/github/release/Oneflow-Inc/libai.svg">
16+
</a>
17+
<a href="https://github.com/Oneflow-Inc/libai/issues">
18+
<img alt="PRs Welcome" src="https://img.shields.io/badge/PRs-welcome-pink.svg">
19+
</a>
20+
</p>
21+
22+
<h4 align="center">
23+
<p>
24+
<a herf="https://github.com/Oneflow-Inc/libai/blob/main/README.md">English</a> |
25+
<b>简体中文</b>
26+
<p>
27+
</h4>
28+
29+
## 简介
30+
LiBai是一个基于OneFlow的大规模预训练开源工具箱, 主分支代码目前支持OneFlow 0.7.0以上的版本。
31+
32+
<details open>
33+
<summary> <b> 主要特性 </b> </summary>
34+
35+
- **支持丰富的并行训练配置**
36+
37+
LiBai 支持了丰富的并行训练配置, 包括数据并行, 模型并行, 流水并行等并行方式, 并且可拓展性好, 易于拓展到更丰富的并行训练模式上。
38+
39+
- **多样化的训练技巧**
40+
41+
LiBai 提供了丰富的开箱即用的训练技巧, 包括但不限于分布式训练, 混合精度训练, 后向重计算, Zero Redundancy Optimizer(ZeRO)等训练方式。
42+
43+
- **同时支持视觉与自然语言处理任务**
44+
45+
LiBai 中内置了CV与NLP相关的数据集处理流程, 包括CIFAR, ImageNet, BERT Dataset等数据集。
46+
47+
- **简单易用，便于上手**
48+
49+
LiBai的模块化设计可以让用户更为方便地将LiBai拓展到自己的项目上:
50+
- 配置系统采用LazyConfig方式, 使得配置系统更加灵活且易于拓展。
51+
- 采用Trainer与Hook结合的方式, 方便用户使用和拓展训练中需要的组件。
52+
- 用户可以在安装好LiBai的基础上灵活地开发自己的任务, 而非强依赖于LiBai中的所有组件。可以查看[基于LiBai的项目](/projects)了解更多细节。
53+
54+
- **速度快，性能高**
55+
56+
</details>
57+
58+
## 安装
59+
请参考[快速入门文档](https://libai.readthedocs.io/en/latest/tutorials/Getting%20Started.html)进行安装。
60+
61+
## 快速入门
62+
请参考[快速入门文档](https://libai.readthedocs.io/en/latest/tutorials/Getting%20Started.html)了解和学习LiBai的基本使用, 后续我们将提供丰富的教程与完整的使用指南。
63+
64+
## 使用文档
65+
请参考[LiBai使用文档](https://libai.readthedocs.io/en/latest/index.html)了解LiBai中相关接口的使用
66+
67+
## 更新日志
68+
69+
最新的**Beta 0.1.0**版本已经在 2022.02.15 发布
70+
- 支持了2D并行ViT模型在ImageNet上的完整训练
71+
- 支持3D并行的BERT模型预训练
72+
73+
历史版本的发布与更新细节请参考[更新日志](./changelog.md)
74+
75+
## 参与贡献
76+
我们欢迎任何有助于提升LiBai的贡献. 请参考[贡献指南](./CONTRIBUTING.md)来了解如何参与贡献
77+
78+
## 许可证
79+
该项目开源自[Apache 2.0 license](LICENSE).
80+
81+
## Citation
82+
如果LiBai对于你的研究项目有帮助的话, 请参考如下的 BibTeX 引用 LiBai:
83+
```BibTeX
84+
@misc{of2021libai,
85+
author = {Xingyu Liao and Peng Cheng and Tianhe Ren and Depeng Liang and
86+
Kai Dang and Yi Wang and Xiaoyu Xu},
87+
title = {LiBai},
88+
howpublished = {\url{https://github.com/Oneflow-Inc/libai}},
89+
year = {2021}
90+
}
91+
```
92+

‎changelog.md

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
## Changelog
2+
3+
### v0.1.0 (15/2/2022)
4+
5+
- Support 3D parallelism [BERT](https://arxiv.org/abs/1810.04805) models for pretraining.
6+
- Support 2D Parallelism [ViT](https://arxiv.org/abs/2010.11929) models for Image Classification.

0 commit comments

Comments
 (0)